MRI quality control for the Italian Neuroimaging Network Initiative: moving towards big data in multiple sclerosis

Automatic quality control of brain 3D FLAIR MRIs for a clinical data warehouse

Clinical data warehouses, which have arisen over the last decade, bring together the medical data of millions of patients and offer the potential to train and validate machine learning models in real-world scenarios. The quality of MRIs collected in clinical data warehouses differs significantly from that generally observed in research datasets, reflecting the variability inherent to clinical practice. Consequently, the use of clinical data requires the implementation of robust quality control tools. By using a substantial number of pre-existing manually labelled T1-weighted MR images (5,500) alongside a smaller set of newly labelled FLAIR images (926), we present a novel semi-supervised adversarial domain adaptation architecture designed to exploit shared representations between MRI sequences thanks to a shared feature extractor, while taking into account the specificities of the FLAIR thanks to a specific classification head for each sequence. This architecture thus consists of a common invariant feature extractor, a domain classifier and two classification heads specific to the source and target, all designed to effectively deal with potential class distribution shifts between the source and target data classes. The primary objectives of this paper were: (1) to identify images which are not proper 3D FLAIR brain MRIs; (2) to rate the overall image quality. For the first objective, our approach demonstrated excellent results, with a balanced accuracy of 89%, comparable to that of human raters. For the second objective, our approach achieved good performance, although lower than that of human raters. Nevertheless, the automatic approach accurately identified bad quality images (balanced accuracy >79%). In conclusion, our proposed approach overcomes the initial barrier of heterogeneous image quality in clinical data warehouses, thereby facilitating the development of new research using clinical routine 3D FLAIR brain images.

Regional hippocampal atrophy reflects memory impairment in patients with early relapsing remitting multiple sclerosis

BACKGROUND

Research work has shown that hippocampal subfields are atrophic to varying extents in multiple sclerosis (MS) patients. However, studies examining the functional implications of subfield-specific hippocampal damage in early MS are limited. We aim to gain insights into the relationship between hippocampal atrophy and memory function by investigating the correlation between global and regional hippocampal atrophy and memory performance in early MS patients.

METHODS

From the Italian Neuroimaging Network Initiative (INNI) dataset, we selected 3D-T1-weighted brain MRIs of 219 early relapsing remitting (RR)MS and 246 healthy controls (HC) to identify hippocampal atrophic areas. At the time of MRI, patients underwent Selective-Reminding-Test (SRT) and Spatial-Recall-Test (SPART) and were classified as mildly (MMI-MS: n.110) or severely (SMI-MS: n:109) memory impaired, according to recently proposed cognitive phenotypes.

RESULTS

Early RRMS showed lower hippocampal volumes compared to HC (p < 0.001), while these did not differ between MMI-MS and SMI-MS. In MMI-MS, lower hippocampal volumes correlated with worse memory tests (r = 0.23-0.37, p ≤ 0.01). Atrophic voxels were diffuse in the hippocampus but more prevalent in cornu ammonis (CA, 79%) than in tail (21%). In MMI-MS, decreased subfield volumes correlated with decreases in memory, particularly in the right CA1 (SRT-recall: r = 0.38; SPART: r = 0.34, p < 0.01). No correlations were found in the SMI-MS group.

CONCLUSION

Hippocampal atrophy spreads from CA to tail from early disease stages. Subfield hippocampal atrophy is associated with memory impairment in MMI-MS, while this correlation is lost in SMI-MS. This plays in favor of a limited capacity for an adaptive functional reorganization of the hippocampi in MS patients.

The brief repeatable battery of neuropsychological tests (BRB-N) version a: update of Italian normative data from the Italian Neuroimaging Network Initiative (INNI)

BACKGROUND

Cognitive impairment is a common clinical manifestation in people with multiple sclerosis (PwMS) and significantly impacts patients' quality life. Cognitive assessment is crucial for treatment decisions and understanding disease progression. Several neuropsychological batteries are used in MS, including the Brief Repeatable Battery of Neuropsychological Tests (BRB-N), Minimal Assessment of Cognitive Function in MS (MACFIMS), and Brief International Cognitive Assessment for MS (BICAMS). However, normative data for BRB-N version A in Italy are outdated.

OBJECTIVES

To revise and update normative data for the BRB-N version A in the Italian population.

METHODS

From the Italian Neuroimaging Network Initiative (INNI) database, we retrospectively selected 342 healthy subjects (172 males and 170 females) evaluated at four Italian INNI-affiliated sites (Milan, Siena, Rome, Naples). The subjects underwent neuropsychological assessment using the BRB-N version A. Regression-based method relying on scaled scores was used to calculate demographic correction procedures.

RESULTS

No significant differences were found in age, education, and sex distribution among the four sites (p ≥ 0.055). Regression analysis provided normative data to calculate demographically adjusted z-scores for each BRB-N version A test.

DISCUSSION

This study provides updated normative data for the BRB-N version A in the Italian population. The use of a regression-based method and scaled scores ensures consistency with other neuropsychological batteries commonly used in Italy, namely MACFIMS and BICAMS. The availability of updated normative data increases reliability of neuropsychological assessment of cognitive function in Italian PwMS and other clinical populations using BRB-N version A, providing valuable insights for both clinical and research applications.

Quantification of Thalamic Atrophy in MS: From the Multicenter Italian Neuroimaging Network Initiative Data Set to Clinical Application

BACKGROUND AND PURPOSE

Thalamic atrophy occurs from the earliest phases of MS; however, this measure is not included in clinical practice. Our purpose was to obtain a reliable segmentation of the thalamus in MS by comparing existing automatic methods cross-sectionally and longitudinally.

MATERIALS AND METHODS

MR images of 141 patients with relapsing-remitting MS (mean age, 38 years; range, 19-58 years; 95 women) and 69 healthy controls (mean age, 36 years; range, 22-69 years; 47 women) were retrieved from the Italian Neuroimaging Network Initiative repository: T1WI, T2WI, and DWI at baseline and after 1 year (136 patients, 31 healthy controls). Three segmentation software programs (FSL-FIRST, FSL-MIST, FreeSurfer) were compared. At baseline, agreement among pipelines, correlations with age, disease duration, clinical score, and T2-hyperintense lesion volume were evaluated. Effect sizes in differentiating patients and controls were assessed cross-sectionally and longitudinally. Variability of longitudinal changes in controls and sample sizes were assessed. False discovery rate-adjusted P < .05 was considered significant.

RESULTS

At baseline, FSL-FIRST and FSL-MIST showed the highest agreement in the results of thalamic volume (R = 0.87, P < .001), with the highest effect size for FSL-MIST (Cohen d = 1.11); correlations with demographic and clinical variables were comparable for all software. Longitudinally, FSL-MIST showed the lowest variability in estimating thalamic volume changes for healthy controls (SD = 1.07%), the highest effect size (Cohen d = 0.44), and the smallest sample size at 80% power level (15 subjects per group).

CONCLUSIONS

Multimodal segmentation by FSL-MIST increased the robustness of the results with better capability to detect small variations in thalamic volumes.

Methods for Brain Atrophy MR Quantification in Multiple Sclerosis: Application to the Multicenter INNI Dataset

BACKGROUND

Current therapeutic strategies in multiple sclerosis (MS) target neurodegeneration. However, the integration of atrophy measures into the clinical scenario is still an unmet need.

PURPOSE

To compare methods for whole-brain and gray matter (GM) atrophy measurements using the Italian Neuroimaging Network Initiative (INNI) dataset.

STUDY TYPE

Retrospective (data available from INNI).

POPULATION

A total of 466 patients with relapsing-remitting MS (mean age = 37.3 ± 10 years, 323 women) and 279 healthy controls (HC; mean age = 38.2 ± 13 years, 164 women).

FIELD STRENGTH/SEQUENCE

A 3.0-T, T1-weighted (spin echo and gradient echo without gadolinium injection) and T2-weighted spin echo scans at baseline and after 1 year (170 MS, 48 HC).

ASSESSMENT

Structural Image Evaluation using Normalization of Atrophy (SIENA-X/XL; version 5.0.9), Statistical Parametric Mapping (SPM-v12); and Jim-v8 (Xinapse Systems, Colchester, UK) software were applied to all subjects.

STATISTICAL TESTS

In MS and HC, we evaluated the intraclass correlation coefficient (ICC) among FSL-SIENA(XL), SPM-v12, and Jim-v8 for cross-sectional whole-brain and GM tissue volumes and their longitudinal changes, the effect size according to the Cohen's d at baseline and the sample size requirement for whole-brain and GM atrophy progression at different power levels (lowest = 0.7, 0.05 alpha level). False discovery rate (Benjamini-Hochberg procedure) correction was applied. A P value <0.05 was considered statistically significant.

RESULTS

SPM-v12 and Jim-v8 showed significant agreement for cross-sectional whole-brain (ICC = 0.93 for HC and ICC = 0.84 for MS) and GM volumes (ICC = 0.66 for HC and ICC = 0.90) and longitudinal assessment of GM atrophy (ICC = 0.35 for HC and ICC = 0.59 for MS), while no significant agreement was found in the comparisons between whole-brain and GM volumes for SIENA-X/XL and both SPM-v12 (P = 0.19 and P = 0.29, respectively) and Jim-v8 (P = 0.21 and P = 0.32, respectively). SPM-v12 and Jim-v8 showed the highest effect size for cross-sectional GM atrophy (Cohen's d = -0.63 and -0.61). Jim-v8 and SIENA(XL) showed the smallest sample size requirements for whole-brain (58) and GM atrophy (152), at 0.7 power level.

DATA CONCLUSION

The findings obtained in this study should be considered when selecting the appropriate brain atrophy pipeline for MS studies.

EVIDENCE LEVEL

4.

TECHNICAL EFFICACY

Stage 1.

Functional connectivity MRI quality control procedures in CONN

Quality control (QC) for functional connectivity magnetic resonance imaging (FC-MRI) is critical to ensure the validity of neuroimaging studies. Noise confounds are common in MRI data and, if not accounted for, may introduce biases in functional measures affecting the validity, replicability, and interpretation of FC-MRI study results. Although FC-MRI analysis rests on the assumption of adequate data processing, QC is underutilized and not systematically reported. Here, we describe a quality control pipeline for the visual and automated evaluation of MRI data implemented as part of the CONN toolbox. We analyzed publicly available resting state MRI data (N = 139 from 7 MRI sites) from the FMRI Open QC Project. Preprocessing steps included realignment, unwarp, normalization, segmentation, outlier identification, and smoothing. Data denoising was performed based on the combination of scrubbing, motion regression, and aCompCor - a principal component characterization of noise from minimally eroded masks of white matter and of cerebrospinal fluid tissues. Participant-level QC procedures included visual inspection of raw-level data and of representative images after each preprocessing step for each run, as well as the computation of automated descriptive QC measures such as average framewise displacement, average global signal change, prevalence of outlier scans, MNI to anatomical and functional overlap, anatomical to functional overlap, residual BOLD timeseries variability, effective degrees of freedom, and global correlation strength. Dataset-level QC procedures included the evaluation of inter-subject variability in the distributions of edge connectivity in a 1,000-node graph (FC distribution displays), and the estimation of residual associations across participants between functional connectivity strength and potential noise indicators such as participant's head motion and prevalence of outlier scans (QC-FC analyses). QC procedures are demonstrated on the reference dataset with an emphasis on visualization, and general recommendations for best practices are discussed in the context of functional connectivity and other fMRI analysis. We hope this work contributes toward the dissemination and standardization of QC testing performance reporting among peers and in scientific journals.

Resting-state functional MRI in multicenter studies on multiple sclerosis: a report on raw data quality and functional connectivity features from the Italian Neuroimaging Network Initiative

The Italian Neuroimaging Network Initiative (INNI) is an expanding repository of brain MRI data from multiple sclerosis (MS) patients recruited at four Italian MRI research sites. We describe the raw data quality of resting-state functional MRI (RS-fMRI) time-series in INNI and the inter-site variability in functional connectivity (FC) features after unified automated data preprocessing. MRI datasets from 489 MS patients and 246 healthy control (HC) subjects were retrieved from the INNI database. Raw data quality metrics included temporal signal-to-noise ratio (tSNR), spatial smoothness (FWHM), framewise displacement (FD), and differential variation in signals (DVARS). Automated preprocessing integrated white-matter lesion segmentation (SAMSEG) into a standard fMRI pipeline (fMRIPrep). FC features were calculated on pre-processed data and harmonized between sites (Combat) prior to assessing general MS-related alterations. Across centers (both groups), median tSNR and FWHM ranged from 47 to 84 and from 2.0 to 2.5, and median FD and DVARS ranged from 0.08 to 0.24 and from 1.06 to 1.22. After preprocessing, only global FC-related features were significantly correlated with FD or DVARS. Across large-scale networks, age/sex/FD-adjusted and harmonized FC features exhibited both inter-site and site-specific inter-group effects. Significant general reductions were obtained for somatomotor and limbic networks in MS patients (vs. HC). The implemented procedures provide technical information on raw data quality and outcome of fully automated preprocessing that might serve as reference in future RS-fMRI studies within INNI. The unified pipeline introduced little bias across sites and appears suitable for multisite FC analyses on harmonized network estimates.

Multicenter data harmonization for regional brain atrophy and application in multiple sclerosis

BACKGROUND

In multiple sclerosis (MS), determination of regional brain atrophy is clinically relevant. However, analysis of large datasets is rare because of the increased variability in multicenter data.

PURPOSE

To compare different methods to correct for center effects. To investigate regional gray matter (GM) volume in relapsing-remitting MS in a large multicenter dataset.

METHODS

MRI scans of 466 MS patients and 279 healthy controls (HC) were retrieved from the Italian Neuroimaging Network Initiative repository. Voxel-based morphometry was performed. The center effect was accounted for with different methods: (a) no correction, (b) factor in the statistical model, (c) ComBat method and (d) subsampling procedure to match single-center distributions. By applying the best correction method, GM atrophy was assessed in MS patients vs HC and according to clinical disability, disease duration and T₂ lesion volume. Results were assessed voxel-wise using general linear model.

RESULTS

The average residuals for the harmonization methods were 5.03 (a), 4.42 (b), 4.26 (c) and 2.98 (d). The comparison between MS patients and HC identified thalami and other deep GM nuclei, the cerebellum and several cortical regions. At single-center analysis, the thalami were always involved, whereas different other regions were found in each center. Cerebellar atrophy correlated with clinical disability, while deep GM nuclei atrophy correlated with T₂-lesion volume.

CONCLUSION

Harmonization based on subsampling more effectively decreased the residuals of the statistical model applied. In comparison with findings from single-center analysis, the multicenter results were more robust, highlighting the importance of data repositories from multiple centers.

Prediction of the information processing speed performance in multiple sclerosis using a machine learning approach in a large multicenter magnetic resonance imaging data set

Many patients with multiple sclerosis (MS) experience information processing speed (IPS) deficits, and the Symbol Digit Modalities Test (SDMT) has been recommended as a valid screening test. Magnetic resonance imaging (MRI) has markedly improved the understanding of the mechanisms associated with cognitive deficits in MS. However, which structural MRI markers are the most closely related to cognitive performance is still unclear. We used the multicenter 3T-MRI data set of the Italian Neuroimaging Network Initiative to extract multimodal data (i.e., demographic, clinical, neuropsychological, and structural MRIs) of 540 MS patients. We aimed to assess, through machine learning techniques, the contribution of brain MRI structural volumes in the prediction of IPS deficits when combined with demographic and clinical features. We trained and tested the eXtreme Gradient Boosting (XGBoost) model following a rigorous validation scheme to obtain reliable generalization performance. We carried out a classification and a regression task based on SDMT scores feeding each model with different combinations of features. For the classification task, the model trained with thalamus, cortical gray matter, hippocampus, and lesions volumes achieved an area under the receiver operating characteristic curve of 0.74. For the regression task, the model trained with cortical gray matter and thalamus volumes, EDSS, nucleus accumbens, lesions, and putamen volumes, and age reached a mean absolute error of 0.95. In conclusion, our results confirmed that damage to cortical gray matter and relevant deep and archaic gray matter structures, such as the thalamus and hippocampus, is among the most relevant predictors of cognitive performance in MS.

MAGNIMS recommendations for harmonization of MRI data in MS multicenter studies

•

Sharing data from cooperative studies is essential to develop new biomarkers in MS.

•

Differences in MRI acquisition, analysis, storage represent a substantial constraint.

•

We review the state of the art and developments in the harmonization of MRI.

•

We provide recommendations to harmonize large MRI datasets in the MS field.

There is an increasing need of sharing harmonized data from large, cooperative studies as this is essential to develop new diagnostic and prognostic biomarkers. In the field of multiple sclerosis (MS), the issue has become of paramount importance due to the need to translate into the clinical setting some of the most recent MRI achievements. However, differences in MRI acquisition parameters, image analysis and data storage across sites, with their potential bias, represent a substantial constraint. This review focuses on the state of the art, recent technical advances, and desirable future developments of the harmonization of acquisition, analysis and storage of large-scale multicentre MRI data of MS cohorts. Huge efforts are currently being made to achieve all the requirements needed to provide harmonized MRI datasets in the MS field, as proper management of large imaging datasets is one of our greatest opportunities and challenges in the coming years. Recommendations based on these achievements will be provided here. Despite the advances that have been made, the complexity of these tasks requires further research by specialized academical centres, with dedicated technical and human resources. Such collective efforts involving different professional figures are of crucial importance to offer to MS patients a personalised management while minimizing consumption of resources.

Cerebellar pathology and disability worsening in relapsing-remitting multiple sclerosis: A retrospective analysis from the CombiRx trial

BACKGROUND AND PURPOSE

Cerebellar damage is a valuable predictor of disability, particularly in progressive multiple sclerosis. It is not clear if it could be an equally useful predictor of motor disability worsening in the relapsing-remitting phenotype.

AIM

We aimed to determine whether cerebellar damage is an equally useful predictor of motor disability worsening in the relapsing-remitting phenotype.

METHODS

Cerebellar lesion loads and volumes were estimated using baseline magnetic resonance imaging from the CombiRx trial (n = 838). The relationship between cerebellar damage and time to disability worsening (confirmed disability progression [CDP], timed 25-foot walk test [T25FWT] score worsening, nine-hole peg test [9HPT] score worsening) was tested in stagewise and stepwise Cox proportional hazards models, accounting for demographics and supratentorial damage.

RESULTS

Shorter time to 9HPT score worsening was associated with higher baseline Expanded Disability Status Scale (EDSS) score (hazard ratio [HR] 1.408, p = 0.0042) and higher volume of supratentorial and cerebellar T2 lesions (HR 1.005 p = 0.0196 and HR 2.211, p = 0.0002, respectively). Shorter time to T25FWT score worsening was associated with higher baseline EDSS (HR 1.232, p = 0.0006). Shorter time to CDP was associated with older age (HR 1.026, p = 0.0010), lower baseline EDSS score (HR 0.428, p < 0.0001) and higher volume of supratentorial T2 lesions (HR 1.024, p < 0.0001).

CONCLUSION

Among the explored outcomes, single time-point evaluation of cerebellar damage only allows the prediction of manual dexterity worsening. In clinical studies the selection of imaging biomarkers should be informed by the outcome of interest.

Fully automated quality control of rigid and affine registrations of T1w and T2w MRI in big data using machine learning

BACKGROUND

Magnetic resonance imaging (MRI)-based morphometry and relaxometry are proven methods for the structural assessment of the human brain in several neurological disorders. These procedures are generally based on T1-weighted (T1w) and/or T2-weighted (T2w) MRI scans, and rigid and affine registrations to a standard template(s) are essential steps in such studies. Therefore, a fully automatic quality control (QC) of these registrations is necessary in big data scenarios to ensure that they are suitable for subsequent processing.

METHOD

A supervised machine learning (ML) framework is proposed by computing similarity metrics such as normalized cross-correlation, normalized mutual information, and correlation ratio locally. We have used these as candidate features for cross-validation and testing of different ML classifiers. For 5-fold repeated stratified grid search cross-validation, 400 correctly aligned, 2000 randomly generated misaligned images were used from the human connectome project young adult (HCP-YA) dataset. To test the cross-validated models, the datasets from autism brain imaging data exchange (ABIDE I) and information eXtraction from images (IXI) were used.

RESULTS

The ensemble classifiers, random forest, and AdaBoost yielded best performance with F1-scores, balanced accuracies, and Matthews correlation coefficients in the range of 0.95-1.00 during cross-validation. The predictive accuracies reached 0.99 on the Test set #1 (ABIDE I), 0.99 without and 0.96 with noise on Test set #2 (IXI, stratified w.r.t scanner vendor and field strength).

CONCLUSIONS

The cross-validated and tested ML models could be used for QC of both T1w and T2w rigid and affine registrations in large-scale MRI studies.

Effects of Ibudilast on MRI Measures in the Phase 2 SPRINT-MS Study

OBJECTIVE

To determine whether ibudilast has an effect on brain volume and new lesions in progressive forms of multiple sclerosis (MS).

METHODS

A randomized, placebo-controlled, blinded study evaluated ibudilast at a dose of up to 100 mg over 96 weeks in primary and secondary progressive MS. In this secondary analysis of a previously reported trial, secondary and tertiary endpoints included gray matter atrophy, new or enlarging T2 lesions as measured every 24 weeks, and new T1 hypointensities at 96 weeks. Whole brain atrophy measured by structural image evaluation, using normalization, of atrophy (SIENA) was a sensitivity analysis.

RESULTS

A total of 129 participants were assigned to ibudilast and 126 to placebo. New or enlarging T2 lesions were observed in 37.2% on ibudilast and 29.0% on placebo (p = 0.82). New T1 hypointense lesions at 96 weeks were observed in 33.3% on ibudilast and 23.5% on placebo (p = 0.11). Gray matter atrophy was reduced by 35% for those on ibudilast vs placebo (p = 0.038). Progression of whole brain atrophy by SIENA was slowed by 20% in the ibudilast group compared with placebo (p = 0.08).

CONCLUSION

Ibudilast treatment was associated with a reduction in gray matter atrophy. Ibudilast treatment was not associated with a reduction in new or enlarging T2 lesions or new T1 lesions. An effect on brain volume contributes to prior data that ibudilast appears to affect markers associated with neurodegenerative processes, but not inflammatory processes.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that for people with MS, ibudilast does not significantly reduce new or enlarging T2 lesions or new T1 lesions.

Data Collection in Multiple Sclerosis: The MSDS Approach

Multiple sclerosis (MS) is a frequent chronic inflammatory disease of the central nervous system that affects patients over decades. As the monitoring and treatment of MS become more personalized and complex, the individual assessment and collection of different parameters ranging from clinical assessments via laboratory and imaging data to patient-reported data become increasingly important for innovative patient management in MS. These aspects predestine electronic data processing for use in MS documentation. Such technologies enable the rapid exchange of health information between patients, practitioners, and caregivers, regardless of time and location. In this perspective paper, we present our digital strategy from Dresden, where we are developing the Multiple Sclerosis Documentation System (MSDS) into an eHealth platform that can be used for multiple purposes. Various use cases are presented that implement this software platform and offer an important perspective for the innovative digital patient management in the future. A holistic patient management of the MS, electronically supported by clinical pathways, will have an important impact on other areas of patient care, such as neurorehabilitation.