HACA3: A unified approach for multi-site MR image harmonization

User requirements for quantitative radiological reports in multiple sclerosis

OBJECTIVES

Quantitative radiological reports (QReports) can enhance clinical management of multiple sclerosis (MS) by including quantitative data from MRI scans. However, the lack of consensus on the specific information to include, on and clinicians' preferences, hinders the adoption of these imaging analysis tools. This study aims to facilitate the clinical implementation of QReports by determining clinicians' requirements regarding their use in MS management.

MATERIALS AND METHODS

A four-phase Delphi panel approach was employed, involving neurologists and (neuro)radiologists across Europe. Initial interviews with experts helped develop a questionnaire addressing various QReport aspects. This questionnaire underwent refinement based on feedback and was distributed through the MAGNIMS network. A second questionnaire, incorporating additional questions, was circulated following a plenary discussion at the MAGNIMS workshop in Milan in November 2023. Responses from both questionnaire iterations were collected and analyzed, with adjustments made based on participant feedback.

RESULTS

The study achieved a 49.6% response rate, involving 78 respondents. Key preferences and barriers to QReport adoption were identified, highlighting the importance of integration into clinical workflows, cost-effectiveness, educational support for interpretation, and validation standards. Strong consensus emerged on including detailed lesion information and specific brain and spinal cord volume measurements. Concerns regarding report generation time, data protection, and reliability were also raised.

CONCLUSION

While QReports show potential for improving MS management, incorporation of the key metrics and addressing the identified barriers related to cost, validation, integration, and clinician education is crucial for practical implementation. These recommendations for developers to refine QReports could enhance their utility and adoption in clinical practice.

KEY POINTS

Question A lack of consensus on essential features for quantitative magnetic resonance imaging reports limits their integration into multiple sclerosis management. Findings This study identified key preferences, including detailed lesion information, specific brain and spinal cord measurements, and rigorous validation for effective quantitative reports. Clinical relevance This study identified essential features and barriers for implementing quantitative radiological reports in multiple sclerosis management, aiming to enhance clinical workflows, improve disease monitoring, and ultimately provide better, data-driven care for patients through tailored imaging solutions.

Harmonizing flows: Leveraging normalizing flows for unsupervised and source-free MRI harmonization

Lack of standardization and various intrinsic parameters for magnetic resonance (MR) image acquisition results in heterogeneous images across different sites and devices, which adversely affects the generalization of deep neural networks. To alleviate this issue, this work proposes a novel unsupervised harmonization framework that leverages normalizing flows to align MR images, thereby emulating the distribution of a source domain. The proposed strategy comprises three key steps. Initially, a normalizing flow network is trained to capture the distribution characteristics of the source domain. Then, we train a shallow harmonizer network to reconstruct images from the source domain via their augmented counterparts. Finally, during inference, the harmonizer network is updated to ensure that the output images conform to the learned source domain distribution, as modeled by the normalizing flow network. Our approach, which is unsupervised, source-free, and task-agnostic is assessed in the context of both adults and neonatal cross-domain brain MRI segmentation, as well as neonatal brain age estimation, demonstrating its generalizability across tasks and population demographics. The results underscore its superior performance compared to existing methodologies. The code is available at https://github.com/farzad-bz/Harmonizing-Flows.

Disentangled latent energy-based style translation: An image-level structural MRI harmonization framework

Brain magnetic resonance imaging (MRI) has been extensively employed across clinical and research fields, but often exhibits sensitivity to site effects arising from non-biological variations such as differences in field strength and scanner vendors. Numerous retrospective MRI harmonization techniques have demonstrated encouraging outcomes in reducing the site effects at image level. However, existing methods generally suffer from high computational requirements and limited generalizability, restricting their applicability to unseen MRIs. In this paper, we design a novel disentangled latent energy-based style translation (DLEST) framework for unpaired image-level MRI harmonization, consisting of (a) site-invariant image generation (SIG), (b) site-specific style translation (SST), and (c) site-specific MRI synthesis (SMS). Specifically, the SIG employs a latent autoencoder to encode MRIs into a low-dimensional latent space and reconstruct MRIs from latent codes. The SST utilizes an energy-based model to comprehend global latent distribution of a target domain and translate source latent codes towards the target domain, while SMS enables MRI synthesis with a target-specific style. By disentangling image generation and style translation in latent space, the DLEST can achieve efficient style translation. Our model was trained on T1-weighted MRIs from a public dataset (with 3,984 subjects across 58 acquisition sites/settings) and validated on an independent dataset (with 9 traveling subjects scanned in 11 sites/settings) in four tasks: histogram and feature visualization, site classification, brain tissue segmentation, and site-specific structural MRI synthesis. Qualitative and quantitative results demonstrate the superiority of our method over several state-of-the-arts.

The Psychosis MRI Shared Data Resource (Psy-ShareD)

Neuroimaging research in the field of schizophrenia and other psychotic disorders has sought to investigate neuroanatomical markers, relative to healthy control groups. In recent decades, a large number of structural magnetic resonance imaging (MRI) studies have been funded and undertaken, but their small sample sizes and heterogeneous methods have led to inconsistencies across findings. To tackle this, efforts have been made to combine datasets across studies and sites. While notable recent multicentre initiatives and the resulting meta- and mega-analytical outputs have progressed the field, efforts have generally been restricted to MRI scans in one or two illness stages, often overlook patient heterogeneity, and study populations have rarely been globally representative of the diversity of patients who experience psychosis. Furthermore, access to these datasets is often restricted to consortia members who can contribute data, likely from research institutions located in high-income countries. The Psychosis MRI Shared Data Resource (Psy-ShareD) is a new open access structural MRI data sharing partnership that will host pre-existing structural T1-weighted MRI data collected across multiple sites worldwide, including the Global South. MRI T1 data included in Psy-ShareD will be available in image and feature-level formats, having been harmonised using state-of-the-art approaches. All T1 data will be linked to demographic and illness-related (diagnosis, symptoms, medication status) measures, and in a number of datasets, IQ and cognitive data, and medication history will also be available, allowing subgroup and dimensional analyses. Psy-ShareD will be free-to-access for all researchers. Importantly, comprehensive data catalogues, scientific support and training resources will be available to facilitate use by early career researchers and build capacity in the field. We are actively seeking new collaborators to contribute further T1 data. Collaborators will benefit in terms of authorships, as all publications arising from Psy-ShareD will include data contributors as authors.

Within-person changes in objectively measured activity levels as a predictor of brain atrophy in multiple sclerosis

Objective

The evaluate the association between changes in accelerometry-derived activity patterns and brain atrophy in people with multiple sclerosis (PwMS).

Methods

We included PwMS aged ≥40 years with approximately annual brain MRI who wore GT9X Actigraph accelerometers every three months over two years. Functional principal components analysis (fPCA) summarized overall activity and timing. Additional indices included total and 2-hour specific activity, sedentary time, and circadian rhythm parameters. Whole brain segmentation used SLANT-CRUISE. Generalized estimating equations quantifying between- and within-person effects modeled associations between accelerometry changes and MRI outcomes, adjusting for demographic and clinical factors.

Results

We included 233 PwMS (mean age: 54.4 years, SD: 8.6, 30% male) who wore accelerometers an average of 6.3 times over 58 days across two years. fPCA showed within-person increases in the first fPC, representing low nighttime and high morning activity, were associated with slower brain atrophy (per 1 SD increase: 0.24%; 95% CI: 0.10, 0.40; p=0.0009). Similarly, a 10% increase in 8:00-10:00 AM activity was associated with 0.49% higher whole brain volume (95% CI: 0.19, 0.79; p=0.001) over time, while increased nighttime activity (0:00-2:00) was linked to -0.28% brain volume loss (95% CI: -0.48, -0.08; p=0.007). Higher moderate-to-vigorous activity and daytime activity were associated with greater brain volume preservation longitudinally.

Interpretation

Changes in activity patterns, particularly increased nighttime activity and reduced morning activity, are linked to brain atrophy in PwMS. Accelerometry offers a scalable, sensitive method for tracking MS progression and may be beneficial as a recruitment or outcome measure in trials.

Brain atrophy assessment in multiple sclerosis: technical- and subject-related barriers for translation to real-world application in individual subjects

INTRODUCTION

Brain atrophy is a well-established MRI outcome for predicting clinical progression and monitoring treatment response in persons with multiple sclerosis (pwMS) at the group level. Despite the important progress made, the translation of brain atrophy assessment into clinical practice faces several challenges.

AREAS COVERED

In this review, the authors discuss technical- and subject-related barriers for implementing brain atrophy assessment as part of the clinical routine at the individual level. Substantial progress has been made to understand and mitigate technical barriers behind MRI acquisition. Numerous research and commercial segmentation techniques for volume estimation are available and technically validated, but their clinical value has not been fully established. A systematic assessment of subject-related barriers, which include genetic, environmental, biological, lifestyle, comorbidity, and aging confounders, is critical for the interpretation of brain atrophy measures at the individual subject level. Educating both medical providers and pwMS will help better clarify the benefits and limitations of assessing brain atrophy for disease monitoring and prognosis.

EXPERT OPINION

Integrating brain atrophy assessment into clinical practice for pwMS requires overcoming technical and subject-related challenges. Advances in MRI standardization, artificial intelligence, and clinician education will facilitate this process, improving disease management and potentially reducing long-term healthcare costs.

OpenMAP-T1: A Rapid Deep-Learning Approach to Parcellate 280 Anatomical Regions to Cover the Whole Brain

This study introduces OpenMAP-T1, a deep-learning-based method for rapid and accurate whole-brain parcellation in T1- weighted brain MRI, which aims to overcome the limitations of conventional normalization-to-atlas-based approaches and multi-atlas label-fusion (MALF) techniques. Brain image parcellation is a fundamental process in neuroscientific and clinical research, enabling a detailed analysis of specific cerebral regions. Normalization-to-atlas-based methods have been employed for this task, but they face limitations due to variations in brain morphology, especially in pathological conditions. The MALF techniques improved the accuracy of the image parcellation and robustness to variations in brain morphology, but at the cost of high computational demand that requires a lengthy processing time. OpenMAP-T1 integrates several convolutional neural network models across six phases: preprocessing; cropping; skull-stripping; parcellation; hemisphere segmentation; and final merging. This process involves standardizing MRI images, isolating the brain tissue, and parcellating it into 280 anatomical structures that cover the whole brain, including detailed gray and white matter structures, while simplifying the parcellation processes and incorporating robust training to handle various scan types and conditions. The OpenMAP-T1 was validated on the Johns Hopkins University atlas library and eight available open resources, including real-world clinical images, and the demonstration of robustness across different datasets with variations in scanner types, magnetic field strengths, and image processing techniques, such as defacing. Compared with existing methods, OpenMAP-T1 significantly reduced the processing time per image from several hours to less than 90 s without compromising accuracy. It was particularly effective in handling images with intensity inhomogeneity and varying head positions, conditions commonly seen in clinical settings. The adaptability of OpenMAP-T1 to a wide range of MRI datasets and its robustness to various scan conditions highlight its potential as a versatile tool in neuroimaging.

Scanner-specific optimisation of automated lesion segmentation in MS

BACKGROUND & OBJECTIVE

Automatic lesion segmentation techniques on MRI scans of people with multiple sclerosis (pwMS) could support lesion detection and segmentation in trials and clinical practice. However, knowledge on their reliability across scanners is limited, hampering clinical implementation. The aim of this study was to investigate the within-scanner repeatability and between-scanner reproducibility of lesion segmentation tools in pwMS across three different scanners and examine their accuracy compared to manual segmentations with and without optimization.

METHODS

30 pwMS underwent a scan and rescan on three MRI scanners. GE Discovery MR750 (3.0 T), Siemens Sola (1.5 T) and Siemens Vida (3.0 T)). 3D-FLuid Attenuated Inversion Recovery (3D-FLAIR) and 3D T1-weighted scans were acquired on each scanner. Lesion segmentation involved preprocessing and automatic segmentation using the Lesion Segmentation Toolbox (LST) and nicMSlesions (nicMS) as well as manual segmentation. Both automated segmentation techniques were used with default settings, and with settings optimized to match manual segmentations for each scanner specifically and combined for the three scanners. LST settings were optimized by adjusting the threshold to improve the Dice similarity coefficient (DSC) for each scanner separately and a combined threshold for all scanners. For nicMS the last layers were retrained, once with the multi-scanner data to represent a combined optimization and once separately for each scanner for scanner specific optimization. Volumes and counts were extracted. DSC was calculated for accuracy, and reliability was assessed using intra-class correlation coefficients (ICC). Differences in DSC between software was tested with a repeated measures ANOVA and when appropriate post-hoc paired t-tests using Bonferroni correction.

RESULTS

Scanner-specific optimization significantly improved DSC for LST compared to default and combined settings, except for the GE scanner. NicMS showed significantly higher DSC for both the scanner-specific and combined optimization than default. Within-scanner repeatability was excellent (ICC>0.9) for volume and counts. Between-scanner ICC for volume between Vida and Sola was higher (0.94-0.99) than between GE MR750 and Vida or Sola (0.18-0.93), with improved ICCs for nicMS scanner-specific (0.87-0.93) compared to others (0.18-0.79). This was not present for Sola vs. Vida where all ICCs were excellent (>0.94).

CONCLUSION

Scanner-specific optimization strategies proved effective in mitigating inter-scanner variability, addressing the issue of insufficient reproducibility and accuracy found with default settings.

SiMix: A domain generalization method for cross-site brain MRI harmonization via site mixing

Brain magnetic resonance imaging (MRI) is widely used in clinical practice for disease diagnosis. However, MRI scans acquired at different sites can have different appearances due to the difference in the hardware, pulse sequence, and imaging parameter. It is important to reduce or eliminate such cross-site variations with brain MRI harmonization so that downstream image processing and analysis is performed consistently. Previous works on the harmonization problem require the data acquired from the sites of interest for model training. But in real-world scenarios there can be test data from a new site of interest after the model is trained, and training data from the new site is unavailable when the model is trained. In this case, previous methods cannot optimally handle the test data from the new unseen site. To address the problem, in this work we explore domain generalization for brain MRI harmonization and propose Site Mix (SiMix). We assume that images of travelling subjects are acquired at a few existing sites for model training. To allow the training data to better represent the test data from unseen sites, we first propose to mix the training images belonging to different sites stochastically, which substantially increases the diversity of the training data while preserving the authenticity of the mixed training images. Second, at test time, when a test image from an unseen site is given, we propose a multiview strategy that perturbs the test image with preserved authenticity and ensembles the harmonization results of the perturbed images for improved harmonization quality. To validate SiMix, we performed experiments on the publicly available SRPBS dataset and MUSHAC dataset that comprised brain MRI acquired at nine and two different sites, respectively. The results indicate that SiMix improves brain MRI harmonization for unseen sites, and it is also beneficial to the harmonization of existing sites.

Deep learning for the harmonization of structural MRI scans: a survey

Medical imaging datasets for research are frequently collected from multiple imaging centers using different scanners, protocols, and settings. These variations affect data consistency and compatibility across different sources. Image harmonization is a critical step to mitigate the effects of factors like inherent differences between various vendors, hardware upgrades, protocol changes, and scanner calibration drift, as well as to ensure consistent data for medical image processing techniques. Given the critical importance and widespread relevance of this issue, a vast array of image harmonization methodologies have emerged, with deep learning-based approaches driving substantial advancements in recent times. The goal of this review paper is to examine the latest deep learning techniques employed for image harmonization by analyzing cutting-edge architectural approaches in the field of medical image harmonization, evaluating both their strengths and limitations. This paper begins by providing a comprehensive fundamental overview of image harmonization strategies, covering three critical aspects: established imaging datasets, commonly used evaluation metrics, and characteristics of different scanners. Subsequently, this paper analyzes recent structural MRI (Magnetic Resonance Imaging) harmonization techniques based on network architecture, network learning algorithm, network supervision strategy, and network output. The underlying architectures include U-Net, Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), flow-based generative models, transformer-based approaches, as well as custom-designed network architectures. This paper investigates the effectiveness of Disentangled Representation Learning (DRL) as a pivotal learning algorithm in harmonization. Lastly, the review highlights the primary limitations in harmonization techniques, specifically the lack of comprehensive quantitative comparisons across different methods. The overall aim of this review is to serve as a guide for researchers and practitioners to select appropriate architectures based on their specific conditions and requirements. It also aims to foster discussions around ongoing challenges in the field and shed light on promising future research directions with the potential for significant advancements.

Exploring approaches to tackle cross-domain challenges in brain medical image segmentation: a systematic review

Introduction

Brain medical image segmentation is a critical task in medical image processing, playing a significant role in the prediction and diagnosis of diseases such as stroke, Alzheimer's disease, and brain tumors. However, substantial distribution discrepancies among datasets from different sources arise due to the large inter-site discrepancy among different scanners, imaging protocols, and populations. This leads to cross-domain problems in practical applications. In recent years, numerous studies have been conducted to address the cross-domain problem in brain image segmentation.

Methods

This review adheres to the standards of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for data processing and analysis. We retrieved relevant papers from PubMed, Web of Science, and IEEE databases from January 2018 to December 2023, extracting information about the medical domain, imaging modalities, methods for addressing cross-domain issues, experimental designs, and datasets from the selected papers. Moreover, we compared the performance of methods in stroke lesion segmentation, white matter segmentation and brain tumor segmentation.

Results

A total of 71 studies were included and analyzed in this review. The methods for tackling the cross-domain problem include Transfer Learning, Normalization, Unsupervised Learning, Transformer models, and Convolutional Neural Networks (CNNs). On the ATLAS dataset, domain-adaptive methods showed an overall improvement of ~3 percent in stroke lesion segmentation tasks compared to non-adaptive methods. However, given the diversity of datasets and experimental methodologies in current studies based on the methods for white matter segmentation tasks in MICCAI 2017 and those for brain tumor segmentation tasks in BraTS, it is challenging to intuitively compare the strengths and weaknesses of these methods.

Conclusion

Although various techniques have been applied to address the cross-domain problem in brain image segmentation, there is currently a lack of unified dataset collections and experimental standards. For instance, many studies are still based on n-fold cross-validation, while methods directly based on cross-validation across sites or datasets are relatively scarce. Furthermore, due to the diverse types of medical images in the field of brain segmentation, it is not straightforward to make simple and intuitive comparisons of performance. These challenges need to be addressed in future research.

Cross-site prognosis prediction for nasopharyngeal carcinoma from incomplete multi-modal data

Accurate prognosis prediction for nasopharyngeal carcinoma based on magnetic resonance (MR) images assists in the guidance of treatment intensity, thus reducing the risk of recurrence and death. To reduce repeated labor and sufficiently explore domain knowledge, aggregating labeled/annotated data from external sites enables us to train an intelligent model for a clinical site with unlabeled data. However, this task suffers from the challenges of incomplete multi-modal examination data fusion and image data heterogeneity among sites. This paper proposes a cross-site survival analysis method for prognosis prediction of nasopharyngeal carcinoma from domain adaptation viewpoint. Utilizing a Cox model as the basic framework, our method equips it with a cross-attention based multi-modal fusion regularization. This regularization model effectively fuses the multi-modal information from multi-parametric MR images and clinical features onto a domain-adaptive space, despite the absence of some modalities. To enhance the feature discrimination, we also extend the contrastive learning technique to censored data cases. Compared with the conventional approaches which directly deploy a trained survival model in a new site, our method achieves superior prognosis prediction performance in cross-site validation experiments. These results highlight the key role of cross-site adaptability of our method and support its value in clinical practice.

Image harmonization improves consistency of intra-rater delineations of MS lesions in heterogeneous MRI

Clinical magnetic resonance images (MRIs) lack a standard intensity scale due to differences in scanner hardware and the pulse sequences used to acquire the images. When MRIs are used for quantification, as in the evaluation of white matter lesions (WMLs) in multiple sclerosis, this lack of intensity standardization becomes a critical problem affecting both the staging and tracking of the disease and its treatment. This paper presents a study of harmonization on WML segmentation consistency, which is evaluated using an object detection classification scheme that incorporates manual delineations from both the original and harmonized MRIs. A cohort of ten people scanned on two different imaging platforms was studied. An expert rater, blinded to the image source, manually delineated WMLs on images from both scanners before and after harmonization. It was found that there is closer agreement in both global and per-lesion WML volume and spatial distribution after harmonization, demonstrating the importance of image harmonization prior to the creation of manual delineations. These results could lead to better truth models in both the development and evaluation of automated lesion segmentation algorithms.