Multi-site fMRI analysis using privacy-preserving federated learning and domain adaptation: ABIDE results

BrainDAS: Structure-aware domain adaptation network for multi-site brain network analysis

In the medical field, datasets are mostly integrated across sites due to difficult data acquisition and insufficient data at a single site. The domain shift problem caused by the heterogeneous distribution among multi-site data makes autism spectrum disorder (ASD) hard to identify. Recently, domain adaptation has received considerable attention as a promising solution. However, domain adaptation on graph data like brain networks has not been fully studied. It faces two major challenges: (1) complex graph structure; and (2) multiple source domains. To overcome the issues, we propose an end-to-end structure-aware domain adaptation framework for brain network analysis (BrainDAS) using resting-state functional magnetic resonance imaging (rs-fMRI). The proposed approach contains two stages: supervision-guided multi-site graph domain adaptation with dynamic kernel generation and graph classification with attention-based graph pooling. We evaluate our BrainDAS on a public dataset provided by Autism Brain Imaging Data Exchange (ABIDE) which includes 871 subjects from 17 different sites, surpassing state-of-the-art algorithms in several different evaluation settings. Furthermore, our promising results demonstrate the interpretability and generalization of the proposed method. Our code is available at https://github.com/songruoxian/BrainDAS.

Federated learning for medical image analysis: A survey

Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.

Diagnosing schizophrenia using deep learning: Novel interpretation approaches and multi-site validation

Schizophrenia is a profound and enduring mental disorder that imposes significant negative impacts on individuals, their families, and society at large. The development of more accurate and objective diagnostic tools for schizophrenia can be expedited through the employment of deep learning (DL), that excels at deciphering complex hierarchical non-linear patterns. However, the limited interpretability of deep learning has eroded confidence in the model and restricted its clinical utility. At the same time, if the data source is only derived from a single center, the model's generalizability is difficult to test. To enhance the model's reliability and applicability, leave-one-center-out validation with a large and diverse sample from multiple centers is crucial. In this study, we utilized Nine different global centers to train and test the 3D Resnet model's generalizability, resulting in an 82% classification performance (area under the curve) on all datasets sourced from different countries, employing a leave-one-center-out-validation approach. Per our approximation of the feature significance of each region on the atlas, we identified marked differences in the thalamus, pallidum, and inferior frontal gyrus between individuals with schizophrenia and healthy controls, lending credence to prior research findings. At the same time, in order to translate the model's output into clinically applicable insights, the SHapley Additive exPlanations (SHAP) permutation explainer method with an anatomical atlas have been refined, thereby offering precise neuroanatomical and functional interpretations of different brain regions.

Gradient Matching Federated Domain Adaptation for Brain Image Classification

Federated learning has shown its unique advantages in many different tasks, including brain image analysis. It provides a new way to train deep learning models while protecting the privacy of medical image data from multiple sites. However, previous studies suggest that domain shift across different sites may influence the performance of federated models. As a solution, we propose a gradient matching federated domain adaptation (GM-FedDA) method for brain image classification, aiming to reduce domain discrepancy with the assistance of a public image dataset and train robust local federated models for target sites. It mainly includes two stages: 1) pretraining stage; we propose a one-common-source adversarial domain adaptation (OCS-ADA) strategy, i.e., adopting ADA with gradient matching loss to pretrain encoders for reducing domain shift at each target site (private data) with the assistance of a common source domain (public data) and 2) fine-tuning stage; we develop a gradient matching federated (GM-Fed) fine-tuning method for updating local federated models pretrained with the OCS-ADA strategy, i.e., pushing the optimization direction of a local federated model toward its specific local minimum by minimizing gradient matching loss between sites. Using fully connected networks as local models, we validate our method with the diagnostic classification tasks of schizophrenia and major depressive disorder based on multisite resting-state functional MRI (fMRI), respectively. Results show that the proposed GM-FedDA method outperforms other commonly used methods, suggesting the potential of our method in brain imaging analysis and other fields, which need to utilize multisite data while preserving data privacy.

Parkinson's Disease Recognition Using Decorrelated Convolutional Neural Networks: Addressing Imbalance and Scanner Bias in rs-fMRI Data

Parkinson's disease (PD) is a neurodegenerative and progressive disease that impacts the nerve cells in the brain and varies from person to person. The exact cause of PD is still unknown, and the diagnosis of PD does not include a specific objective test with certainty. Although deep learning has made great progress in medical neuroimaging analysis, these methods are very susceptible to biases present in neuroimaging datasets. An innovative decorrelated deep learning technique is introduced to mitigate class bias and scanner bias while simultaneously focusing on finding distinguishing characteristics in resting-state functional MRI (rs-fMRI) data, which assists in recognizing PD with good accuracy. The decorrelation function reduces the nonlinear correlation between features and bias in order to learn bias-invariant features. The publicly available Parkinson's Progression Markers Initiative (PPMI) dataset, referred to as a single-scanner imbalanced dataset in this study, was used to validate our method. The imbalanced dataset problem affects the performance of the deep learning framework by overfitting to the majority class. To resolve this problem, we propose a new decorrelated convolutional neural network (DcCNN) framework by applying decorrelation-based optimization to convolutional neural networks (CNNs). An analysis of evaluation metrics comparisons shows that integrating the decorrelation function boosts the performance of PD recognition by removing class bias. Specifically, our DcCNN models perform significantly better than existing traditional approaches to tackle the imbalance problem. Finally, the same framework can be extended to create scanner-invariant features without significantly impacting the performance of a model. The obtained dataset is a multiscanner dataset, which leads to scanner bias due to the differences in acquisition protocols and scanners. The multiscanner dataset is a combination of two publicly available datasets, namely, PPMI and FTLDNI-the frontotemporal lobar degeneration neuroimaging initiative (NIFD) dataset. The results of t-distributed stochastic neighbor embedding (t-SNE) and scanner classification accuracy of our proposed feature extraction-DcCNN (FE-DcCNN) model validated the effective removal of scanner bias. Our method achieves an average accuracy of 77.80% on a multiscanner dataset for differentiating PD from a healthy control, which is superior to the DcCNN model trained on a single-scanner imbalanced dataset.

Efficient Federated Learning for distributed NeuroImaging Data

Recent advancements in neuroimaging have led to greater data sharing among the scientific community. However, institutions frequently maintain control over their data, citing concerns related to research culture, privacy, and accountability. This creates a demand for innovative tools capable of analyzing amalgamated datasets without the need to transfer actual data between entities. To address this challenge, we propose a decentralized sparse federated learning (FL) strategy. This approach emphasizes local training of sparse models to facilitate efficient communication within such frameworks. By capitalizing on model sparsity and selectively sharing parameters between client sites during the training phase, our method significantly lowers communication overheads. This advantage becomes increasingly pronounced when dealing with larger models and accommodating the diverse resource capabilities of various sites. We demonstrate the effectiveness of our approach through the application to the Adolescent Brain Cognitive Development (ABCD) dataset.

Identifying autism spectrum disorder from multi-modal data with privacy-preserving

The application of deep learning models to precision medical diagnosis often requires the aggregation of large amounts of medical data to effectively train high-quality models. However, data privacy protection mechanisms make it difficult to perform medical data collection from different medical institutions. In autism spectrum disorder (ASD) diagnosis, automatic diagnosis using multimodal information from heterogeneous data has not yet achieved satisfactory performance. To address the privacy preservation issue as well as to improve ASD diagnosis, we propose a deep learning framework using multimodal feature fusion and hypergraph neural networks for disease prediction in federated learning (FedHNN). By introducing the federated learning strategy, each local model is trained and computed independently in a distributed manner without data sharing, allowing rapid scaling of medical datasets to achieve robust and scalable deep learning predictive models. To further improve the performance with privacy preservation, we improve the hypergraph model for multimodal fusion to make it suitable for autism spectrum disorder (ASD) diagnosis tasks by capturing the complementarity and correlation between modalities through a hypergraph fusion strategy. The results demonstrate that our proposed federated learning-based prediction model is superior to all local models and outperforms other deep learning models. Overall, our proposed FedHNN has good results in the work of using multi-site data to improve the performance of ASD identification.

Dynamic graph transformer network via dual-view connectivity for autism spectrum disorder identification

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that requires objective and accurate identification methods for effective early intervention. Previous population-based methods via functional connectivity (FC) analysis ignore the differences between positive and negative FCs, which provide the potential information complementarity. And they also require additional information to construct a pre-defined graph. Meanwhile, two challenging demand attentions are the imbalance of performance caused by the class distribution and the inherent heterogeneity of multi-site data. In this paper, we propose a novel dynamic graph Transformer network based on dual-view connectivity for ASD Identification. It is based on the Autoencoders, which regard the input feature as individual feature and without any inductive bias. First, a dual-view feature extractor is designed to extract individual and complementary information from positive and negative connectivity. Then Graph Transformer network is innovated with a hot plugging K-Nearest Neighbor (KNN) algorithm module which constructs a dynamic population graph without any additional information. Additionally, we introduce the PolyLoss function and the Vrex method to address the class imbalance and improve the model's generalizability. The evaluation experiment on 1102 subjects from the ABIDE I dataset demonstrates our method can achieve superior performance over several state-of-the-art methods and satisfying generalizability for ASD identification.

One model to unite them all: Personalized federated learning of multi-contrast MRI synthesis

Curation of large, diverse MRI datasets via multi-institutional collaborations can help improve learning of generalizable synthesis models that reliably translate source- onto target-contrast images. To facilitate collaborations, federated learning (FL) adopts decentralized model training while mitigating privacy concerns by avoiding sharing of imaging data. However, conventional FL methods can be impaired by the inherent heterogeneity in the data distribution, with domain shifts evident within and across imaging sites. Here we introduce the first personalized FL method for MRI Synthesis (pFLSynth) that improves reliability against data heterogeneity via model specialization to individual sites and synthesis tasks (i.e., source-target contrasts). To do this, pFLSynth leverages an adversarial model equipped with novel personalization blocks that control the statistics of generated feature maps across the spatial/channel dimensions, given latent variables specific to sites and tasks. To further promote communication efficiency and site specialization, partial network aggregation is employed over later generator stages while earlier generator stages and the discriminator are trained locally. As such, pFLSynth enables multi-task training of multi-site synthesis models with high generalization performance across sites and tasks. Comprehensive experiments demonstrate the superior performance and reliability of pFLSynth in MRI synthesis against prior federated methods.

A comparison between centralized and asynchronous federated learning approaches for survival outcome prediction using clinical and PET data from non-small cell lung cancer patients

BACKGROUND AND OBJECTIVE

Survival analysis plays an essential role in the medical field for optimal treatment decision-making. Recently, survival analysis based on the deep learning (DL) approach has been proposed and is demonstrating promising results. However, developing an ideal prediction model requires integrating large datasets across multiple institutions, which poses challenges concerning medical data privacy.

METHODS

In this paper, we propose FedSurv, an asynchronous federated learning (FL) framework designed to predict survival time using clinical information and positron emission tomography (PET)-based features. This study used two datasets: a public radiogenic dataset of non-small cell lung cancer (NSCLC) from the Cancer Imaging Archive (RNSCLC), and an in-house dataset from the Chonnam National University Hwasun Hospital (CNUHH) in South Korea, consisting of clinical risk factors and F-18 fluorodeoxyglucose (FDG) PET images in NSCLC patients. Initially, each dataset was divided into multiple clients according to histological attributes, and each client was trained using the proposed DL model to predict individual survival time. The FL framework collected weights and parameters from the clients, which were then incorporated into the global model. Finally, the global model aggregated all weights and parameters and redistributed the updated model weights to each client. We evaluated different frameworks including single-client-based approach, centralized learning and FL.

RESULTS

We evaluated our method on two independent datasets. First, on the RNSCLC dataset, the mean absolute error (MAE) was 490.80±22.95 d and the C-Index was 0.69±0.01. Second, on the CNUHH dataset, the MAE was 494.25±40.16 d and the C-Index was 0.71±0.01. The FL approach achieved centralized method performance in PET-based survival time prediction and outperformed single-client-based approaches.

CONCLUSIONS

Our results demonstrated the feasibility and effectiveness of employing FL for individual survival prediction in NSCLC patients, using clinical information and PET-based features.

Multi-kernel Learning Fusion Algorithm Based on RNN and GRU for ASD Diagnosis and Pathogenic Brain Region Extraction

Autism spectrum disorder (ASD) is a complex, severe disorder related to brain development. It impairs patient language communication and social behaviors. In recent years, ASD researches have focused on a single-modal neuroimaging data, neglecting the complementarity between multi-modal data. This omission may lead to poor classification. Therefore, it is important to study multi-modal data of ASD for revealing its pathogenesis. Furthermore, recurrent neural network (RNN) and gated recurrent unit (GRU) are effective for sequence data processing. In this paper, we introduce a novel framework for a Multi-Kernel Learning Fusion algorithm based on RNN and GRU (MKLF-RAG). The framework utilizes RNN and GRU to provide feature selection for data of different modalities. Then these features are fused by MKLF algorithm to detect the pathological mechanisms of ASD and extract the most relevant the Regions of Interest (ROIs) for the disease. The MKLF-RAG proposed in this paper has been tested in a variety of experiments with the Autism Brain Imaging Data Exchange (ABIDE) database. Experimental findings indicate that our framework notably enhances the classification accuracy for ASD. Compared with other methods, MKLF-RAG demonstrates superior efficacy across multiple evaluation metrics and could provide valuable insights into the early diagnosis of ASD.

Artificial Intelligence in Surgical Research: Accomplishments and Future Directions

MINI-ABSTRACT

The study introduces various methods of performing conventional ML and their implementation in surgical areas, and the need to move beyond these traditional approaches given the advent of big data.

OBJECTIVE

Investigate current understanding and future directions of machine learning applications, such as risk stratification, clinical data analytics, and decision support, in surgical practice.

SUMMARY BACKGROUND DATA

The advent of the electronic health record, near unlimited computing, and open-source computational packages have created an environment for applying artificial intelligence, machine learning, and predictive analytic techniques to healthcare. The "hype" phase has passed, and algorithmic approaches are being developed for surgery patients through all stages of care, involving preoperative, intraoperative, and postoperative components. Surgeons must understand and critically evaluate the strengths and weaknesses of these methodologies.

METHODS

The current body of AI literature was reviewed, emphasizing on contemporary approaches important in the surgical realm.

RESULTS AND CONCLUSIONS

The unrealized impacts of AI on clinical surgery and its subspecialties are immense. As this technology continues to pervade surgical literature and clinical applications, knowledge of its inner workings and shortcomings is paramount in determining its appropriate implementation.

Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives

Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.

Efficacy of federated learning on genomic data: a study on the UK Biobank and the 1000 Genomes Project

Combining training data from multiple sources increases sample size and reduces confounding, leading to more accurate and less biased machine learning models. In healthcare, however, direct pooling of data is often not allowed by data custodians who are accountable for minimizing the exposure of sensitive information. Federated learning offers a promising solution to this problem by training a model in a decentralized manner thus reducing the risks of data leakage. Although there is increasing utilization of federated learning on clinical data, its efficacy on individual-level genomic data has not been studied. This study lays the groundwork for the adoption of federated learning for genomic data by investigating its applicability in two scenarios: phenotype prediction on the UK Biobank data and ancestry prediction on the 1000 Genomes Project data. We show that federated models trained on data split into independent nodes achieve performance close to centralized models, even in the presence of significant inter-node heterogeneity. Additionally, we investigate how federated model accuracy is affected by communication frequency and suggest approaches to reduce computational complexity or communication costs.

Automated diagnosis of autism with artificial intelligence: State of the art

Autism spectrum disorder (ASD) represents a panel of conditions that begin during the developmental period and result in impairments of personal, social, academic, or occupational functioning. Early diagnosis is directly related to a better prognosis. Unfortunately, the diagnosis of ASD requires a long and exhausting subjective process. We aimed to review the state of the art for automated autism diagnosis and recognition in this research. In February 2022, we searched multiple databases and sources of gray literature for eligible studies. We used an adapted version of the QUADAS-2 tool to assess the risk of bias in the studies. A brief report of the methods and results of each study is presented. Data were synthesized for each modality separately using the Split Component Synthesis (SCS) method. We assessed heterogeneity using the I 2 statistics and evaluated publication bias using trim and fill tests combined with ln DOR. Confidence in cumulative evidence was assessed using the GRADE approach for diagnostic studies. We included 344 studies from 186,020 participants (51,129 are estimated to be unique) for nine different modalities in this review, from which 232 reported sufficient data for meta-analysis. The area under the curve was in the range of 0.71-0.90 for all the modalities. The studies on EEG data provided the best accuracy, with the area under the curve ranging between 0.85 and 0.93. We found that the literature is rife with bias and methodological/reporting flaws. Recommendations are provided for future research to provide better studies and fill in the current knowledge gaps.

A multi-center distributed learning approach for Parkinson's disease classification using the traveling model paradigm

Distributed learning is a promising alternative to central learning for machine learning (ML) model training, overcoming data-sharing problems in healthcare. Previous studies exploring federated learning (FL) or the traveling model (TM) setup for medical image-based disease classification often relied on large databases with a limited number of centers or simulated artificial centers, raising doubts about real-world applicability. This study develops and evaluates a convolution neural network (CNN) for Parkinson's disease classification using data acquired by 83 diverse real centers around the world, mostly contributing small training samples. Our approach specifically makes use of the TM setup, which has proven effective in scenarios with limited data availability but has never been used for image-based disease classification. Our findings reveal that TM is effective for training CNN models, even in complex real-world scenarios with variable data distributions. After sufficient training cycles, the TM-trained CNN matches or slightly surpasses the performance of the centrally trained counterpart (AUROC of 83% vs. 80%). Our study highlights, for the first time, the effectiveness of TM in 3D medical image classification, especially in scenarios with limited training samples and heterogeneous distributed data. These insights are relevant for situations where ML models are supposed to be trained using data from small or remote medical centers, and rare diseases with sparse cases. The simplicity of this approach enables a broad application to many deep learning tasks, enhancing its clinical utility across various contexts and medical facilities.

A Systematic Review and Identification of the Challenges of Deep Learning Techniques for Undersampled Magnetic Resonance Image Reconstruction

Deep learning (DL) in magnetic resonance imaging (MRI) shows excellent performance in image reconstruction from undersampled k-space data. Artifact-free and high-quality MRI reconstruction is essential for ensuring accurate diagnosis, supporting clinical decision-making, enhancing patient safety, facilitating efficient workflows, and contributing to the validity of research studies and clinical trials. Recently, deep learning has demonstrated several advantages over conventional MRI reconstruction methods. Conventional methods rely on manual feature engineering to capture complex patterns and are usually computationally demanding due to their iterative nature. Conversely, DL methods use neural networks with hundreds of thousands of parameters and automatically learn relevant features and representations directly from the data. Nevertheless, there are some limitations to DL-based techniques concerning MRI reconstruction tasks, such as the need for large, labeled datasets, the possibility of overfitting, and the complexity of model training. Researchers are striving to develop DL models that are more efficient, adaptable, and capable of providing valuable information for medical practitioners. We provide a comprehensive overview of the current developments and clinical uses by focusing on state-of-the-art DL architectures and tools used in MRI reconstruction. This study has three objectives. Our main objective is to describe how various DL designs have changed over time and talk about cutting-edge tactics, including their advantages and disadvantages. Hence, data pre- and post-processing approaches are assessed using publicly available MRI datasets and source codes. Secondly, this work aims to provide an extensive overview of the ongoing research on transformers and deep convolutional neural networks for rapid MRI reconstruction. Thirdly, we discuss several network training strategies, like supervised, unsupervised, transfer learning, and federated learning for rapid and efficient MRI reconstruction. Consequently, this article provides significant resources for future improvement of MRI data pre-processing and fast image reconstruction.

Unified fair federated learning for digital healthcare

Federated learning (FL) is a promising approach for healthcare institutions to train high-quality medical models collaboratively while protecting sensitive data privacy. However, FL models encounter fairness issues at diverse levels, leading to performance disparities across different subpopulations. To address this, we propose Federated Learning with Unified Fairness Objective (FedUFO), a unified framework consolidating diverse fairness levels within FL. By leveraging distributionally robust optimization and a unified uncertainty set, it ensures consistent performance across all subpopulations and enhances the overall efficacy of FL in healthcare and other domains while maintaining accuracy levels comparable with those of existing methods. Our model was validated by applying it to four digital healthcare tasks using real-world datasets in federated settings. Our collaborative machine learning paradigm not only promotes artificial intelligence in digital healthcare but also fosters social equity by embodying fairness.

Survey of Medical Applications of Federated Learning

OBJECTIVES

Medical artificial intelligence (AI) has recently attracted considerable attention. However, training medical AI models is challenging due to privacy-protection regulations. Among the proposed solutions, federated learning (FL) stands out. FL involves transmitting only model parameters without sharing the original data, making it particularly suitable for the medical field, where data privacy is paramount. This study reviews the application of FL in the medical domain.

METHODS

We conducted a literature search using the keywords "federated learning" in combination with "medical," "healthcare," or "clinical" on Google Scholar and PubMed. After reviewing titles and abstracts, 58 papers were selected for analysis. These FL studies were categorized based on the types of data used, the target disease, the use of open datasets, the local model of FL, and the neural network model. We also examined issues related to heterogeneity and security.

RESULTS

In the investigated FL studies, the most commonly used data type was image data, and the most studied target diseases were cancer and COVID-19. The majority of studies utilized open datasets. Furthermore, 72% of the FL articles addressed heterogeneity issues, while 50% discussed security concerns.

CONCLUSIONS

FL in the medical domain appears to be in its early stages, with most research using open data and focusing on specific data types and diseases for performance verification purposes. Nonetheless, medical FL research is anticipated to be increasingly applied and to become a vital component of multi-institutional research.

Preserving specificity in federated graph learning for fMRI-based neurological disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) offers a non-invasive approach to examining abnormal brain connectivity associated with brain disorders. Graph neural network (GNN) gains popularity in fMRI representation learning and brain disorder analysis with powerful graph representation capabilities. Training a general GNN often necessitates a large-scale dataset from multiple imaging centers/sites, but centralizing multi-site data generally faces inherent challenges related to data privacy, security, and storage burden. Federated Learning (FL) enables collaborative model training without centralized multi-site fMRI data. Unfortunately, previous FL approaches for fMRI analysis often ignore site-specificity, including demographic factors such as age, gender, and education level. To this end, we propose a specificity-aware federated graph learning (SFGL) framework for rs-fMRI analysis and automated brain disorder identification, with a server and multiple clients/sites for federated model aggregation and prediction. At each client, our model consists of a shared and a personalized branch, where parameters of the shared branch are sent to the server while those of the personalized branch remain local. This can facilitate knowledge sharing among sites and also helps preserve site specificity. In the shared branch, we employ a spatio-temporal attention graph isomorphism network to learn dynamic fMRI representations. In the personalized branch, we integrate vectorized demographic information (i.e., age, gender, and education years) and functional connectivity networks to preserve site-specific characteristics. Representations generated by the two branches are then fused for classification. Experimental results on two fMRI datasets with a total of 1218 subjects suggest that SFGL outperforms several state-of-the-art approaches.

On the challenges and perspectives of foundation models for medical image analysis

This article discusses the opportunities, applications and future directions of large-scale pretrained models, i.e., foundation models, which promise to significantly improve the analysis of medical images. Medical foundation models have immense potential in solving a wide range of downstream tasks, as they can help to accelerate the development of accurate and robust models, reduce the dependence on large amounts of labeled data, preserve the privacy and confidentiality of patient data. Specifically, we illustrate the "spectrum" of medical foundation models, ranging from general imaging models, modality-specific models, to organ/task-specific models, and highlight their challenges, opportunities and applications. We also discuss how foundation models can be leveraged in downstream medical tasks to enhance the accuracy and efficiency of medical image analysis, leading to more precise diagnosis and treatment decisions.

Cybersecurity in neural interfaces: Survey and future trends

With the joint advancement in areas such as pervasive neural data sensing, neural computing, neuromodulation and artificial intelligence, neural interface has become a promising technology facilitating both the closed-loop neurorehabilitation for neurologically impaired patients and the intelligent man-machine interactions for general application purposes. However, although neural interface has been widely studied, few previous studies focused on the cybersecurity issues in related applications. In this survey, we systematically investigated possible cybersecurity risks in neural interfaces, together with potential solutions to these problems. Importantly, our survey considers interfacing techniques on both central nervous systems (i.e., brain-computer interfaces) and peripheral nervous systems (i.e., general neural interfaces), covering diverse neural modalities such as electroencephalography, electromyography and more. Moreover, our survey is organized on three different levels: (1) the data level, which mainly focuses on the privacy leakage issue via attacking and analyzing neural database of users; (2) the permission level, which mainly focuses on the prospects and risks to directly use real time neural signals as biometrics for continuous and unobtrusive user identity verification; and (3) the model level, which mainly focuses on adversarial attacks and defenses on both the forward neural decoding models (e.g. via machine learning) and the backward feedback implementation models (e.g. via neuromodulation and stimulation). This is the first study to systematically investigate cybersecurity risks and possible solutions in neural interfaces which covers both central and peripheral nervous systems, and considers multiple different levels to provide a complete picture of this issue.

Unsupervised contrastive graph learning for resting-state functional MRI analysis and brain disorder detection

Resting-state functional magnetic resonance imaging (rs-fMRI) helps characterize regional interactions that occur in the human brain at a resting state. Existing research often attempts to explore fMRI biomarkers that best predict brain disease progression using machine/deep learning techniques. Previous fMRI studies have shown that learning-based methods usually require a large amount of labeled training data, limiting their utility in clinical practice where annotating data is often time-consuming and labor-intensive. To this end, we propose an unsupervised contrastive graph learning (UCGL) framework for fMRI-based brain disease analysis, in which a pretext model is designed to generate informative fMRI representations using unlabeled training data, followed by model fine-tuning to perform downstream disease identification tasks. Specifically, in the pretext model, we first design a bi-level fMRI augmentation strategy to increase the sample size by augmenting blood-oxygen-level-dependent (BOLD) signals, and then employ two parallel graph convolutional networks for fMRI feature extraction in an unsupervised contrastive learning manner. This pretext model can be optimized on large-scale fMRI datasets, without requiring labeled training data. This model is further fine-tuned on to-be-analyzed fMRI data for downstream disease detection in a task-oriented learning manner. We evaluate the proposed method on three rs-fMRI datasets for cross-site and cross-dataset learning tasks. Experimental results suggest that the UCGL outperforms several state-of-the-art approaches in automated diagnosis of three brain diseases (i.e., major depressive disorder, autism spectrum disorder, and Alzheimer's disease) with rs-fMRI data.

Federated Domain Adaptation via Transformer for Multi-Site Alzheimer's Disease Diagnosis

In multi-site studies of Alzheimer's disease (AD), the difference of data in multi-site datasets leads to the degraded performance of models in the target sites. The traditional domain adaptation method requires sharing data from both source and target domains, which will lead to data privacy issue. To solve it, federated learning is adopted as it can allow models to be trained with multi-site data in a privacy-protected manner. In this paper, we propose a multi-site federated domain adaptation framework via Transformer (FedDAvT), which not only protects data privacy, but also eliminates data heterogeneity. The Transformer network is used as the backbone network to extract the correlation between the multi-template region of interest features, which can capture the brain abundant information. The self-attention maps in the source and target domains are aligned by applying mean squared error for subdomain adaptation. Finally, we evaluate our method on the multi-site databases based on three AD datasets. The experimental results show that the proposed FedDAvT is quite effective, achieving accuracy rates of 88.75%, 69.51%, and 69.88% on the AD vs. NC, MCI vs. NC, and AD vs. MCI two-way classification tasks, respectively.

The Journey of Data Within a Global Data Sharing Initiative: A Federated 3-Layer Data Analysis Pipeline to Scale Up Multiple Sclerosis Research

BACKGROUND

Investigating low-prevalence diseases such as multiple sclerosis is challenging because of the rather small number of individuals affected by this disease and the scattering of real-world data across numerous data sources. These obstacles impair data integration, standardization, and analysis, which negatively impact the generation of significant meaningful clinical evidence.

OBJECTIVE

This study aims to present a comprehensive, research question-agnostic, multistakeholder-driven end-to-end data analysis pipeline that accommodates 3 prevalent data-sharing streams: individual data sharing, core data set sharing, and federated model sharing.

METHODS

A demand-driven methodology is employed for standardization, followed by 3 streams of data acquisition, a data quality enhancement process, a data integration procedure, and a concluding analysis stage to fulfill real-world data-sharing requirements. This pipeline's effectiveness was demonstrated through its successful implementation in the COVID-19 and multiple sclerosis global data sharing initiative.

RESULTS

The global data sharing initiative yielded multiple scientific publications and provided extensive worldwide guidance for the community with multiple sclerosis. The pipeline facilitated gathering pertinent data from various sources, accommodating distinct sharing streams and assimilating them into a unified data set for subsequent statistical analysis or secure data examination. This pipeline contributed to the assembly of the largest data set of people with multiple sclerosis infected with COVID-19.

CONCLUSIONS

The proposed data analysis pipeline exemplifies the potential of global stakeholder collaboration and underlines the significance of evidence-based decision-making. It serves as a paradigm for how data sharing initiatives can propel advancements in health care, emphasizing its adaptability and capacity to address diverse research inquiries.

Genome interpretation in a federated learning context allows the multi-center exome-based risk prediction of Crohn's disease patients

High-throughput sequencing allowed the discovery of many disease variants, but nowadays it is becoming clear that the abundance of genomics data mostly just moved the bottleneck in Genetics and Precision Medicine from a data availability issue to a data interpretation issue. To solve this empasse it would be beneficial to apply the latest Deep Learning (DL) methods to the Genome Interpretation (GI) problem, similarly to what AlphaFold did for Structural Biology. Unfortunately DL requires large datasets to be viable, and aggregating genomics datasets poses several legal, ethical and infrastructural complications. Federated Learning (FL) is a Machine Learning (ML) paradigm designed to tackle these issues. It allows ML methods to be collaboratively trained and tested on collections of physically separate datasets, without requiring the actual centralization of sensitive data. FL could thus be key to enable DL applications to GI on sufficiently large genomics data. We propose FedCrohn, a FL GI Neural Network model for the exome-based Crohn's Disease risk prediction, providing a proof-of-concept that FL is a viable paradigm to build novel ML GI approaches. We benchmark it in several realistic scenarios, showing that FL can indeed provide performances similar to conventional ML on centralized data, and that collaborating in FL initiatives is likely beneficial for most of the medical centers participating in them.

Applications of machine learning and deep learning in SPECT and PET imaging: General overview, challenges and future prospects

The integration of positron emission tomography (PET) and single-photon emission computed tomography (SPECT) imaging techniques with machine learning (ML) algorithms, including deep learning (DL) models, is a promising approach. This integration enhances the precision and efficiency of current diagnostic and treatment strategies while offering invaluable insights into disease mechanisms. In this comprehensive review, we delve into the transformative impact of ML and DL in this domain. Firstly, a brief analysis is provided of how these algorithms have evolved and which are the most widely applied in this domain. Their different potential applications in nuclear imaging are then discussed, such as optimization of image adquisition or reconstruction, biomarkers identification, multimodal fusion and the development of diagnostic, prognostic, and disease progression evaluation systems. This is because they are able to analyse complex patterns and relationships within imaging data, as well as extracting quantitative and objective measures. Furthermore, we discuss the challenges in implementation, such as data standardization and limited sample sizes, and explore the clinical opportunities and future horizons, including data augmentation and explainable AI. Together, these factors are propelling the continuous advancement of more robust, transparent, and reliable systems.

Source Free Semi-Supervised Transfer Learning for Diagnosis of Mental Disorders on fMRI Scans

The high prevalence of mental disorders gradually poses a huge pressure on the public healthcare services. Deep learning-based computer-aided diagnosis (CAD) has emerged to relieve the tension in healthcare institutions by detecting abnormal neuroimaging-derived phenotypes. However, training deep learning models relies on sufficient annotated datasets, which can be costly and laborious. Semi-supervised learning (SSL) and transfer learning (TL) can mitigate this challenge by leveraging unlabeled data within the same institution and advantageous information from source domain, respectively. This work is the first attempt to propose an effective semi-supervised transfer learning (SSTL) framework dubbed S3TL for CAD of mental disorders on fMRI data. Within S3TL, a secure cross-domain feature alignment method is developed to generate target-related source model in SSL. Subsequently, we propose an enhanced dual-stage pseudo-labeling approach to assign pseudo-labels for unlabeled samples in target domain. Finally, an advantageous knowledge transfer method is conducted to improve the generalization capability of the target model. Comprehensive experimental results demonstrate that S3TL achieves competitive accuracies of 69.14%, 69.65%, and 72.62% on ABIDE-I, ABIDE-II, and ADHD-200 datasets, respectively. Furthermore, the simulation experiments also demonstrate the application potential of S3TL through model interpretation analysis and federated learning extension.

Developing a privacy-preserving deep learning model for glaucoma detection: a multicentre study with federated learning

BACKGROUND

Deep learning (DL) is promising to detect glaucoma. However, patients' privacy and data security are major concerns when pooling all data for model development. We developed a privacy-preserving DL model using the federated learning (FL) paradigm to detect glaucoma from optical coherence tomography (OCT) images.

METHODS

This is a multicentre study. The FL paradigm consisted of a 'central server' and seven eye centres in Hong Kong, the USA and Singapore. Each centre first trained a model locally with its own OCT optic disc volumetric dataset and then uploaded its model parameters to the central server. The central server used FedProx algorithm to aggregate all centres' model parameters. Subsequently, the aggregated parameters are redistributed to each centre for its local model optimisation. We experimented with three three-dimensional (3D) networks to evaluate the stabilities of the FL paradigm. Lastly, we tested the FL model on two prospectively collected unseen datasets.

RESULTS

We used 9326 volumetric OCT scans from 2785 subjects. The FL model performed consistently well with different networks in 7 centres (accuracies 78.3%-98.5%, 75.9%-97.0%, and 78.3%-97.5%, respectively) and stably in the 2 unseen datasets (accuracies 84.8%-87.7%, 81.3%-84.8%, and 86.0%-87.8%, respectively). The FL model achieved non-inferior performance in classifying glaucoma compared with the traditional model and significantly outperformed the individual models.

CONCLUSION

The 3D FL model could leverage all the datasets and achieve generalisable performance, without data exchange across centres. This study demonstrated an OCT-based FL paradigm for glaucoma identification with ensured patient privacy and data security, charting another course toward the real-world transition of artificial intelligence in ophthalmology.

Medical Imaging Applications of Federated Learning

Since its introduction in 2016, researchers have applied the idea of Federated Learning (FL) to several domains ranging from edge computing to banking. The technique's inherent security benefits, privacy-preserving capabilities, ease of scalability, and ability to transcend data biases have motivated researchers to use this tool on healthcare datasets. While several reviews exist detailing FL and its applications, this review focuses solely on the different applications of FL to medical imaging datasets, grouping applications by diseases, modality, and/or part of the body. This Systematic Literature review was conducted by querying and consolidating results from ArXiv, IEEE Xplorer, and PubMed. Furthermore, we provide a detailed description of FL architecture, models, descriptions of the performance achieved by FL models, and how results compare with traditional Machine Learning (ML) models. Additionally, we discuss the security benefits, highlighting two primary forms of privacy-preserving techniques, including homomorphic encryption and differential privacy. Finally, we provide some background information and context regarding where the contributions lie. The background information is organized into the following categories: architecture/setup type, data-related topics, security, and learning types. While progress has been made within the field of FL and medical imaging, much room for improvement and understanding remains, with an emphasis on security and data issues remaining the primary concerns for researchers. Therefore, improvements are constantly pushing the field forward. Finally, we highlighted the challenges in deploying FL in medical imaging applications and provided recommendations for future directions.

Federated Semi-Supervised Learning for Medical Image Segmentation via Pseudo-Label Denoising

Distributed big data and digital healthcare technologies have great potential to promote medical services, but challenges arise when it comes to learning predictive model from diverse and complex e-health datasets. Federated Learning (FL), as a collaborative machine learning technique, aims to address the challenges by learning a joint predictive model across multi-site clients, especially for distributed medical institutions or hospitals. However, most existing FL methods assume that clients possess fully labeled data for training, which is often not the case in e-health datasets due to high labeling costs or expertise requirement. Therefore, this work proposes a novel and feasible approach to learn a Federated Semi-Supervised Learning (FSSL) model from distributed medical image domains, where a federated pseudo-labeling strategy for unlabeled clients is developed based on the embedded knowledge learned from labeled clients. This greatly mitigates the annotation deficiency at unlabeled clients and leads to a cost-effective and efficient medical image analysis tool. We demonstrated the effectiveness of our method by achieving significant improvements compared to the state-of-the-art in both fundus image and prostate MRI segmentation tasks, resulting in the highest Dice scores of 89.23% and 91.95% respectively even with only a few labeled clients participating in model training. This reveals the superiority of our method for practical deployment, ultimately facilitating the wider use of FL in healthcare and leading to better patient outcomes.

Federated learning for medical imaging radiology

Federated learning (FL) is gaining wide acceptance across the medical AI domains. FL promises to provide a fairly acceptable clinical-grade accuracy, privacy, and generalisability of machine learning models across multiple institutions. However, the research on FL for medical imaging AI is still in its early stages. This paper presents a review of recent research to outline the difference between state-of-the-art [SOTA] (published literature) and state-of-the-practice [SOTP] (applied research in realistic clinical environments). Furthermore, the review outlines the future research directions considering various factors such as data, learning models, system design, governance, and human-in-loop to translate the SOTA into SOTP and effectively collaborate across multiple institutions.

Multi-Scale Dynamic Graph Learning for Brain Disorder Detection With Functional MRI

Resting-state functional magnetic resonance imaging (rs-fMRI) has been widely used in the detection of brain disorders such as autism spectrum disorder based on various machine/deep learning techniques. Learning-based methods typically rely on functional connectivity networks (FCNs) derived from blood-oxygen-level-dependent time series of rs-fMRI data to capture interactions between brain regions-of-interest (ROIs). Graph neural networks have been recently used to extract fMRI features from graph-structured FCNs, but cannot effectively characterize spatiotemporal dynamics of FCNs, e.g., the functional connectivity of brain ROIs is dynamically changing in a short period of time. Also, many studies usually focus on single-scale topology of FCN, thereby ignoring the potential complementary topological information of FCN at different spatial resolutions. To this end, in this paper, we propose a multi-scale dynamic graph learning (MDGL) framework to capture multi-scale spatiotemporal dynamic representations of rs-fMRI data for automated brain disorder diagnosis. The MDGL framework consists of three major components: 1) multi-scale dynamic FCN construction using multiple brain atlases to model multi-scale topological information, 2) multi-scale dynamic graph representation learning to capture spatiotemporal information conveyed in fMRI data, and 3) multi-scale feature fusion and classification. Experimental results on two datasets show that MDGL outperforms several state-of-the-art methods.

Asynchronous Federated Learning for Improved Cardiovascular Disease Prediction Using Artificial Intelligence

Healthcare professionals consider predicting heart disease an essential task and deep learning has proven to be a promising approach for achieving this goal. This research paper introduces a novel method called the asynchronous federated deep learning approach for cardiac prediction (AFLCP), which combines a heart disease dataset and deep neural networks (DNNs) with an asynchronous learning technique. The proposed approach employs a method for asynchronously updating the parameters of DNNs and incorporates a temporally weighted aggregation technique to enhance the accuracy and convergence of the central model. To evaluate the effectiveness of the proposed AFLCP method, two datasets with various DNN architectures are tested, and the results demonstrate that the AFLCP approach outperforms the baseline method in terms of both communication cost and model accuracy.

FedNI: Federated Graph Learning With Network Inpainting for Population-Based Disease Prediction

Graph Convolutional Neural Networks (GCNs) are widely used for graph analysis. Specifically, in medical applications, GCNs can be used for disease prediction on a population graph, where graph nodes represent individuals and edges represent individual similarities. However, GCNs rely on a vast amount of data, which is challenging to collect for a single medical institution. In addition, a critical challenge that most medical institutions continue to face is addressing disease prediction in isolation with incomplete data information. To address these issues, Federated Learning (FL) allows isolated local institutions to collaboratively train a global model without data sharing. In this work, we propose a framework, FedNI, to leverage network inpainting and inter-institutional data via FL. Specifically, we first federatively train missing node and edge predictor using a graph generative adversarial network (GAN) to complete the missing information of local networks. Then we train a global GCN node classifier across institutions using a federated graph learning platform. The novel design enables us to build more accurate machine learning models by leveraging federated learning and also graph learning approaches. We demonstrate that our federated model outperforms local and baseline FL methods with significant margins on two public neuroimaging datasets.

Algorithmic fairness in artificial intelligence for medicine and healthcare

In healthcare, the development and deployment of insufficiently fair systems of artificial intelligence (AI) can undermine the delivery of equitable care. Assessments of AI models stratified across subpopulations have revealed inequalities in how patients are diagnosed, treated and billed. In this Perspective, we outline fairness in machine learning through the lens of healthcare, and discuss how algorithmic biases (in data acquisition, genetic variation and intra-observer labelling variability, in particular) arise in clinical workflows and the resulting healthcare disparities. We also review emerging technology for mitigating biases via disentanglement, federated learning and model explainability, and their role in the development of AI-based software as a medical device.

A novel federated deep learning scheme for glioma and its subtype classification

Background

Deep learning (DL) has shown promising results in molecular-based classification of glioma subtypes from MR images. DL requires a large number of training data for achieving good generalization performance. Since brain tumor datasets are usually small in size, combination of such datasets from different hospitals are needed. Data privacy issue from hospitals often poses a constraint on such a practice. Federated learning (FL) has gained much attention lately as it trains a central DL model without requiring data sharing from different hospitals.

Method

We propose a novel 3D FL scheme for glioma and its molecular subtype classification. In the scheme, a slice-based DL classifier, EtFedDyn, is exploited which is an extension of FedDyn, with the key differences on using focal loss cost function to tackle severe class imbalances in the datasets, and on multi-stream network to exploit MRIs in different modalities. By combining EtFedDyn with domain mapping as the pre-processing and 3D scan-based post-processing, the proposed scheme makes 3D brain scan-based classification on datasets from different dataset owners. To examine whether the FL scheme could replace the central learning (CL) one, we then compare the classification performance between the proposed FL and the corresponding CL schemes. Furthermore, detailed empirical-based analysis were also conducted to exam the effect of using domain mapping, 3D scan-based post-processing, different cost functions and different FL schemes.

Results

Experiments were done on two case studies: classification of glioma subtypes (IDH mutation and wild-type on TCGA and US datasets in case A) and glioma grades (high/low grade glioma HGG and LGG on MICCAI dataset in case B). The proposed FL scheme has obtained good performance on the test sets (85.46%, 75.56%) for IDH subtypes and (89.28%, 90.72%) for glioma LGG/HGG all averaged on five runs. Comparing with the corresponding CL scheme, the drop in test accuracy from the proposed FL scheme is small (-1.17%, -0.83%), indicating its good potential to replace the CL scheme. Furthermore, the empirically tests have shown that an increased classification test accuracy by applying: domain mapping (0.4%, 1.85%) in case A; focal loss function (1.66%, 3.25%) in case A and (1.19%, 1.85%) in case B; 3D post-processing (2.11%, 2.23%) in case A and (1.81%, 2.39%) in case B and EtFedDyn over FedAvg classifier (1.05%, 1.55%) in case A and (1.23%, 1.81%) in case B with fast convergence, which all contributed to the improvement of overall performance in the proposed FL scheme.

Conclusion

The proposed FL scheme is shown to be effective in predicting glioma and its subtypes by using MR images from test sets, with great potential of replacing the conventional CL approaches for training deep networks. This could help hospitals to maintain their data privacy, while using a federated trained classifier with nearly similar performance as that from a centrally trained one. Further detailed experiments have shown that different parts in the proposed 3D FL scheme, such as domain mapping (make datasets more uniform) and post-processing (scan-based classification), are essential.

Decentralized federated learning through proxy model sharing

Institutions in highly regulated domains such as finance and healthcare often have restrictive rules around data sharing. Federated learning is a distributed learning framework that enables multi-institutional collaborations on decentralized data with improved protection for each collaborator's data privacy. In this paper, we propose a communication-efficient scheme for decentralized federated learning called ProxyFL, or proxy-based federated learning. Each participant in ProxyFL maintains two models, a private model, and a publicly shared proxy model designed to protect the participant's privacy. Proxy models allow efficient information exchange among participants without the need of a centralized server. The proposed method eliminates a significant limitation of canonical federated learning by allowing model heterogeneity; each participant can have a private model with any architecture. Furthermore, our protocol for communication by proxy leads to stronger privacy guarantees using differential privacy analysis. Experiments on popular image datasets, and a cancer diagnostic problem using high-quality gigapixel histology whole slide images, show that ProxyFL can outperform existing alternatives with much less communication overhead and stronger privacy.

Multiple sclerosis lesion segmentation: revisiting weighting mechanisms for federated learning

Background and introduction

Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's applications remain suboptimal in neuroimage analysis tasks such as lesion segmentation in multiple sclerosis (MS), due to variance in lesion characteristics imparted by different scanners and acquisition parameters.

Methods

In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation performance. In addition, the segmentation loss function in each client is also re-weighted according to the lesion volume for the data during training.

Results

The proposed method has been validated on two FL MS segmentation scenarios using public and clinical datasets. Specifically, the case-wise and voxel-wise Dice score of the proposed method under the first public dataset is 65.20 and 74.30, respectively. On the second in-house dataset, the case-wise and voxel-wise Dice score is 53.66, and 62.31, respectively.

Discussions and conclusions

The Comparison experiments on two FL MS segmentation scenarios using public and clinical datasets have demonstrated the effectiveness of the proposed method by significantly outperforming other FL methods. Furthermore, the segmentation performance of FL incorporating our proposed aggregation mechanism can achieve comparable performance to that from centralized training with all the raw data.

Influence of Data Distribution on Federated Learning Performance in Tumor Segmentation

Purpose

To investigate the correlation between differences in data distributions and federated deep learning (Fed-DL) algorithm performance in tumor segmentation on CT and MR images.

Materials and Methods

Two Fed-DL datasets were retrospectively collected (from November 2020 to December 2021): one dataset of liver tumor CT images (Federated Imaging in Liver Tumor Segmentation [or, FILTS]; three sites, 692 scans) and one publicly available dataset of brain tumor MR images (Federated Tumor Segmentation [or, FeTS]; 23 sites, 1251 scans). Scans from both datasets were grouped according to site, tumor type, tumor size, dataset size, and tumor intensity. To quantify differences in data distributions, the following four distance metrics were calculated: earth mover's distance (EMD), Bhattacharyya distance (BD), χ² distance (CSD), and Kolmogorov-Smirnov distance (KSD). Both federated and centralized nnU-Net models were trained by using the same grouped datasets. Fed-DL model performance was evaluated by using the ratio of Dice coefficients, θ, between federated and centralized models trained and tested on the same 80:20 split datasets.

Results

The Dice coefficient ratio (θ) between federated and centralized models was strongly negatively correlated with the distances between data distributions, with correlation coefficients of -0.920 for EMD, -0.893 for BD, and -0.899 for CSD. However, KSD was weakly correlated with θ, with a correlation coefficient of -0.479.

Conclusion

Performance of Fed-DL models in tumor segmentation on CT and MRI datasets was strongly negatively correlated with the distances between data distributions.Keywords: CT, Abdomen/GI, Liver, Comparative Studies, MR Imaging, Brain/Brain Stem, Convolutional Neural Network (CNN), Federated Deep Learning, Tumor Segmentation, Data Distribution Supplemental material is available for this article. © RSNA, 2023See also the commentary by Kwak and Bai in this issue.

Federated Learning for Medical Image Analysis with Deep Neural Networks

Medical image analysis using deep neural networks (DNN) has demonstrated state-of-the-art performance in image classification and segmentation tasks, aiding disease diagnosis. The accuracy of the DNN is largely governed by the quality and quantity of the data used to train the model. However, for the medical images, the critical security and privacy concerns regarding sharing of local medical data across medical establishments precludes exploiting the full DNN potential for clinical diagnosis. The federated learning (FL) approach enables the use of local model's parameters to train a global model, while ensuring data privacy and security. In this paper, we review the federated learning applications in medical image analysis with DNNs, highlight the security concerns, cover some efforts to improve FL model performance, and describe the challenges and future research directions.

Privacy-preserving artificial intelligence in healthcare: Techniques and applications

There has been an increasing interest in translating artificial intelligence (AI) research into clinically-validated applications to improve the performance, capacity, and efficacy of healthcare services. Despite substantial research worldwide, very few AI-based applications have successfully made it to clinics. Key barriers to the widespread adoption of clinically validated AI applications include non-standardized medical records, limited availability of curated datasets, and stringent legal/ethical requirements to preserve patients' privacy. Therefore, there is a pressing need to improvise new data-sharing methods in the age of AI that preserve patient privacy while developing AI-based healthcare applications. In the literature, significant attention has been devoted to developing privacy-preserving techniques and overcoming the issues hampering AI adoption in an actual clinical environment. To this end, this study summarizes the state-of-the-art approaches for preserving privacy in AI-based healthcare applications. Prominent privacy-preserving techniques such as Federated Learning and Hybrid Techniques are elaborated along with potential privacy attacks, security challenges, and future directions.

Applications of generative adversarial networks in neuroimaging and clinical neuroscience

Generative adversarial networks (GANs) are one powerful type of deep learning models that have been successfully utilized in numerous fields. They belong to the broader family of generative methods, which learn to generate realistic data with a probabilistic model by learning distributions from real samples. In the clinical context, GANs have shown enhanced capabilities in capturing spatially complex, nonlinear, and potentially subtle disease effects compared to traditional generative methods. This review critically appraises the existing literature on the applications of GANs in imaging studies of various neurological conditions, including Alzheimer's disease, brain tumors, brain aging, and multiple sclerosis. We provide an intuitive explanation of various GAN methods for each application and further discuss the main challenges, open questions, and promising future directions of leveraging GANs in neuroimaging. We aim to bridge the gap between advanced deep learning methods and neurology research by highlighting how GANs can be leveraged to support clinical decision making and contribute to a better understanding of the structural and functional patterns of brain diseases.

MIC-Net: A deep network for cross-site segmentation of COVID-19 infection in the fog-assisted IoMT

The automatic segmentation of COVID-19 pneumonia from a computerized tomography (CT) scan has become a major interest for scholars in developing a powerful diagnostic framework in the Internet of Medical Things (IoMT). Federated deep learning (FDL) is considered a promising approach for efficient and cooperative training from multi-institutional image data. However, the nonindependent and identically distributed (Non-IID) data from health care remain a remarkable challenge, limiting the applicability of FDL in the real world. The variability in features incurred by different scanning protocols, scanners, or acquisition parameters produces the learning drift phenomena during the training, which impairs both the training speed and segmentation performance of the model. This paper proposes a novel FDL approach for reliable and efficient multi-institutional COVID-19 segmentation, called MIC-Net. MIC-Net consists of three main building modules: the down-sampler, context enrichment (CE) module, and up-sampler. The down-sampler was designed to effectively learn both local and global representations from input CT scans by combining the advantages of lightweight convolutional and attention modules. The contextual enrichment (CE) module is introduced to enable the network to capture the contextual representation that can be later exploited to enrich the semantic knowledge of the up-sampler through skip connections. To further tackle the inter-site heterogeneity within the model, the approach uses an adaptive and switchable normalization (ASN) to adaptively choose the best normalization strategy according to the underlying data. A novel federated periodic selection protocol (FED-PCS) is proposed to fairly select the training participants according to their resource state, data quality, and loss of a local model. The results of an experimental evaluation of MIC-Net on three publicly available data sets show its robust performance, with an average dice score of 88.90% and an average surface dice of 87.53%.

Federated Transfer Learning for Low-dose PET Denoising: A Pilot Study with Simulated Heterogeneous Data

Positron emission tomography (PET) with a reduced injection dose, i.e., low-dose PET, is an efficient way to reduce radiation dose. However, low-dose PET reconstruction suffers from a low signal-to-noise ratio (SNR), affecting diagnosis and other PET-related applications. Recently, deep learning-based PET denoising methods have demonstrated superior performance in generating high-quality reconstruction. However, these methods require a large amount of representative data for training, which can be difficult to collect and share due to medical data privacy regulations. Moreover, low-dose PET data at different institutions may use different low-dose protocols, leading to non-identical data distribution. While previous federated learning (FL) algorithms enable multi-institution collaborative training without the need of aggregating local data, it is challenging for previous methods to address the large domain shift caused by different low-dose PET settings, and the application of FL to PET is still under-explored. In this work, we propose a federated transfer learning (FTL) framework for low-dose PET denoising using heterogeneous low-dose data. Our experimental results on simulated multi-institutional data demonstrate that our method can efficiently utilize heterogeneous low-dose data without compromising data privacy for achieving superior low-dose PET denoising performance for different institutions with different low-dose settings, as compared to previous FL methods.

Unsupervised cross-domain functional MRI adaptation for automated major depressive disorder identification

Resting-state functional magnetic resonance imaging (rs-fMRI) data have been widely used for automated diagnosis of brain disorders such as major depressive disorder (MDD) to assist in timely intervention. Multi-site fMRI data have been increasingly employed to augment sample size and improve statistical power for investigating MDD. However, previous studies usually suffer from significant inter-site heterogeneity caused for instance by differences in scanners and/or scanning protocols. To address this issue, we develop a novel discrepancy-based unsupervised cross-domain fMRI adaptation framework (called UFA-Net) for automated MDD identification. The proposed UFA-Net is designed to model spatio-temporal fMRI patterns of labeled source and unlabeled target samples via an attention-guided graph convolution module, and also leverage a maximum mean discrepancy constrained module for unsupervised cross-site feature alignment between two domains. To the best of our knowledge, this is one of the first attempts to explore unsupervised rs-fMRI adaptation for cross-site MDD identification. Extensive evaluation on 681 subjects from two imaging sites shows that the proposed method outperforms several state-of-the-art methods. Our method helps localize disease-associated functional connectivity abnormalities and is therefore well interpretable and can facilitate fMRI-based analysis of MDD in clinical practice.

Memory-aware curriculum federated learning for breast cancer classification

BACKGROUND AND OBJECTIVE

For early breast cancer detection, regular screening with mammography imaging is recommended. Routine examinations result in datasets with a predominant amount of negative samples. The limited representativeness of positive cases can be problematic for learning Computer-Aided Diagnosis (CAD) systems. Collecting data from multiple institutions is a potential solution to mitigate this problem. Recently, federated learning has emerged as an effective tool for collaborative learning. In this setting, local models perform computation on their private data to update the global model. The order and the frequency of local updates influence the final global model. In the context of federated adversarial learning to improve multi-site breast cancer classification, we investigate the role of the order in which samples are locally presented to the optimizers.

METHODS

We define a novel memory-aware curriculum learning method for the federated setting. We aim to improve the consistency of the local models penalizing inconsistent predictions, i.e., forgotten samples. Our curriculum controls the order of the training samples prioritizing those that are forgotten after the deployment of the global model. Our approach is combined with unsupervised domain adaptation to deal with domain shift while preserving data privacy.

RESULTS

Two classification metrics: area under the receiver operating characteristic curve (ROC-AUC) and area under the curve for the precision-recall curve (PR-AUC) are used to evaluate the performance of the proposed method. Our method is evaluated with three clinical datasets from different vendors. An ablation study showed the improvement of each component of our method. The AUC and PR-AUC are improved on average by 5% and 6%, respectively, compared to the conventional federated setting.

CONCLUSIONS

We demonstrated the benefits of curriculum learning for the first time in a federated setting. Our results verified the effectiveness of the memory-aware curriculum federated learning for the multi-site breast cancer classification. Our code is publicly available at: https://github.com/ameliajimenez/curriculum-federated-learning.

Imaging genomics: data fusion in uncovering disease heritability

Sequencing of the human genome in the early 2000s enabled probing of the genetic basis of disease on a scale previously unimaginable. Now, two decades later, after interrogating millions of markers in thousands of individuals, a significant portion of disease heritability still remains hidden. Recent efforts to unravel this 'missing heritability' have focused on garnering new insight from merging different data types, including medical imaging. Imaging offers promising intermediate phenotypes to bridge the gap between genetic variation and disease pathology. In this review we outline this fusion and provide examples of imaging genomics in a range of diseases, from oncology to cardiovascular and neurodegenerative disease. Finally, we discuss how ongoing revolutions in data science and sharing are primed to advance the field.

Adversarial learning-based multi-level dense-transmission knowledge distillation for AP-ROP detection

The Aggressive Posterior Retinopathy of Prematurity (AP-ROP) is the major cause of blindness for premature infants. The automatic diagnosis method has become an important tool for detecting AP-ROP. However, most existing automatic diagnosis methods were with heavy complexity, which hinders the development of the detecting devices. Hence, a small network (student network) with a high imitation ability is exactly needed, which can mimic a large network (teacher network) with promising diagnostic performance. Also, if the student network is too small due to the increasing gap between teacher and student networks, the diagnostic performance will drop. To tackle the above issues, we propose a novel adversarial learning-based multi-level dense knowledge distillation method for detecting AP-ROP. Specifically, the pre-trained teacher network is utilized to train multiple intermediate-size networks (i.e., teacher-assistant networks) and one student network by dense transmission mode, where the knowledge from all upper-level networks is transmitted to the current lower-level network. To ensure that two adjacent networks can distill the abundant knowledge, the adversarial learning module is leveraged to enforce the lower-level network to generate the features that are similar to those of the upper-level network. Extensive experiments demonstrate that our proposed method can realize the effective knowledge distillation from the teacher to student networks. We achieve a promising knowledge distillation performance for our private dataset and a public dataset, which can provide a new insight for devising lightweight detecting systems of fundus diseases for practical use.