Mitigating site effects in covariance for machine learning in neuroimaging data

Examining litter specific variability in mice and its impact on neurodevelopmental studies

Our current understanding of litter variability in neurodevelopmental studies using mice may limit translation of neuroscientific findings. Higher variance of measures across litters than within, often termed intra-litter likeness, may be attributable to both pre- and postnatal environment. This study aimed to assess the litter-effect within behavioral assessments (2 timepoints) and anatomy using T1-weighted magnetic resonance images across 72 brain region volumes (4 timepoints) (36 C57bl/6J inbred mice; 7 litters: 19F/17M). Between-litter comparisons of brain and behavioral measures and their associations were evaluated using univariate and multivariate techniques. A power analysis using simulation methods was then performed on modeled neurodevelopment and to evaluate trade-offs between number-of-litters, number-of-mice-per-litter, and sample size. Our results show litter-specific developmental effects, from the adolescent period to adulthood for brain structure volumes and behaviors, and for their associations in adulthood. Our power simulation analysis suggests increasing the number-of-litters in experimental designs to achieve the smallest total sample size necessary for detecting different rates of change in specific brain regions. Our results demonstrate how litter-specific effects may influence development and that increasing the litters to the total sample size ratio should be strongly considered when designing neurodevelopmental studies.

Generalizable neuromarker for autism spectrum disorder across imaging sites and developmental stages: A multi-site study

Autism spectrum disorder (ASD) is a lifelong condition, and its underlying biological mechanisms remain elusive. The complexity of various factors, including inter-site and development-related differences, makes it challenging to develop generalizable neuroimaging-based biomarkers for ASD. This study used a large-scale, multi-site dataset of 730 Japanese adults to develop a generalizable neuromarker for ASD across independent sites (U.S., Belgium, and Japan) and different developmental stages (children and adolescents). Our adult ASD neuromarker achieved successful generalization for the US and Belgium adults (area under the curve [AUC] = 0.70) and Japanese adults (AUC = 0.81). The neuromarker demonstrated significant generalization for children (AUC = 0.66) and adolescents (AUC = 0.71; all P < 0.05 , family-wise-error corrected). We identified 141 functional connections (FCs) important for discriminating individuals with ASD from TDCs. These FCs largely centered on social brain regions such as the amygdala, hippocampus, dorsomedial and ventromedial prefrontal cortices, and temporal cortices. Finally, we mapped schizophrenia (SCZ) and major depressive disorder (MDD) onto the biological axis defined by the neuromarker and explored the biological continuity of ASD with SCZ and MDD. We observed that SCZ, but not MDD, was located proximate to ASD on the biological dimension defined by the ASD neuromarker. The successful generalization in multifarious datasets and the observed relations of ASD with SCZ on the biological dimensions provide new insights for a deeper understanding of ASD.

Multivariate Residualization in Medical Imaging Analysis

Nuisance variables in medical imaging research are common, complicating association and prediction studies based on image data. Medical image data are typically high dimensional, often consisting of many highly correlated features. As a result, computationally efficient and robust methods to address nuisance variables are difficult to implement. By-region univariate residualization is commonly used to remove the influence of nuisance variables, as are various extensions. However, these methods neglect multivariate properties and may fail to fully remove influence related to the joint distribution of these regions. Some methods, such as functional regression and others, do consider multivariate properties when controlling for nuisance variables. However, the utility of these methods is limited for data with many image regions due to computational and model complexity. We develop a multivariate residualization method to estimate the association between the image and nuisance variable using a machine learning algorithm and then compute the orthogonal projection of each subject's image data onto this space. We illustrate this method's performance in a set of simulation studies and apply it to data from the Alzheimer's Disease Neuroimaging Initiative (ADNI).

AI-based dimensional neuroimaging system for characterizing heterogeneity in brain structure and function in major depressive disorder: COORDINATE-MDD consortium design and rationale

BACKGROUND

Efforts to develop neuroimaging-based biomarkers in major depressive disorder (MDD), at the individual level, have been limited to date. As diagnostic criteria are currently symptom-based, MDD is conceptualized as a disorder rather than a disease with a known etiology; further, neural measures are often confounded by medication status and heterogeneous symptom states.

METHODS

We describe a consortium to quantify neuroanatomical and neurofunctional heterogeneity via the dimensions of novel multivariate coordinate system (COORDINATE-MDD). Utilizing imaging harmonization and machine learning methods in a large cohort of medication-free, deeply phenotyped MDD participants, patterns of brain alteration are defined in replicable and neurobiologically-based dimensions and offer the potential to predict treatment response at the individual level. International datasets are being shared from multi-ethnic community populations, first episode and recurrent MDD, which are medication-free, in a current depressive episode with prospective longitudinal treatment outcomes and in remission. Neuroimaging data consist of de-identified, individual, structural MRI and resting-state functional MRI with additional positron emission tomography (PET) data at specific sites. State-of-the-art analytic methods include automated image processing for extraction of anatomical and functional imaging variables, statistical harmonization of imaging variables to account for site and scanner variations, and semi-supervised machine learning methods that identify dominant patterns associated with MDD from neural structure and function in healthy participants.

RESULTS

We are applying an iterative process by defining the neural dimensions that characterise deeply phenotyped samples and then testing the dimensions in novel samples to assess specificity and reliability. Crucially, we aim to use machine learning methods to identify novel predictors of treatment response based on prospective longitudinal treatment outcome data, and we can externally validate the dimensions in fully independent sites.

CONCLUSION

We describe the consortium, imaging protocols and analytics using preliminary results. Our findings thus far demonstrate how datasets across many sites can be harmonized and constructively pooled to enable execution of this large-scale project.

Multicenter data harmonization for regional brain atrophy and application in multiple sclerosis

BACKGROUND

In multiple sclerosis (MS), determination of regional brain atrophy is clinically relevant. However, analysis of large datasets is rare because of the increased variability in multicenter data.

PURPOSE

To compare different methods to correct for center effects. To investigate regional gray matter (GM) volume in relapsing-remitting MS in a large multicenter dataset.

METHODS

MRI scans of 466 MS patients and 279 healthy controls (HC) were retrieved from the Italian Neuroimaging Network Initiative repository. Voxel-based morphometry was performed. The center effect was accounted for with different methods: (a) no correction, (b) factor in the statistical model, (c) ComBat method and (d) subsampling procedure to match single-center distributions. By applying the best correction method, GM atrophy was assessed in MS patients vs HC and according to clinical disability, disease duration and T₂ lesion volume. Results were assessed voxel-wise using general linear model.

RESULTS

The average residuals for the harmonization methods were 5.03 (a), 4.42 (b), 4.26 (c) and 2.98 (d). The comparison between MS patients and HC identified thalami and other deep GM nuclei, the cerebellum and several cortical regions. At single-center analysis, the thalami were always involved, whereas different other regions were found in each center. Cerebellar atrophy correlated with clinical disability, while deep GM nuclei atrophy correlated with T₂-lesion volume.

CONCLUSION

Harmonization based on subsampling more effectively decreased the residuals of the statistical model applied. In comparison with findings from single-center analysis, the multicenter results were more robust, highlighting the importance of data repositories from multiple centers.

Leveraging Machine Learning for Gaining Neurobiological and Nosological Insights in Psychiatric Research

Much attention is currently devoted to developing diagnostic classifiers for mental disorders. Complementing these efforts, we highlight the potential of machine learning to gain biological insights into the psychopathology and nosology of mental disorders. Studies to this end have mainly used brain imaging data, which can be obtained noninvasively from large cohorts and have repeatedly been argued to reveal potentially intermediate phenotypes. This may become particularly relevant in light of recent efforts to identify magnetic resonance imaging-derived biomarkers that yield insight into pathophysiological processes as well as to refine the taxonomy of mental illness. In particular, the accuracy of machine learning models may be used as dependent variables to identify features relevant to pathophysiology. Moreover, such approaches may help disentangle the dimensional (within diagnosis) and often overlapping (across diagnoses) symptomatology of psychiatric illness. We also point out a multiview perspective that combines data from different sources, bridging molecular and system-level information. Finally, we summarize recent efforts toward a data-driven definition of subtypes or disease entities through unsupervised and semisupervised approaches. The latter, blending unsupervised and supervised concepts, may represent a particularly promising avenue toward dissecting heterogeneous categories. Finally, we raise several technical and conceptual aspects related to the reviewed approaches. In particular, we discuss common pitfalls pertaining to flawed input data or analytic procedures that would likely lead to unreliable outputs.

Coordinated cortical thickness alterations across six neurodevelopmental and psychiatric disorders

Neuropsychiatric disorders are increasingly conceptualized as overlapping spectra sharing multi-level neurobiological alterations. However, whether transdiagnostic cortical alterations covary in a biologically meaningful way is currently unknown. Here, we studied co-alteration networks across six neurodevelopmental and psychiatric disorders, reflecting pathological structural covariance. In 12,024 patients and 18,969 controls from the ENIGMA consortium, we observed that co-alteration patterns followed normative connectome organization and were anchored to prefrontal and temporal disease epicenters. Manifold learning revealed frontal-to-temporal and sensory/limbic-to-occipitoparietal transdiagnostic gradients, differentiating shared illness effects on cortical thickness along these axes. The principal gradient aligned with a normative cortical thickness covariance gradient and established a transcriptomic link to cortico-cerebello-thalamic circuits. Moreover, transdiagnostic gradients segregated functional networks involved in basic sensory, attentional/perceptual, and domain-general cognitive processes, and distinguished between regional cytoarchitectonic profiles. Together, our findings indicate that shared illness effects occur in a synchronized fashion and along multiple levels of hierarchical cortical organization.

Improved generalized ComBat methods for harmonization of radiomic features

Radiomic approaches in precision medicine are promising, but variation associated with image acquisition factors can result in severe biases and low generalizability. Multicenter datasets used in these studies are often heterogeneous in multiple imaging parameters and/or have missing information, resulting in multimodal radiomic feature distributions. ComBat is a promising harmonization tool, but it only harmonizes by single/known variables and assumes standardized input data are normally distributed. We propose a procedure that sequentially harmonizes for multiple batch effects in an optimized order, called OPNested ComBat. Furthermore, we propose to address bimodality by employing a Gaussian Mixture Model (GMM) grouping considered as either a batch variable (OPNested + GMM) or as a protected clinical covariate (OPNested - GMM). Methods were evaluated on features extracted with CapTK and PyRadiomics from two public lung computed tomography (CT) datasets. We found that OPNested ComBat improved harmonization performance over standard ComBat. OPNested + GMM ComBat exhibited the best harmonization performance but the lowest predictive performance, while OPNested - GMM ComBat showed poorer harmonization performance, but the highest predictive performance. Our findings emphasize that improved harmonization performance is no guarantee of improved predictive performance, and that these methods show promise for superior standardization of datasets heterogeneous in multiple or unknown imaging parameters and greater generalizability.

Site effects how-to and when: An overview of retrospective techniques to accommodate site effects in multi-site neuroimaging analyses

Site differences, or systematic differences in feature distributions across multiple data-acquisition sites, are a known source of heterogeneity that may adversely affect large-scale meta- and mega-analyses of independently collected neuroimaging data. They influence nearly all multi-site imaging modalities and biomarkers, and methods to compensate for them can improve reliability and generalizability in the analysis of genetics, omics, and clinical data. The origins of statistical site effects are complex and involve both technical differences (scanner vendor, head coil, acquisition parameters, imaging processing) and differences in sample characteristics (inclusion/exclusion criteria, sample size, ancestry) between sites. In an age of expanding international consortium research, there is a growing need to disentangle technical site effects from sample characteristics of interest. Numerous statistical and machine learning methods have been developed to control for, model, or attenuate site effects - yet to date, no comprehensive review has discussed the benefits and drawbacks of each for different use cases. Here, we provide an overview of the different existing statistical and machine learning methods developed to remove unwanted site effects from independently collected neuroimaging samples. We focus on linear mixed effect models, the ComBat technique and its variants, adjustments based on image quality metrics, normative modeling, and deep learning approaches such as generative adversarial networks. For each method, we outline the statistical foundation and summarize strengths and weaknesses, including their assumptions and conditions of use. We provide information on software availability and comment on the ease of use and the applicability of these methods to different types of data. We discuss validation and comparative reports, mention caveats and provide guidance on when to use each method, depending on context and specific research questions.

A guide for researchers seeking training in retrospective data harmonization for population neuroscience studies of Alzheimer's disease and related dementias

Due to needs surrounding rigor and reproducibility, subgroup specific disease knowledge, and questions of external validity, data harmonization is an essential tool in population neuroscience of Alzheimer's disease and related dementias (ADRD). Systematic harmonization of data elements is necessary to pool information from heterogeneous samples, and such pooling allows more expansive evaluations of health disparities, more precise effect estimates, and more opportunities to discover effective prevention or treatment strategies. The key goal of this Tutorial in Population Neuroimaging Curriculum, Instruction, and Pedagogy article is to guide researchers in creating a customized population neuroscience of ADRD harmonization training plan to fit their needs or those of their mentees. We provide brief guidance for retrospective data harmonization of multiple data types in this area, including: (1) clinical and demographic, (2) neuropsychological, and (3) neuroimaging data. Core competencies and skills are reviewed, and resources are provided to fill gaps in training as well as data needs. We close with an example study in which harmonization is a critical tool. While several aspects of this tutorial focus specifically on ADRD, the concepts and resources are likely to benefit population neuroscientists working in a range of research areas.

Assessing Multi-Site rs-fMRI-Based Connectomic Harmonization Using Information Theory

Several harmonization techniques have recently been proposed for connectomics/networks derived from resting-state functional magnetic resonance imaging (rs-fMRI) acquired at multiple sites. These techniques have the objective of mitigating site-specific biases that complicate its subsequent analysis and, therefore, compromise the quality of the results when these images are analyzed together. Thus, harmonization is indispensable when large cohorts are required in which the data obtained must be independent of the particular condition of each resonator, its make and model, its calibration, and other features or artifacts that may affect the significance of the acquisition. To date, no assessment of the actual efficacy of these harmonization techniques has been proposed. In this work, we apply recently introduced Information Theory tools to analyze the effectiveness of these techniques, developing a methodology that allows us to compare different harmonization models. We demonstrate the usefulness of this methodology by applying it to some of the most widespread harmonization frameworks and datasets. As a result, we are able to show that some of these techniques are indeed ineffective since the acquisition site can still be determined from the fMRI data after the processing.

Parsimonious model for mass-univariate vertexwise analysis

Purpose: Covariance between gray-matter measurements can reflect structural or functional brain networks though it has also been shown to be influenced by confounding factors (e.g., age, head size, and scanner), which could lead to lower mapping precision (increased size of associated clusters) and create distal false positives associations in mass-univariate vertexwise analyses. Approach: We evaluated this concern by performing state-of-the-art mass-univariate analyses (general linear model, GLM) on traits simulated from real vertex-wise gray matter data (including cortical and subcortical thickness and surface area). We contrasted the results with those from linear mixed models (LMMs), which have been shown to overcome similar issues in omics association studies. Results: We showed that when performed on a large sample ( N = 8662 , UK Biobank), GLMs yielded greatly inflated false positive rate (cluster false discovery rate > 0.6 ). We showed that LMMs resulted in more parsimonious results: smaller clusters and reduced false positive rate but at a cost of increased computation. Next, we performed mass-univariate association analyses on five real UKB traits (age, sex, BMI, fluid intelligence, and smoking status) and LMM yielded fewer and more localized associations. We identified 19 significant clusters displaying small associations with age, sex, and BMI, which suggest a complex architecture of at least dozens of associated areas with those phenotypes. Conclusions: The published literature could contain a large proportion of redundant (possibly confounded) associations that are largely prevented using LMMs. The parsimony of LMMs results from controlling for the joint effect of all vertices, which prevents local and distal redundant associations from reaching significance.

Two subtypes of schizophrenia identified by an individual-level atypical pattern of tensor-based morphometric measurement

Difficulties in parsing the multiaspect heterogeneity of schizophrenia (SCZ) based on current nosology highlight the need to subtype SCZ using objective biomarkers. Here, utilizing a large-scale multisite SCZ dataset, we identified and validated 2 neuroanatomical subtypes with individual-level abnormal patterns of the tensor-based morphometric measurement. Remarkably, compared with subtype 1, which showed moderate deficits of some subcortical nuclei and an enlarged striatum and cerebellum, subtype 2, which showed cerebellar atrophy and more severe subcortical nuclei atrophy, had a higher subscale score of negative symptoms, which is considered to be a core aspect of SCZ and is associated with functional outcome. Moreover, with the neuroimaging-clinic association analysis, we explored the detailed relationship between the heterogeneity of clinical symptoms and the heterogeneous abnormal neuroanatomical patterns with respect to the 2 subtypes. And the neuroimaging-transcription association analysis highlighted several potential heterogeneous biological factors that may underlie the subtypes. Our work provided an effective framework for investigating the heterogeneity of SCZ from multilevel aspects and may provide new insights for precision psychiatry.

Harmonizing functional connectivity reduces scanner effects in community detection

Community detection on graphs constructed from functional magnetic resonance imaging (fMRI) data has led to important insights into brain functional organization. Large studies of brain community structure often include images acquired on multiple scanners across different studies. Differences in scanner can introduce variability into the downstream results, and these differences are often referred to as scanner effects. Such effects have been previously shown to significantly impact common network metrics. In this study, we identify scanner effects in data-driven community detection results and related network metrics. We assess a commonly employed harmonization method and propose new methodology for harmonizing functional connectivity that leverage existing knowledge about network structure as well as patterns of covariance in the data. Finally, we demonstrate that our new methods reduce scanner effects in community structure and network metrics. Our results highlight scanner effects in studies of brain functional organization and provide additional tools to address these unwanted effects. These findings and methods can be incorporated into future functional connectivity studies, potentially preventing spurious findings and improving reliability of results.

A deep learning-based multisite neuroimage harmonization framework established with a traveling-subject dataset

The accumulation of multisite large-sample MRI datasets collected during large brain research projects in the last decade has provided critical resources for understanding the neurobiological mechanisms underlying cognitive functions and brain disorders. However, the significant site effects observed in imaging data and their derived structural and functional features have prevented the derivation of consistent findings across multiple studies. The development of harmonization methods that can effectively eliminate complex site effects while maintaining biological characteristics in neuroimaging data has become a vital and urgent requirement for multisite imaging studies. Here, we propose a deep learning-based framework to harmonize imaging data obtained from pairs of sites, in which site factors and brain features can be disentangled and encoded. We trained the proposed framework with a publicly available traveling subject dataset from the Strategic Research Program for Brain Sciences (SRPBS) and harmonized the gray matter volume maps derived from eight source sites to a target site. The proposed framework significantly eliminated intersite differences in gray matter volumes. The embedded encoders successfully captured both the abstract textures of site factors and the concrete brain features. Moreover, the proposed framework exhibited outstanding performance relative to conventional statistical harmonization methods in terms of site effect removal, data distribution homogenization, and intrasubject similarity improvement. Finally, the proposed harmonization network provided fixable expandability, through which new sites could be linked to the target site via indirect schema without retraining the whole model. Together, the proposed method offers a powerful and interpretable deep learning-based harmonization framework for multisite neuroimaging data that can enhance reliability and reproducibility in multisite studies regarding brain development and brain disorders.

Parameters from site classification to harmonize MRI clinical studies: Application to a multi-site Parkinson's disease dataset

Multi‐site MRI datasets are crucial for big data research. However, neuroimaging studies must face the batch effect. Here, we propose an approach that uses the predictive probabilities provided by Gaussian processes (GPs) to harmonize clinical‐based studies. A multi‐site dataset of 216 Parkinson's disease (PD) patients and 87 healthy subjects (HS) was used. We performed a site GP classification using MRI data. The outcomes estimated from this classification, redefined like Weighted HARMonization PArameters (WHARMPA), were used as regressors in two different clinical studies: A PD versus HS machine learning classification using GP, and a VBM comparison (FWE‐p < .05, k = 100). Same studies were also conducted using conventional Boolean site covariates, and without information about site belonging. The results from site GP classification provided high scores, balanced accuracy (BAC) was 98.39% for grey matter images. PD versus HS classification performed better when the WHARMPA were used to harmonize (BAC = 78.60%; AUC = 0.90) than when using the Boolean site information (BAC = 56.31%; AUC = 0.71) and without it (BAC = 57.22%; AUC = 0.73). The VBM analysis harmonized using WHARMPA provided larger and more statistically robust clusters in regions previously reported in PD than when the Boolean site covariates or no corrections were added to the model. In conclusion, WHARMPA might encode global site‐effects quantitatively and allow the harmonization of data. This method is user‐friendly and provides a powerful solution, without complex implementations, to clean the analyses by removing variability associated with the differences between sites.