How to control for confounds in decoding analyses of neuroimaging data

A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods

The comprehensive integration of machine learning healthcare models within clinical practice remains suboptimal, notwithstanding the proliferation of high-performing solutions reported in the literature. A predominant factor hindering widespread adoption pertains to an insufficiency of evidence affirming the reliability of the aforementioned models. Recently, uncertainty quantification methods have been proposed as a potential solution to quantify the reliability of machine learning models and thus increase the interpretability and acceptability of the results. In this review, we offer a comprehensive overview of the prevailing methods proposed to quantify the uncertainty inherent in machine learning models developed for various medical image tasks. Contrary to earlier reviews that exclusively focused on probabilistic methods, this review also explores non-probabilistic approaches, thereby furnishing a more holistic survey of research pertaining to uncertainty quantification for machine learning models. Analysis of medical images with the summary and discussion on medical applications and the corresponding uncertainty evaluation protocols are presented, which focus on the specific challenges of uncertainty in medical image analysis. We also highlight some potential future research work at the end. Generally, this review aims to allow researchers from both clinical and technical backgrounds to gain a quick and yet in-depth understanding of the research in uncertainty quantification for medical image analysis machine learning models.

Accurate identification of individuals with subjective cognitive decline using 3D regional fractal dimensions on structural magnetic resonance imaging

BACKGROUND AND OBJECTIVE

Accurate identification of individuals with subjective cognitive decline (SCD) is crucial for early intervention and prevention of neurodegenerative diseases. Fractal dimensionality (FD) has emerged as a robust and replicable measure, surpassing traditional geometric metrics, in characterizing the intricate fractal geometrical properties of brain structure. Nevertheless, the effectiveness of FD in identifying individuals with SCD remains largely unclear. A 3D regional FD method can be suggested to characterize and quantify the spatial complexity of the precise gray matter, providing insights into cognitive aging and aiding in the automated identification of individuals with SCD.

METHODS

This study introduces a novel integer ratio based 3D box-counting fractal analysis (IRBCFA) to quantify regional fractal dimensions (FDs) in structural magnetic resonance imaging (MRI) data. The innovative method overcomes limitations of conventional box-counting techniques by accommodating arbitrary box sizes, thereby enhancing the precision of FD estimation in small, yet neurologically significant, brain regions.

RESULTS

The application of IRBCFA to two publicly available datasets, OASIS-3 and ADNI, consisting of 520 and 180 subjects, respectively. The method identified discriminative regions of interest (ROIs) predominantly within the limbic system, fronto-parietal region, occipito-temporal region, and basal ganglia-thalamus region. These ROIs exhibited significant correlations with cognitive functions, including executive functioning, memory, social cognition, and sensory perception, suggesting their potential as neuroimaging markers for SCD. The identification model trained on these ROIs demonstrated exceptional performance achieving over 93 % accuracy on the discovery dataset and exceeding 87 % on the independent testing dataset. Furthermore, an exchange experiment between datasets revealed a substantial overlap in discriminative ROIs, highlighting the robustness of our method across diverse populations.

CONCLUSION

Our findings indicate that IRBCFA can serve as a valuable tool for quantifying the spatial complexity of gray matter, providing insights into cognitive aging and aiding in the automated identification of individuals with SCD. The demonstrated generalizability and robustness of this method position it as a promising tool for neurodegenerative disease research and offer potential for clinical applications.

Shared and distinctive brain networks underlying trait and state rumination

Although trait and state rumination play a central role in the exacerbation of negative affect, evidence suggests that they are weakly correlated and exert distinct influences on emotional reactivity to stressors. Whether trait and state rumination share a common or exhibit distinct neural substrate remains unclear. In this study, we utilized functional near-infrared spectroscopy (fNIRS) combined with connectome-based predictive modeling (CPM) to identify neural fingerprints associated with trait and state rumination. CPM identified distinctive functional connectivity (FC) profiles that contribute to the prediction of trait rumination, primarily involving FC within the default mode network (DMN) and the dorsal attention network (DAN) as well as FC between the DMN, control network (CN), DAN, and salience network (SN). Conversely, state rumination was predominantly associated with FC between the DMN and CN. Furthermore, the predictive features of trait rumination can be robustly generalized to predict state rumination, and vice versa. In conclusion, this study illuminates the importance of both DMN and non-DMN systems in the emergence and persistence of rumination. While trait rumination was associated with stronger and broader FC than state rumination, the generalizability of the predictive features underscores the presence of shared neural mechanisms between the two forms of rumination. These identified connectivity fingerprints may hold promise as targets for innovative therapeutic interventions aimed at mitigating rumination-related negative affect.

Power and reproducibility in the external validation of brain-phenotype predictions

Brain-phenotype predictive models seek to identify reproducible and generalizable brain-phenotype associations. External validation, or the evaluation of a model in external datasets, is the gold standard in evaluating the generalizability of models in neuroimaging. Unlike typical studies, external validation involves two sample sizes: the training and the external sample sizes. Thus, traditional power calculations may not be appropriate. Here we ran over 900 million resampling-based simulations in functional and structural connectivity data to investigate the relationship between training sample size, external sample size, phenotype effect size, theoretical power and simulated power. Our analysis included a wide range of datasets: the Healthy Brain Network, the Adolescent Brain Cognitive Development Study, the Human Connectome Project (Development and Young Adult), the Philadelphia Neurodevelopmental Cohort, the Queensland Twin Adolescent Brain Project, and the Chinese Human Connectome Project; and phenotypes: age, body mass index, matrix reasoning, working memory, attention problems, anxiety/depression symptoms and relational processing. High effect size predictions achieved adequate power with training and external sample sizes of a few hundred individuals, whereas low and medium effect size predictions required hundreds to thousands of training and external samples. In addition, most previous external validation studies used sample sizes prone to low power, and theoretical power curves should be adjusted for the training sample size. Furthermore, model performance in internal validation often informed subsequent external validation performance (Pearson's r difference <0.2), particularly for well-harmonized datasets. These results could help decide how to power future external validation studies.

Functional Connectivity-Based Searchlight Multivariate Pattern Analysis for Discriminating Schizophrenia Patients and Predicting Clinical Variables

BACKGROUND

Schizophrenia, a multifaceted psychiatric disorder characterized by functional dysconnectivity, poses significant challenges in clinical practice. This study explores the potential of functional connectivity (FC)-based searchlight multivariate pattern analysis (CBS-MVPA) to discriminate between schizophrenia patients and healthy controls while also predicting clinical variables.

STUDY DESIGN

We enrolled 112 schizophrenia patients and 119 demographically matched healthy controls. Resting-state functional magnetic resonance imaging data were collected, and whole-brain FC subnetworks were constructed. Additionally, clinical assessments and cognitive evaluations yielded a dataset comprising 36 clinical variables. Finally, CBS-MVPA was utilized to identify subnetworks capable of effectively distinguishing between the patient and control groups and predicting clinical scores.

STUDY RESULTS

The CBS-MVPA approach identified 63 brain subnetworks exhibiting significantly high classification accuracies, ranging from 62.2% to 75.6%, in distinguishing individuals with schizophrenia from healthy controls. Among them, 5 specific subnetworks centered on the dorsolateral superior frontal gyrus, orbital part of inferior frontal gyrus, superior occipital gyrus, hippocampus, and parahippocampal gyrus showed predictive capabilities for clinical variables within the schizophrenia cohort.

CONCLUSION

This study highlights the potential of CBS-MVPA as a valuable tool for localizing the information related to schizophrenia in terms of brain network abnormalities and capturing the relationship between these abnormalities and clinical variables, and thus, deepens our understanding of the neurological mechanisms of schizophrenia.

Predicting clinical outcomes of SARS-CoV-2 infection during the Omicron wave using machine learning

The Omicron SARS-CoV-2 variant continues to strain healthcare systems. Developing tools that facilitate the identification of patients at highest risk of adverse outcomes is a priority. The study objectives are to develop population-scale predictive models that: 1) identify predictors of adverse outcomes with Omicron surge SARS-CoV-2 infections, and 2) predict the impact of prioritized vaccination of high-risk groups for said outcome. We prepared a retrospective longitudinal observational study of a national cohort of 172,814 patients in the U.S. Veteran Health Administration who tested positive for SARS-CoV-2 from January 15 to August 15, 2022. We utilized sociodemographic characteristics, comorbidities, and vaccination status, at time of testing positive for SARS-CoV-2 to predict hospitalization, escalation of care (high-flow oxygen, mechanical ventilation, vasopressor use, dialysis, or extracorporeal membrane oxygenation), and death within 30 days. Machine learning models demonstrated that advanced age, high comorbidity burden, lower body mass index, unvaccinated status, and oral anticoagulant use were the important predictors of hospitalization and escalation of care. Similar factors predicted death. However, anticoagulant use did not predict mortality risk. The all-cause death model showed the highest discrimination (Area Under the Curve (AUC) = 0.903, 95% Confidence Interval (CI): 0.895, 0.911) followed by hospitalization (AUC = 0.822, CI: 0.818, 0.826), then escalation of care (AUC = 0.793, CI: 0.784, 0.805). Assuming a vaccine efficacy range of 70.8 to 78.7%, our simulations projected that targeted prevention in the highest risk group may have reduced 30-day hospitalization and death in more than 2 of 5 unvaccinated patients.

Differential predictability of cognitive profiles from brain structure in older males and females

Structural brain imaging parameters may successfully predict cognitive performance in neurodegenerative diseases but mostly fail to predict cognitive abilities in healthy older adults. One important aspect contributing to this might be sex differences. Behaviorally, older males and females have been found to differ in terms of cognitive profiles, which cannot be captured by examining them as one homogenous group. In the current study, we examined whether the prediction of cognitive performance from brain structure, i.e. region-wise grey matter volume (GMV), would benefit from the investigation of sex-specific cognitive profiles in a large sample of older adults (1000BRAINS; N = 634; age range 55-85 years). Prediction performance was assessed using a machine learning (ML) approach. Targets represented a) a whole-sample cognitive component solution extracted from males and females, and b) sex-specific cognitive components. Results revealed a generally low predictability of cognitive profiles from region-wise GMV. In males, low predictability was observed across both, the whole sample as well as sex-specific cognitive components. In females, however, predictability differences across sex-specific cognitive components were observed, i.e. visual working memory (WM) and executive functions showed higher predictability than fluency and verbal WM. Hence, results accentuated that addressing sex-specific cognitive profiles allowed a more fine-grained investigation of predictability differences, which may not be observable in the prediction of the whole-sample solution. The current findings not only emphasize the need to further investigate the predictive power of each cognitive component, but they also emphasize the importance of sex-specific analyses in older adults.

Data leakage inflates prediction performance in connectome-based machine learning models

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Blood and urine multi-omics analysis of the impact of e-vaping, smoking, and cessation: from exposome to molecular responses

Cigarette smoking is a major preventable cause of morbidity and mortality. While quitting smoking is the best option, switching from cigarettes to non-combustible alternatives (NCAs) such as e-vapor products is a viable harm reduction approach for smokers who would otherwise continue to smoke. A key challenge for the clinical assessment of NCAs is that self-reported product use can be unreliable, compromising the proper evaluation of their risk reduction potential. In this cross-sectional study of 205 healthy volunteers, we combined comprehensive exposure characterization with in-depth multi-omics profiling to compare effects across four study groups: cigarette smokers (CS), e-vapor users (EV), former smokers (FS), and never smokers (NS). Multi-omics analyses included metabolomics, transcriptomics, DNA methylomics, proteomics, and lipidomics. Comparison of the molecular effects between CS and NS recapitulated several previous observations, such as increased inflammatory markers in CS. Generally, FS and EV demonstrated intermediate molecular effects between the NS and CS groups. Stratification of the FS and EV by combustion exposure markers suggested that this position on the spectrum between CS and NS was partially driven by non-compliance/dual use. Overall, this study highlights the importance of in-depth exposure characterization before biological effect characterization for any NCA assessment study.

Machine learning models for diagnosis and risk prediction in eating disorders, depression, and alcohol use disorder

This study uses machine learning models to uncover diagnostic and risk prediction markers for eating disorders (EDs), major depressive disorder (MDD), and alcohol use disorder (AUD). Utilizing case-control samples (ages 18-25 years) and a longitudinal population-based sample (n=1,851), the models, incorporating diverse data domains, achieved high accuracy in classifying EDs, MDD, and AUD from healthy controls. The area under the receiver operating characteristic curves (AUC-ROC [95% CI]) reached 0.92 [0.86-0.97] for AN and 0.91 [0.85-0.96] for BN, without relying on body mass index as a predictor. The classification accuracies for MDD (0.91 [0.88-0.94]) and AUD (0.80 [0.74-0.85]) were also high. Each data domain emerged as accurate classifiers individually, with personality distinguishing AN, BN, and their controls with AUC-ROCs ranging from 0.77 to 0.89. The models demonstrated high transdiagnostic potential, as those trained for EDs were also accurate in classifying AUD and MDD from healthy controls, and vice versa (AUC-ROCs, 0.75-0.93). Shared predictors, such as neuroticism, hopelessness, and symptoms of attention-deficit/hyperactivity disorder, were identified as reliable classifiers. For risk prediction in the longitudinal population sample, the models exhibited moderate performance (AUC-ROCs, 0.64-0.71), highlighting the potential of combining multi-domain data for precise diagnostic and risk prediction applications in psychiatry.

Brain Functional Connectivity and Anatomical Features as Predictors of Cognitive Behavioral Therapy Outcome for Anxiety in Youths

Background

Because pediatric anxiety disorders precede the onset of many other problems, successful prediction of response to the first-line treatment, cognitive-behavioral therapy (CBT), could have major impact. However, existing clinical models are weakly predictive. The current study evaluates whether structural and resting-state functional magnetic resonance imaging can predict post-CBT anxiety symptoms.

Methods

Two datasets were studied: (A) one consisted of n=54 subjects with an anxiety diagnosis, who received 12 weeks of CBT, and (B) one consisted of n=15 subjects treated for 8 weeks. Connectome Predictive Modeling (CPM) was used to predict treatment response, as assessed with the PARS; additionally we investigated models using anatomical features, instead of functional connectivity. The main analysis included network edges positively correlated with treatment outcome, and age, sex, and baseline anxiety severity as predictors. Results from alternative models and analyses also are presented. Model assessments utilized 1000 bootstraps, resulting in a 95% CI for R2, r and mean absolute error (MAE).

Outcomes

The main model showed a mean absolute error of approximately 3.5 (95%CI: [3.1-3.8]) points a R2 of 0.08 [-0.14 - 0.26] and r of 0.38 [0.24 - 0.511]. When testing this model in the left-out sample (B) the results were similar, with a MAE of 3.4 [2.8 - 4.7], R2-0.65 [-2.29 - 0.16] and r of 0.4 [0.24 - 0.54]. The anatomical metrics showed a similar pattern, where models rendered overall low R2.

Interpretation

The analysis showed that models based on earlier promising results failed to predict clinical outcomes. Despite the small sample size, the current study does not support extensive use of CPM to predict outcome in pediatric anxiety.

The effects of data leakage on connectome-based machine learning models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Can we diagnose mental disorders in children? A large-scale assessment of machine learning on structural neuroimaging of 6916 children in the adolescent brain cognitive development study

Background

Prediction of mental disorders based on neuroimaging is an emerging area of research with promising first results in adults. However, research on the unique demographic of children is underrepresented and it is doubtful whether findings obtained on adults can be transferred to children.

Methods

Using data from 6916 children aged 9-10 in the multicenter Adolescent Brain Cognitive Development study, we extracted 136 regional volume and thickness measures from structural magnetic resonance images to rigorously evaluate the capabilities of machine learning to predict 10 different psychiatric disorders: major depressive disorder, bipolar disorder (BD), psychotic symptoms, attention deficit hyperactivity disorder (ADHD), oppositional defiant disorder, conduct disorder, post-traumatic stress disorder, obsessive-compulsive disorder, generalized anxiety disorder, and social anxiety disorder. For each disorder, we performed cross-validation and assessed whether models discovered a true pattern in the data via permutation testing.

Results

Two of 10 disorders can be detected with statistical significance when using advanced models that (i) allow for non-linear relationships between neuroanatomy and disorder, (ii) model interdependencies between disorders, and (iii) avoid confounding due to sociodemographic factors: ADHD (AUROC = 0.567, p = 0.002) and BD (AUROC = 0.551, p = 0.002). In contrast, traditional models perform consistently worse and predict only ADHD with statistical significance (AUROC = 0.529, p = 0.002).

Conclusion

While the modest absolute classification performance does not warrant application in the clinic, our results provide empirical evidence that embracing and explicitly accounting for the complexities of mental disorders via advanced machine learning models can discover patterns that would remain hidden with traditional models.

Predicting long-term outcome in anorexia nervosa: a machine learning analysis of brain structure at different stages of weight recovery

BACKGROUND

Anorexia nervosa (AN) is characterized by sizable, widespread gray matter (GM) reductions in the acutely underweight state. However, evidence for persistent alterations after weight-restoration has been surprisingly scarce despite high relapse rates, frequent transitions to other psychiatric disorders, and generally unfavorable outcome. While most studies investigated brain regions separately (univariate analysis), psychiatric disorders can be conceptualized as brain network disorders characterized by multivariate alterations with only subtle local effects. We tested for persistent multivariate structural brain alterations in weight-restored individuals with a history of AN, investigated their putative biological substrate and relation with 1-year treatment outcome.

METHODS

We trained machine learning models on regional GM measures to classify healthy controls (HC) (N = 289) from individuals at three stages of AN: underweight patients starting intensive treatment (N = 165, used as baseline), patients after partial weight-restoration (N = 115), and former patients after stable and full weight-restoration (N = 89). Alterations after weight-restoration were related to treatment outcome and characterized both anatomically and functionally.

RESULTS

Patients could be classified from HC when underweight (ROC-AUC = 0.90) but also after partial weight-restoration (ROC-AUC = 0.64). Alterations after partial weight-restoration were more pronounced in patients with worse outcome and were not detected in long-term weight-recovered individuals, i.e. those with favorable outcome. These alterations were more pronounced in regions with greater functional connectivity, not merely explained by body mass index, and even increases in cortical thickness were observed (insula, lateral orbitofrontal, temporal pole).

CONCLUSIONS

Analyzing persistent multivariate brain structural alterations after weight-restoration might help to develop personalized interventions after discharge from inpatient treatment.

Computerized analysis of hypomimia and hypokinetic dysarthria for improved diagnosis of Parkinson's disease

Background and Objective

An aging society requires easy-to-use approaches for diagnosis and monitoring of neurodegenerative disorders, such as Parkinson's disease (PD), so that clinicians can effectively adjust a treatment policy and improve patients' quality of life. Current methods of PD diagnosis and monitoring usually require the patients to come to a hospital, where they undergo several neurological and neuropsychological examinations. These examinations are usually time-consuming, expensive, and performed just a few times per year. Hence, this study explores the possibility of fusing computerized analysis of hypomimia and hypokinetic dysarthria (two motor symptoms manifested in the majority of PD patients) with the goal of proposing a new methodology of PD diagnosis that could be easily integrated into mHealth systems.

Methods

We enrolled 73 PD patients and 46 age- and gender-matched healthy controls, who performed several speech/voice tasks while recorded by a microphone and a camera. Acoustic signals were parametrized in the fields of phonation, articulation and prosody. Video recordings of a face were analyzed in terms of facial landmarks movement. Both modalities were consequently modeled by the XGBoost algorithm.

Results

The acoustic analysis enabled diagnosis of PD with 77% balanced accuracy, while in the case of the facial analysis, we observed 81% balanced accuracy. The fusion of both modalities increased the balanced accuracy to 83% (88% sensitivity and 78% specificity). The most informative speech exercise in the multimodality system turned out to be a tongue twister. Additionally, we identified muscle movements that are characteristic of hypomimia.

Conclusions

The introduced methodology, which is based on the myriad of speech exercises likewise audio and video modality, allows for the detection of PD with an accuracy of up to 83%. The speech exercise - tongue twisters occurred to be the most valuable from the clinical point of view. Additionally, the clinical interpretation of the created models is illustrated. The presented computer-supported methodology could serve as an extra tool for neurologists in PD detection and the proposed potential solution of mHealth will facilitate the patient's and doctor's life.

Power and reproducibility in the external validation of brain-phenotype predictions

Identifying reproducible and generalizable brain-phenotype associations is a central goal of neuroimaging. Consistent with this goal, prediction frameworks evaluate brain-phenotype models in unseen data. Most prediction studies train and evaluate a model in the same dataset. However, external validation, or the evaluation of a model in an external dataset, provides a better assessment of robustness and generalizability. Despite the promise of external validation and calls for its usage, the statistical power of such studies has yet to be investigated. In this work, we ran over 60 million simulations across several datasets, phenotypes, and sample sizes to better understand how the sizes of the training and external datasets affect statistical power. We found that prior external validation studies used sample sizes prone to low power, which may lead to false negatives and effect size inflation. Furthermore, increases in the external sample size led to increased simulated power directly following theoretical power curves, whereas changes in the training dataset size offset the simulated power curves. Finally, we compared the performance of a model within a dataset to the external performance. The within-dataset performance was typically within r=0.2 of the cross-dataset performance, which could help decide how to power future external validation studies. Overall, our results illustrate the importance of considering the sample sizes of both the training and external datasets when performing external validation.

Advanced brain age in community-dwelling population with combined physical and cognitive impairments

We investigated whether advanced brain biological age is associated with accelerated age-related physical and/or cognitive functional decline: mobility impairment no disability (MIND), cognitive impairment no dementia (CIND), and physio-cognitive decline syndrome (PCDS). We constructed a brain age prediction model using gray matter features from the magnetic resonance imaging of 1482 healthy individuals (aged 18-92 years). Predicted and chronological age differences were obtained (brain age gap [BAG]) and analyzed in another 1193 community-dwelling population aged ≥50 years. Among the 1193 participants, there were 501, 346, 148, and 198 in the robust, CIND, MIND, and PCDS groups, respectively. Participants with PCDS had significantly larger BAG (BAG = 2.99 ± 8.97) than the robust (BAG = -0.49 ± 9.27, p = 0.002; η2 = 0.014), CIND (BAG = 0.47 ± 9.16, p = 0.02; η2 = 0.01), and MIND (BAG = 0.36 ± 9.69, p = 0.036; η2 = 0.013) groups. Advanced brain aging is involved in the pathophysiology of the co-occurrence of physical and cognitive decline in the older people. The PCDS may be a clinical phenotype reflective of accelerated biological age in community-dwelling older individuals.

Individual risk attitudes arise from noise in neurocognitive magnitude representations

Humans are generally risk averse, preferring smaller certain over larger uncertain outcomes. Economic theories usually explain this by assuming concave utility functions. Here, we provide evidence that risk aversion can also arise from relative underestimation of larger monetary payoffs, a perceptual bias rooted in the noisy logarithmic coding of numerical magnitudes. We confirmed this with psychophysics and functional magnetic resonance imaging, by measuring behavioural and neural acuity of magnitude representations during a magnitude perception task and relating these measures to risk attitudes during separate risky financial decisions. Computational modelling indicated that participants use similar mental magnitude representations in both tasks, with correlated precision across perceptual and risky choices. Participants with more precise magnitude representations in parietal cortex showed less variable behaviour and less risk aversion. Our results highlight that at least some individual characteristics of economic behaviour can reflect capacity limitations in perceptual processing rather than processes that assign subjective values to monetary outcomes.

Accurate sex prediction of cisgender and transgender individuals without brain size bias

The increasing use of machine learning approaches on neuroimaging data comes with the important concern of confounding variables which might lead to biased predictions and in turn spurious conclusions about the relationship between the features and the target. A prominent example is the brain size difference between women and men. This difference in total intracranial volume (TIV) can cause bias when employing machine learning approaches for the investigation of sex differences in brain morphology. A TIV-biased model will not capture qualitative sex differences in brain organization but rather learn to classify an individual's sex based on brain size differences, thus leading to spurious and misleading conclusions, for example when comparing brain morphology between cisgender- and transgender individuals. In this study, TIV bias in sex classification models applied to cis- and transgender individuals was systematically investigated by controlling for TIV either through featurewise confound removal or by matching the training samples for TIV. Our results provide strong evidence that models not biased by TIV can classify the sex of both cis- and transgender individuals with high accuracy, highlighting the importance of appropriate modeling to avoid bias in automated decision making.

A review on multiple sclerosis prognostic findings from imaging, inflammation, and mental health studies

Magnetic resonance imaging (MRI) of the brain is commonly used to detect where chronic and active lesions are in multiple sclerosis (MS). MRI is also extensively used as a tool to calculate and extrapolate brain health by way of volumetric analysis or advanced imaging techniques. In MS patients, psychiatric symptoms are common comorbidities, with depression being the main one. Even though these symptoms are a major determinant of quality of life in MS, they are often overlooked and undertreated. There has been evidence of bidirectional interactions between the course of MS and comorbid psychiatric symptoms. In order to mitigate disability progression in MS, treating psychiatric comorbidities should be investigated and optimized. New research for the prediction of disease states or phenotypes of disability have advanced, primarily due to new technologies and a better understanding of the aging brain.

Questionnaire and structural imaging data accurately predict headache improvement in patients with acute post-traumatic headache attributed to mild traumatic brain injury

BACKGROUND

Our prior work demonstrated that questionnaires assessing psychosocial symptoms have utility for predicting improvement in patients with acute post-traumatic headache following mild traumatic brain injury. In this cohort study, we aimed to determine whether prediction accuracy can be refined by adding structural magnetic resonance imaging (MRI) brain measures to the model.

METHODS

Adults with acute post-traumatic headache (enrolled 0-59 days post-mild traumatic brain injury) underwent T1-weighted brain MRI and completed three questionnaires (Sports Concussion Assessment Tool, Pain Catastrophizing Scale, and the Trait Anxiety Inventory Scale). Individuals with post-traumatic headache completed an electronic headache diary allowing for determination of headache improvement at three- and at six-month follow-up. Questionnaire and MRI measures were used to train prediction models of headache improvement and headache trajectory.

RESULTS

Forty-three patients with post-traumatic headache (mean age = 43.0, SD = 12.4; 27 females/16 males) and 61 healthy controls were enrolled (mean age = 39.1, SD = 12.8; 39 females/22 males). The best model achieved cross-validation Area Under the Curve of 0.801 and 0.805 for predicting headache improvement at three and at six months. The top contributing MRI features for the prediction included curvature and thickness of superior, middle, and inferior temporal, fusiform, inferior parietal, and lateral occipital regions. Patients with post-traumatic headache who did not improve by three months had less thickness and higher curvature measures and notably greater baseline differences in brain structure vs. healthy controls (thickness: p < 0.001, curvature: p = 0.012) than those who had headache improvement.

CONCLUSIONS

A model including clinical questionnaire data and measures of brain structure accurately predicted headache improvement in patients with post-traumatic headache and achieved improvement compared to a model developed using questionnaire data alone.

Stimulus-specific random effects inflate false-positive classification accuracy in multivariate-voxel-pattern-analysis: A solution with generalized mixed-effects modelling

When conducting multivariate-voxel pattern analysis (MVPA), researchers typically compute the average accuracy for each subject and statistically test if the average accuracy is different from the chance level across subjects (by-subject analysis). We argue that this traditional by-subject analysis leads to inflated Type-1 error rates, regardless of the type of machine learning method used (e.g., support vector machine). This is because by-subject analysis does not consider the variance attributed to the idiosyncratic features of the stimuli that have a common influence on all subjects (i.e., the random stimulus effect). As a solution, we proposed the use of generalized linear mixed-effects modelling to evaluate average accuracy. This method only requires post-classification data (i.e., it does not consider the type of classification methods used) and is easily implemented in the analysis pipeline with common statistical software (SPSS, R, Python, etc.). Using both statistical simulation and real fMRI data analysis, we demonstrated that the traditional by-subject method indeed increases Type-1 error rates to a considerable degree, while generalized mixed-effects modelling that incorporates random stimulus effects can indeed maintain the nominal Type-1 error rates.

Long-Term and Meditation-Specific Modulations of Brain Connectivity Revealed Through Multivariate Pattern Analysis

Neuroimaging studies have provided evidence that extensive meditation practice modifies the functional and structural properties of the human brain, such as large-scale brain region interplay. However, it remains unclear how different meditation styles are involved in the modulation of these large-scale brain networks. Here, using machine learning and fMRI functional connectivity, we investigated how focused attention and open monitoring meditation styles impact large-scale brain networks. Specifically, we trained a classifier to predict the meditation style in two groups of subjects: expert Theravada Buddhist monks and novice meditators. We showed that the classifier was able to discriminate the meditation style only in the expert group. Additionally, by inspecting the trained classifier, we observed that the Anterior Salience and the Default Mode networks were relevant for the classification, in line with their theorized involvement in emotion and self-related regulation in meditation. Interestingly, results also highlighted the role of specific couplings between areas crucial for regulating attention and self-awareness as well as areas related to processing and integrating somatosensory information. Finally, we observed a larger involvement of left inter-hemispheric connections in the classification. In conclusion, our work supports the evidence that extensive meditation practice modulates large-scale brain networks, and that the different meditation styles differentially affect connections that subserve style-specific functions.

Adversarial confound regression and uncertainty measurements to classify heterogeneous clinical MRI in Mass General Brigham

In this work, we introduce a novel deep learning architecture, MUCRAN (Multi-Confound Regression Adversarial Network), to train a deep learning model on clinical brain MRI while regressing demographic and technical confounding factors. We trained MUCRAN using 17,076 clinical T1 Axial brain MRIs collected from Massachusetts General Hospital before 2019 and demonstrated that MUCRAN could successfully regress major confounding factors in the vast clinical dataset. We also applied a method for quantifying uncertainty across an ensemble of these models to automatically exclude out-of-distribution data in AD detection. By combining MUCRAN and the uncertainty quantification method, we showed consistent and significant increases in the AD detection accuracy for newly collected MGH data (post-2019; 84.6% with MUCRAN vs. 72.5% without MUCRAN) and for data from other hospitals (90.3% from Brigham and Women's Hospital and 81.0% from other hospitals). MUCRAN offers a generalizable approach for deep-learning-based disease detection in heterogenous clinical data.

Quality standards and recommendations for research in music and neuroplasticity

Research on how music influences brain plasticity has gained momentum in recent years. Considering, however, the nonuniform methodological standards implemented, the findings end up being nonreplicable and less generalizable. To address the need for a standardized baseline of research quality, we gathered all the studies in the music and neuroplasticity field in 2019 and appraised their methodological rigor systematically and critically. The aim was to provide a preliminary and, at the minimum, acceptable quality threshold-and, ipso facto, suggested recommendations-whereupon further discussion and development may take place. Quality appraisal was performed on 89 articles by three independent raters, following a standardized scoring system. The raters' scoring was cross-referenced following an inter-rater reliability measure, and further studied by performing multiple ratings comparisons and matrix analyses. The results for methodological quality were at a quite good level (quantitative articles: mean = 0.737, SD = 0.084; qualitative articles: mean = 0.677, SD = 0.144), following a moderate but statistically significant level of agreement between the raters (W = 0.44, χ² = 117.249, p = 0.020). We conclude that the standards for implementation and reporting are of high quality; however, certain improvements are needed to reach the stringent levels presumed for such an influential interdisciplinary scientific field.

Representations within the Intraparietal Sulcus Distinguish Numerical Tasks and Formats

How does our brain understand the number five when it is written as an Arabic numeral, and when presented as five fingers held up? Four facets have been implicated in adult numerical processing: semantic, visual, manual, and phonological/verbal. Here, we ask how the brain represents each, using a combination of tasks and stimuli. We collected fMRI data from adult participants while they completed our novel "four number code" paradigm. In this paradigm, participants viewed one of two stimulus types to tap into the visual and manual number codes, respectively. Concurrently, they completed one of two tasks to tap into the semantic and phonological/verbal number codes, respectively. Classification analyses revealed that neural codes representing distinctions between the number comparison and phonological tasks were generalizable across format (e.g., Arabic numerals to hands) within intraparietal sulcus (IPS), angular gyrus, and precentral gyrus. Neural codes representing distinctions between formats were generalizable across tasks within visual areas such as fusiform gyrus and calcarine sulcus, as well as within IPS. Our results identify the neural facets of numerical processing within a single paradigm and suggest that IPS is sensitive to distinctions between semantic and phonological/verbal, as well as visual and manual, facets of number representations.

Leveraging Machine Learning for Gaining Neurobiological and Nosological Insights in Psychiatric Research

Much attention is currently devoted to developing diagnostic classifiers for mental disorders. Complementing these efforts, we highlight the potential of machine learning to gain biological insights into the psychopathology and nosology of mental disorders. Studies to this end have mainly used brain imaging data, which can be obtained noninvasively from large cohorts and have repeatedly been argued to reveal potentially intermediate phenotypes. This may become particularly relevant in light of recent efforts to identify magnetic resonance imaging-derived biomarkers that yield insight into pathophysiological processes as well as to refine the taxonomy of mental illness. In particular, the accuracy of machine learning models may be used as dependent variables to identify features relevant to pathophysiology. Moreover, such approaches may help disentangle the dimensional (within diagnosis) and often overlapping (across diagnoses) symptomatology of psychiatric illness. We also point out a multiview perspective that combines data from different sources, bridging molecular and system-level information. Finally, we summarize recent efforts toward a data-driven definition of subtypes or disease entities through unsupervised and semisupervised approaches. The latter, blending unsupervised and supervised concepts, may represent a particularly promising avenue toward dissecting heterogeneous categories. Finally, we raise several technical and conceptual aspects related to the reviewed approaches. In particular, we discuss common pitfalls pertaining to flawed input data or analytic procedures that would likely lead to unreliable outputs.

Confound-leakage: confound removal in machine learning leads to leakage

BACKGROUND

Machine learning (ML) approaches are a crucial component of modern data analysis in many fields, including epidemiology and medicine. Nonlinear ML methods often achieve accurate predictions, for instance, in personalized medicine, as they are capable of modeling complex relationships between features and the target. Problematically, ML models and their predictions can be biased by confounding information present in the features. To remove this spurious signal, researchers often employ featurewise linear confound regression (CR). While this is considered a standard approach for dealing with confounding, possible pitfalls of using CR in ML pipelines are not fully understood.

RESULTS

We provide new evidence that, contrary to general expectations, linear confound regression can increase the risk of confounding when combined with nonlinear ML approaches. Using a simple framework that uses the target as a confound, we show that information leaked via CR can increase null or moderate effects to near-perfect prediction. By shuffling the features, we provide evidence that this increase is indeed due to confound-leakage and not due to revealing of information. We then demonstrate the danger of confound-leakage in a real-world clinical application where the accuracy of predicting attention-deficit/hyperactivity disorder is overestimated using speech-derived features when using depression as a confound.

CONCLUSIONS

Mishandling or even amplifying confounding effects when building ML models due to confound-leakage, as shown, can lead to untrustworthy, biased, and unfair predictions. Our expose of the confound-leakage pitfall and provided guidelines for dealing with it can help create more robust and trustworthy ML models.

Confounds in neuroimaging: A clear case of sex as a confound in brain-based prediction

Muscle weakness is common in many neurological, neuromuscular, and musculoskeletal conditions. Muscle size only partially explains muscle strength as adaptions within the nervous system also contribute to strength. Brain-based biomarkers of neuromuscular function could provide diagnostic, prognostic, and predictive value in treating these disorders. Therefore, we sought to characterize and quantify the brain's contribution to strength by developing multimodal MRI pipelines to predict grip strength. However, the prediction of strength was not straightforward, and we present a case of sex being a clear confound in brain decoding analyses. While each MRI modality-structural MRI (i.e., gray matter morphometry), diffusion MRI (i.e., white matter fractional anisotropy), resting state functional MRI (i.e., functional connectivity), and task-evoked functional MRI (i.e., left or right hand motor task activation)-and a multimodal prediction pipeline demonstrated significant predictive power for strength (R ² = 0.108-0.536, p ≤ 0.001), after correcting for sex, the predictive power was substantially reduced (R ² = -0.038-0.075). Next, we flipped the analysis and demonstrated that each MRI modality and a multimodal prediction pipeline could significantly predict sex (accuracy = 68.0%-93.3%, AUC = 0.780-0.982, p < 0.001). However, correcting the brain features for strength reduced the accuracy for predicting sex (accuracy = 57.3%-69.3%, AUC = 0.615-0.780). Here we demonstrate the effects of sex-correlated confounds in brain-based predictive models across multiple brain MRI modalities for both regression and classification models. We discuss implications of confounds in predictive modeling and the development of brain-based MRI biomarkers, as well as possible strategies to overcome these barriers.

Reactivating ordinal position information from auditory sequence memory in human brains

Retaining a sequence of events in their order is a core ability of many cognitive functions, such as speech recognition, movement control, and episodic memory. Although content representations have been widely studied in working memory (WM), little is known about how ordinal position information of an auditory sequence is retained in the human brain as well as its coding characteristics. In fact, there is still a lack of an efficient approach to directly accessing the stored ordinal position code during WM retention. Here, 31 participants performed an auditory sequence WM task with their brain activities recorded using electroencephalography (EEG). We developed new triggering events that could successfully reactivate neural representations of ordinal position during the delay period. Importantly, the ordinal position reactivation is further related to recognition behavior, confirming its indexing of WM storage. Furthermore, the ordinal position code displays an intriguing “stable-dynamic” format, i.e. undergoing the same dynamic neutral trajectory in the multivariate neural space during both encoding and retention (whenever reactivated). Overall, our results provide an effective approach to accessing the behaviorally-relevant ordinal position information in auditory sequence WM and reveal its new temporal characteristics.

Sample size requirement for achieving multisite harmonization using structural brain MRI features

When data is pooled across multiple sites, the extracted features are confounded by site effects. Harmonization methods attempt to correct these site effects while preserving the biological variability within the features. However, little is known about the sample size requirement for effectively learning the harmonization parameters and their relationship with the increasing number of sites. In this study, we performed experiments to find the minimum sample size required to achieve multisite harmonization (using neuroHarmonize) using volumetric and surface features by leveraging the concept of learning curves. Our first two experiments show that site-effects are effectively removed in a univariate and multivariate manner; however, it is essential to regress the effect of covariates from the harmonized data additionally. Our following two experiments with actual and simulated data showed that the minimum sample size required for achieving harmonization grows with the increasing average Mahalanobis distances between the sites and their reference distribution. We conclude by positing a general framework to understand the site effects using the Mahalanobis distance. Further, we provide insights on the various factors in a cross-validation design to achieve optimal inter-site harmonization.

Feature-space selection with banded ridge regression

Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.

Determining and Validating Population Differences in Magnetic Resonance Angiography Using Sparse Representation

Goal

Identifying population differences can serve as an insightful tool for diagnostic radiology. To do so, a reliable preprocessing framework and data representation are vital.

Methods

We build a machine learning model to visualize gender differences in the circle of Willis (CoW), an integral part of the brain's vasculature. We start with a dataset of 570 individuals and process them for analysis using 389 for the final analysis.

Results

We find statistical differences between male and female patients in one image plane and visualize where they are. We can see differences between the right and left-hand sides of the brain confirmed using Support Vector Machines (SVM).

Conclusion

This process can be applied to detect population variations in the vasculature automatically.

Significance

It can guide debugging and inferring complex machine learning algorithms such as SVM and deep learning models.

Identification of causal effects of neuroanatomy on cognitive decline requires modeling unobserved confounders

INTRODUCTION

Carrying out a randomized controlled trial to estimate the causal effects of regional brain atrophy due to Alzheimer's disease (AD) is impossible. Instead, we must estimate causal effects from observational data. However, this generally requires knowing and having recorded all confounders, which is often unrealistic.

METHODS

We provide an approach that leverages the dependencies among multiple neuroanatomical measures to estimate causal effects from observational neuroimaging data without the need to know and record all confounders.

RESULTS

Our analyses of N = 732 $N=732$ subjects from the Alzheimer's Disease Neuroimaging Initiative demonstrate that using our approach results in biologically meaningful conclusions, whereas ignoring unobserved confounding yields results that conflict with established knowledge on cognitive decline due to AD.

DISCUSSION

The findings provide evidence that the impact of unobserved confounding can be substantial. To ensure trustworthy scientific insights, future AD research can account for unobserved confounding via the proposed approach.

Predicting malaria outbreaks from sea surface temperature variability up to 9 months ahead in Limpopo, South Africa, using machine learning

Malaria is the cause of nearly half a million deaths worldwide each year, posing a great socioeconomic burden. Despite recent progress in understanding the influence of climate on malaria infection rates, climatic sources of predictability remain poorly understood and underexploited. Local weather variability alone provides predictive power at short lead times of 1-2 months, too short to adequately plan intervention measures. Here, we show that tropical climatic variability and associated sea surface temperature over the Pacific and Indian Oceans are valuable for predicting malaria in Limpopo, South Africa, up to three seasons ahead. Climatic precursors of malaria outbreaks are first identified via lag-regression analysis of climate data obtained from reanalysis and observational datasets with respect to the monthly malaria case count data provided from 1998-2020 by the Malaria Institute in Tzaneen, South Africa. Out of 11 sea surface temperature sectors analyzed, two regions, the Indian Ocean and western Pacific Ocean regions, emerge as the most robust precursors. The predictive value of these precursors is demonstrated by training a suite of machine-learning classification models to predict whether malaria case counts are above or below the median historical levels and assessing their skills in providing early warning predictions of malaria incidence with lead times ranging from 1 month to a year. Through the development of this prediction system, we find that past information about SST over the western Pacific Ocean offers impressive prediction skills (~80% accuracy) for up to three seasons (9 months) ahead. SST variability over the tropical Indian Ocean is also found to provide good skills up to two seasons (6 months) ahead. This outcome represents an extension of the effective prediction lead time by about one to two seasons compared to previous prediction systems that were more computationally costly compared to the machine learning techniques used in the current study. It also demonstrates the value of climatic information and the prediction framework developed herein for the early planning of interventions against malaria outbreaks.

Structural differences in adolescent brains can predict alcohol misuse

Alcohol misuse during adolescence (AAM) has been associated with disruptive development of adolescent brains. In this longitudinal machine learning (ML) study, we could predict AAM significantly from brain structure (T1-weighted imaging and DTI) with accuracies of 73 -78% in the IMAGEN dataset (n∼1182). Our results not only show that structural differences in brain can predict AAM, but also suggests that such differences might precede AAM behavior in the data. We predicted 10 phenotypes of AAM at age 22 using brain MRI features at ages 14, 19, and 22. Binge drinking was found to be the most predictable phenotype. The most informative brain features were located in the ventricular CSF, and in white matter tracts of the corpus callosum, internal capsule, and brain stem. In the cortex, they were spread across the occipital, frontal, and temporal lobes and in the cingulate cortex. We also experimented with four different ML models and several confound control techniques. Support Vector Machine (SVM) with rbf kernel and Gradient Boosting consistently performed better than the linear models, linear SVM and Logistic Regression. Our study also demonstrates how the choice of the predicted phenotype, ML model, and confound correction technique are all crucial decisions in an explorative ML study analyzing psychiatric disorders with small effect sizes such as AAM.

Late combination shows that MEG adds to MRI in classifying MCI versus controls

Early detection of Alzheimer's disease (AD) is essential for developing effective treatments. Neuroimaging techniques like Magnetic Resonance Imaging (MRI) have the potential to detect brain changes before symptoms emerge. Structural MRI can detect atrophy related to AD, but it is possible that functional changes are observed even earlier. We therefore examined the potential of Magnetoencephalography (MEG) to detect differences in functional brain activity in people with Mild Cognitive Impairment (MCI) - a state at risk of early AD. We introduce a framework for multimodal combination to ask whether MEG data from a resting-state provides complementary information beyond structural MRI data in the classification of MCI versus controls. More specifically, we used multi-kernel learning of support vector machines to classify 163 MCI cases versus 144 healthy elderly controls from the BioFIND dataset. When using the covariance of planar gradiometer data in the low Gamma range (30-48 Hz), we found that adding a MEG kernel improved classification accuracy above kernels that captured several potential confounds (e.g., age, education, time-of-day, head motion). However, accuracy using MEG alone (68%) was worse than MRI alone (71%). When simply concatenating (normalized) features from MEG and MRI into one kernel (Early combination), there was no advantage of combining MEG with MRI versus MRI alone. When combining kernels of modality-specific features (Intermediate combination), there was an improvement in multimodal classification to 74%. The biggest multimodal improvement however occurred when we combined kernels from the predictions of modality-specific classifiers (Late combination), which achieved 77% accuracy (a reliable improvement in terms of permutation testing). We also explored other MEG features, such as the variance versus covariance of magnetometer versus planar gradiometer data within each of 6 frequency bands (delta, theta, alpha, beta, low gamma, or high gamma), and found that they generally provided complementary information for classification above MRI. We conclude that MEG can improve on the MRI-based classification of MCI.

Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging

Brain imaging research enjoys increasing adoption of supervised machine learning for single-participant disease classification. Yet, the success of these algorithms likely depends on population diversity, including demographic differences and other factors that may be outside of primary scientific interest. Here, we capitalize on propensity scores as a composite confound index to quantify diversity due to major sources of population variation. We delineate the impact of population heterogeneity on the predictive accuracy and pattern stability in 2 separate clinical cohorts: the Autism Brain Imaging Data Exchange (ABIDE, n = 297) and the Healthy Brain Network (HBN, n = 551). Across various analysis scenarios, our results uncover the extent to which cross-validated prediction performances are interlocked with diversity. The instability of extracted brain patterns attributable to diversity is located preferentially in regions part of the default mode network. Collectively, our findings highlight the limitations of prevailing deconfounding practices in mitigating the full consequences of population diversity.

How to remove or control confounds in predictive models, with applications to brain biomarkers

BACKGROUND

With increasing data sizes and more easily available computational methods, neurosciences rely more and more on predictive modeling with machine learning, e.g., to extract disease biomarkers. Yet, a successful prediction may capture a confounding effect correlated with the outcome instead of brain features specific to the outcome of interest. For instance, because patients tend to move more in the scanner than controls, imaging biomarkers of a disease condition may mostly reflect head motion, leading to inefficient use of resources and wrong interpretation of the biomarkers.

RESULTS

Here we study how to adapt statistical methods that control for confounds to predictive modeling settings. We review how to train predictors that are not driven by such spurious effects. We also show how to measure the unbiased predictive accuracy of these biomarkers, based on a confounded dataset. For this purpose, cross-validation must be modified to account for the nuisance effect. To guide understanding and practical recommendations, we apply various strategies to assess predictive models in the presence of confounds on simulated data and population brain imaging settings. Theoretical and empirical studies show that deconfounding should not be applied to the train and test data jointly: modeling the effect of confounds, on the training data only, should instead be decoupled from removing confounds.

CONCLUSIONS

Cross-validation that isolates nuisance effects gives an additional piece of information: confound-free prediction accuracy.

Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks

A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains.

Machine learning approaches for prediction of bipolar disorder based on biological, clinical and neuropsychological markers: a systematic review and meta-analysis

Applying machine learning (ML) to objective markers may overcome prognosis uncertainty due to the subjective nature of the diagnosis of bipolar disorder (BD). This PRISMA-compliant meta-analysis provides new systematic evidence of the BD classification accuracy reached by different markers and ML algorithms. We focused on neuroimaging, electrophysiological techniques, peripheral biomarkers, genetic data, neuropsychological or clinical measures, and multimodal approaches. PubMed, Embase and Scopus were searched through 3^rd December 2020. Meta-analyses were performed using random-effect models. Overall, 81 studies were included in this systematic review and 65 in the meta-analysis (11,336 participants, 3,903 BD). The overall pooled classification accuracy was 0.77 (95%CI[0.75;0.80]). Despite subgroup analyses for diagnostic comparison group, psychiatric disorders, marker, ML algorithm, and validation procedure were not significant, linear discriminant analysis significantly outperformed support vector machine for peripheral biomarkers (p=0.03). Sample size was inversely related to accuracy. Evidence of publication bias was detected. Ultimately, although ML reached a high accuracy in differentiating BD from other psychiatric disorders, best practices in methodology are needed for the advancement of future studies.

Action Intentions, Predictive Processing, and Mind Reading: Turning Goalkeepers Into Penalty Killers

The key to action control is one’s ability to adequately predict the consequences of one’s actions. Predictive processing theories assume that forward models enable rapid “preplay” to assess the match between predicted and intended action effects. Here we propose the novel hypothesis that “reading” another’s action intentions requires a rich forward model of that agent’s action. Such a forward model can be obtained and enriched through learning by either practice or simulation. Based on this notion, we ran a series of studies on soccer goalkeepers and novices, who predicted the intended direction of penalties being kicked at them in a computerized penalty-reading task. In line with hypotheses, extensive practice in penalty kicking improved performance in penalty reading among goalkeepers who had extensive prior experience in penalty blocking but not in penalty kicking. A robust benefit in penalty reading did not result from practice in kinesthetic motor imagery of penalty kicking in novice participants. To test whether goalkeepers actually use such penalty-kicking imagery in penalty reading, we trained a machine-learning classifier on multivariate fMRI activity patterns to distinguish motor-imagery-related from attention-related strategies during a penalty-imagery training task. We then applied that classifier to fMRI data related to a separate penalty-reading task and showed that 2/3 of all correctly read penalty kicks were classified as engaging the motor-imagery circuit rather than merely the attention circuit. This study provides initial evidence that, in order to read our opponent’s action intention, it helps to observe their action kinematics, and use our own forward model to predict the sensory consequences of “our” penalty kick if we were to produce these action kinematics ourselves. In sum, it takes practice as a penalty kicker to become a penalty killer.

NBS-Predict: A prediction-based extension of the network-based statistic

Graph models of the brain hold great promise as a framework to study functional and structural brain connectivity across scales and species. The network-based statistic (NBS) is a well-known tool for performing statistical inference on brain graphs, which controls the family-wise error rate in a mass univariate analysis by combining the cluster-based permutation technique and the graph-theoretical concept of connected components. As the NBS is based on group-level inference statistics, it does not inherently enable informed decisions at the level of individuals, which is, however, necessary for the realm of precision medicine. Here we introduce NBS-Predict, a new approach that combines the powerful features of machine learning (ML) and the NBS in a user-friendly graphical user interface (GUI). By combining ML models with connected components in a cross-validation (CV) structure, the new methodology provides a fast and convenient tool to identify generalizable neuroimaging-based biomarkers. The purpose of this paper is to (i) introduce NBS-Predict and evaluate its performance using two sets of simulated data with known ground truths, (ii) demonstrate the application of NBS-Predict in a real case-control study, including resting-state functional magnetic resonance imaging (rs-fMRI) data acquired from patients with schizophrenia, (iii) evaluate NBS-Predict using rs-fMRI data from the Human Connectome Project 1200 subjects release. We found that: (i) NBS-Predict achieved good statistical power on two sets of simulated data; (ii) NBS-Predict classified schizophrenia with an accuracy of 90% using subjects' functional connectivity matrices and identified a subnetwork with reduced connections in the group with schizophrenia, mainly comprising brain regions localized in frontotemporal, visual, and motor areas, as well as in the subcortex; (iii) NBS-Predict also predicted general intelligence scores from resting-state fMRI connectivity matrices with a prediction score of r = 0.2 and identified a large-scale subnetwork associated with general intelligence. Overall results showed that NBS-Predict performed comparable to or better than pre-existing feature selection algorithms (lasso, elastic net, top 5%, p-value thresholding) and connectome-based predictive modeling (CPM) in terms of identifying relevant features and prediction accuracy.

Hierarchical modelling of functional brain networks in population and individuals from big fMRI data

•

We introduce stochastic PROFUMO (sPROFUMO) for inferring functional brain networks from big data.

•

sPROFUMO hierarchically estimates fMRI networks for the population and every individual.

•

We characterised high dimensional resting state fMRI networks from UK Biobank.

•

Model outperforms ICA and dual regression for estimation of individual-specific network topography.

•

We demonstrate the model's utility for predicting cognitive traits, and capturing subject variability in network topographies versus connectivity.

A major goal of large-scale brain imaging datasets is to provide resources for investigating heterogeneous populations. Characterisation of functional brain networks for individual subjects from these datasets will have an enormous potential for prediction of cognitive or clinical traits. We propose for the first time a technique, Stochastic Probabilistic Functional Modes (sPROFUMO), that is scalable to UK Biobank (UKB) with expected 100,000 participants, and hierarchically estimates functional brain networks in individuals and the population, while allowing for bidirectional flow of information between the two. Using simulations, we show the model's utility, especially in scenarios that involve significant cross-subject variability, or require delineation of fine-grained differences between the networks. Subsequently, by applying the model to resting-state fMRI from 4999 UKB subjects, we mapped resting state networks (RSNs) in single subjects with greater detail than has been possible previously in UKB (>100 RSNs), and demonstrate that these RSNs can predict a range of sensorimotor and higher-level cognitive functions. Furthermore, we demonstrate several advantages of the model over independent component analysis combined with dual-regression (ICA-DR), particularly with respect to the estimation of the spatial configuration of the RSNs and the predictive power for cognitive traits. The proposed model and results can open a new door for future investigations into individualised profiles of brain function from big data.

Neuroplasticity within and between Functional Brain Networks in Mental Training Based on Long-Term Meditation

(1) The effects of intensive mental training based on meditation on the functional and structural organization of the human brain have been addressed by several neuroscientific studies. However, how large-scale connectivity patterns are affected by long-term practice of the main forms of meditation, Focused Attention (FA) and Open Monitoring (OM), as well as by aging, has not yet been elucidated. (2) Using functional Magnetic Resonance Imaging (fMRI) and multivariate pattern analysis, we investigated the impact of meditation expertise and age on functional connectivity patterns in large-scale brain networks during different meditation styles in long-term meditators. (3) The results show that fMRI connectivity patterns in multiple key brain networks can differentially predict the meditation expertise and age of long-term meditators. Expertise-predictive patterns are differently affected by FA and OM, while age-predictive patterns are not influenced by the meditation form. The FA meditation connectivity pattern modulated by expertise included nodes and connections implicated in focusing, sustaining and monitoring attention, while OM patterns included nodes associated with cognitive control and emotion regulation. (4) The study highlights a long-term effect of meditation practice on multivariate patterns of functional brain connectivity and suggests that meditation expertise is associated with specific neuroplastic changes in connectivity patterns within and between multiple brain networks.

Spectral clustering based on structural magnetic resonance imaging and its relationship with major depressive disorder and cognitive ability

There is increasing interest in using data-driven unsupervised methods to identify structural underpinnings of common mental illnesses, including major depressive disorder (MDD) and associated traits such as cognition. However, studies are often limited to severe clinical cases with small sample sizes and most do not include replication. Here, we examine two relatively large samples with structural magnetic resonance imaging (MRI), measures of lifetime MDD and cognitive variables: Generation Scotland (GS subsample, N = 980) and UK Biobank (UKB, N = 8,900), for discovery and replication, using an exploratory approach. Regional measures of FreeSurfer derived cortical thickness (CT), cortical surface area (CSA), cortical volume (CV) and subcortical volume (subCV) were input into a clustering process, controlling for common covariates. The main analysis steps involved constructing participant K-nearest neighbour graphs and graph partitioning with Markov stability to determine optimal clustering of participants. Resultant clusters were (1) checked whether they were replicated in an independent cohort and (2) tested for associations with depression status and cognitive measures. Participants separated into two clusters based on structural brain measurements in GS subsample, with large Cohen's d effect sizes between clusters in higher order cortical regions, commonly associated with executive function and decision making. Clustering was replicated in the UKB sample, with high correlations of cluster effect sizes for CT, CSA, CV and subCV between cohorts across regions. The identified clusters were not significantly different with respect to MDD case-control status in either cohort (GS subsample: p_FDR  = .2239-.6585; UKB: p_FDR  = .2003-.7690). Significant differences in general cognitive ability were, however, found between the clusters for both datasets, for CSA, CV and subCV (GS subsample: d = 0.2529-.3490, p_FDR  < .005; UKB: d = 0.0868-0.1070, p_FDR  < .005). Our results suggest that there are replicable natural groupings of participants based on cortical and subcortical brain measures, which may be related to differences in cognitive performance, but not to the MDD case-control status.

Distinct Neural Representations of Content and Ordinal Structure in Auditory Sequence Memory

Two forms of information, frequency (content) and ordinal position (structure), have to be stored when retaining a sequence of auditory tones in working memory (WM). However, the neural representations and coding characteristics of content and structure, particularly during WM maintenance, remain elusive. Here, in two EEG studies in human participants (both sexes), by transiently perturbing the "activity-silent" WM retention state and decoding the reactivated WM information, we demonstrate that content and structure are stored in a dissociative manner with distinct characteristics throughout WM process. First, each tone in the sequence is associated with two codes in parallel, characterizing its frequency and ordinal position, respectively. Second, during retention, a structural retrocue successfully reactivates structure but not content, whereas a following white noise triggers content but not structure. Third, structure representation remains stable, whereas content code undergoes a dynamic transformation through memory progress. Finally, the noise-triggered content reactivations during retention correlate with subsequent WM behavior. Overall, our results support distinct content and structure representations in auditory WM and provide an efficient approach to access the silently stored WM information in the human brain. The dissociation of content and structure could facilitate efficient memory formation via generalizing stable structure to new auditory contents.SIGNIFICANCE STATEMENT In memory experiences, contents do not exist independently but are linked with each other via ordinal structure. For instance, recalling a piece of favorite music relies on correct ordering (sequence structure) of musical tones (content). How are the structure and content for an auditory temporally structured experience maintained in working memory? Here, by using impulse-response approach and time-resolved representational dissimilarity analysis on human EEG recordings in an auditory working memory task, we reveal that content and structure are stored in a dissociated way, which would facilitate efficient and rapid memory formation through generalizing stable structure knowledge to new auditory inputs.

TAPAS: An Open-Source Software Package for Translational Neuromodeling and Computational Psychiatry

Psychiatry faces fundamental challenges with regard to mechanistically guided differential diagnosis, as well as prediction of clinical trajectories and treatment response of individual patients. This has motivated the genesis of two closely intertwined fields: (i) Translational Neuromodeling (TN), which develops "computational assays" for inferring patient-specific disease processes from neuroimaging, electrophysiological, and behavioral data; and (ii) Computational Psychiatry (CP), with the goal of incorporating computational assays into clinical decision making in everyday practice. In order to serve as objective and reliable tools for clinical routine, computational assays require end-to-end pipelines from raw data (input) to clinically useful information (output). While these are yet to be established in clinical practice, individual components of this general end-to-end pipeline are being developed and made openly available for community use. In this paper, we present the Translational Algorithms for Psychiatry-Advancing Science (TAPAS) software package, an open-source collection of building blocks for computational assays in psychiatry. Collectively, the tools in TAPAS presently cover several important aspects of the desired end-to-end pipeline, including: (i) tailored experimental designs and optimization of measurement strategy prior to data acquisition, (ii) quality control during data acquisition, and (iii) artifact correction, statistical inference, and clinical application after data acquisition. Here, we review the different tools within TAPAS and illustrate how these may help provide a deeper understanding of neural and cognitive mechanisms of disease, with the ultimate goal of establishing automatized pipelines for predictions about individual patients. We hope that the openly available tools in TAPAS will contribute to the further development of TN/CP and facilitate the translation of advances in computational neuroscience into clinically relevant computational assays.

Functional and diffusion MRI reveal the neurophysiological basis of neonates' noxious-stimulus evoked brain activity

Understanding the neurophysiology underlying neonatal responses to noxious stimulation is central to improving early life pain management. In this neonatal multimodal MRI study, we use resting-state and diffusion MRI to investigate inter-individual variability in noxious-stimulus evoked brain activity. We observe that cerebral haemodynamic responses to experimental noxious stimulation can be predicted from separately acquired resting-state brain activity (n = 18). Applying this prediction model to independent Developing Human Connectome Project data (n = 215), we identify negative associations between predicted noxious-stimulus evoked responses and white matter mean diffusivity. These associations are subsequently confirmed in the original noxious stimulation paradigm dataset, validating the prediction model. Here, we observe that noxious-stimulus evoked brain activity in healthy neonates is coupled to resting-state activity and white matter microstructure, that neural features can be used to predict responses to noxious stimulation, and that the dHCP dataset could be utilised for future exploratory research of early life pain system neurophysiology.

Cross-Scanner Harmonization of Neuromelanin-Sensitive MRI for Multisite Studies

BACKGROUND

Neuromelanin-sensitive magnetic resonance imaging (NM-MRI) is a validated measure of neuromelanin concentration in the substantia nigra-ventral tegmental area (SN-VTA) complex and is a proxy measure of dopaminergic function with potential as a noninvasive biomarker. The development of generalizable biomarkers requires large-scale samples necessitating harmonization approaches to combine data collected across sites.

PURPOSE

To develop a method to harmonize NM-MRI across scanners and sites.

STUDY TYPE

Prospective.

POPULATION

A total of 128 healthy subjects (18-73 years old; 45% female) from three sites and five MRI scanners.

FIELD STRENGTH/SEQUENCE

3.0 T; NM-MRI two-dimensional gradient-recalled echo with magnetization-transfer pulse and three-dimensional T1-weighted images.

ASSESSMENT

NM-MRI contrast (contrast-to-noise ratio [CNR]) maps were calculated and CNR values within the SN-VTA (defined previously by manual tracing on a standardized NM-MRI template) were determined before harmonization (raw CNR) and after ComBat harmonization (harmonized CNR). Scanner differences were assessed by calculating the classification accuracy of a support vector machine (SVM). To assess the effect of harmonization on biological variability, support vector regression (SVR) was used to predict age and the difference in goodness-of-fit (Δr) was calculated as the correlation (between actual and predicted ages) for the harmonized CNR minus the correlation for the raw CNR.

STATISTICAL TESTS

Permutation tests were used to determine if SVM classification accuracy was above chance level and if SVR Δr was significant. A P-value <0.05 was considered significant.

RESULTS

In the raw CNR, SVM MRI scanner classification was above chance level (accuracy = 86.5%). In the harmonized CNR, the accuracy of the SVM was at chance level (accuracy = 29.5%; P = 0.8542). There was no significant difference in age prediction using the raw or harmonized CNR (Δr = -0.06; P = 0.7304).

DATA CONCLUSION

ComBat harmonization removes differences in SN-VTA CNR across scanners while preserving biologically meaningful variability associated with age.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY: 1.