Hybrid hyperalignment: A single high-dimensional model of shared information embedded in cortical patterns of response and functional connectivity

Movies reveal the fine-grained organization of infant visual cortex

Studying infant minds with movies is a promising way to increase engagement relative to traditional tasks. However, the spatial specificity and functional significance of movie-evoked activity in infants remains unclear. Here, we investigated what movies can reveal about the organization of the infant visual system. We collected fMRI data from 15 awake infants and toddlers aged 5-23 months who attentively watched a movie. The activity evoked by the movie reflected the functional profile of visual areas. Namely, homotopic areas from the two hemispheres responded similarly to the movie, whereas distinct areas responded dissimilarly, especially across dorsal and ventral visual cortex. Moreover, visual maps that typically require time-intensive and complicated retinotopic mapping could be predicted, albeit imprecisely, from movie-evoked activity in both data-driven analyses (i.e. independent component analysis) at the individual level and by using functional alignment into a common low-dimensional embedding to generalize across participants. These results suggest that the infant visual system is already structured to process dynamic, naturalistic information and that fine-grained cortical organization can be discovered from movie data.

Task relevant autoencoding enhances machine learning for human neuroscience

In human neuroscience, machine learning can help reveal lower-dimensional neural representations relevant to subjects' behavior. However, state-of-the-art models typically require large datasets to train, and so are prone to overfitting on human neuroimaging data that often possess few samples but many input dimensions. Here, we capitalized on the fact that the features we seek in human neuroscience are precisely those relevant to subjects' behavior rather than noise or other irrelevant factors. We thus developed a Task-Relevant Autoencoder via Classifier Enhancement (TRACE) designed to identify behaviorally-relevant target neural patterns. We benchmarked TRACE against a standard autoencoder and other models for two severely truncated machine learning datasets (to match the data typically available in functional magnetic resonance imaging [fMRI] data for an individual subject), then evaluated all models on fMRI data from 59 subjects who observed animals and objects. TRACE outperformed alternative models nearly unilaterally, showing up to 12% increased classification accuracy and up to 56% improvement in discovering "cleaner", task-relevant representations. These results showcase TRACE's potential for a wide variety of data related to human behavior.

Understanding the neural code of stress to control anhedonia

Anhedonia, the diminished drive to seek, value, and learn about rewards, is a core feature of major depressive disorder1-3. The neural underpinnings of anhedonia and how this emotional state drives behaviour remain unclear. Here we investigated the neural code of anhedonia by taking advantage of the fact that when mice are exposed to traumatic social stress, susceptible animals become socially withdrawn and anhedonic, whereas others remain resilient. By performing high-density electrophysiology to record neural activity patterns in the basolateral amygdala (BLA) and ventral CA1 (vCA1), we identified neural signatures of susceptibility and resilience. When mice actively sought rewards, BLA activity in resilient mice showed robust discrimination between reward choices. By contrast, susceptible mice exhibited a rumination-like signature, in which BLA neurons encoded the intention to switch or stay on a previously chosen reward. Manipulation of vCA1 inputs to the BLA in susceptible mice rescued dysfunctional neural dynamics, amplified dynamics associated with resilience, and reversed anhedonic behaviour. Finally, when animals were at rest, the spontaneous BLA activity of susceptible mice showed a greater number of distinct neural population states. This spontaneous activity allowed us to decode group identity and to infer whether a mouse had a history of stress better than behavioural outcomes alone. This work reveals population-level neural dynamics that explain individual differences in responses to traumatic stress, and suggests that modulating vCA1-BLA inputs can enhance resilience by regulating these dynamics.

Domain adaptation in small-scale and heterogeneous biological datasets

Machine-learning models are key to modern biology, yet models trained on one dataset are often not generalizable to other datasets from different cohorts or laboratories due to both technical and biological differences. Domain adaptation, a type of transfer learning, alleviates this problem by aligning different datasets so that models can be applied across them. However, most state-of-the-art domain adaptation methods were designed for large-scale data such as images, whereas biological datasets are smaller and have more features, and these are also complex and heterogeneous. This Review discusses domain adaptation methods in the context of such biological data to inform biologists and guide future domain adaptation research. We describe the benefits and challenges of domain adaptation in biological research and critically explore some of its objectives, strengths, and weaknesses. We argue for the incorporation of domain adaptation techniques to the computational biologist's toolkit, with further development of customized approaches.

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test a number of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral judgments of similarity better reflect neural representational geometry than semantic or visual models throughout much of cortex. The sociality and transitivity models in particular captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, our findings expand the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Traditional neural decoders model the relationship between neural activity and behavior within individual trials of a single experimental session, neglecting correlations across trials and sessions. However, animals exhibit similar neural activities when performing the same behavioral task, and their behaviors are influenced by past experiences from previous trials. To exploit these informative correlations in large datasets, we introduce two complementary models: a multi-session reduced-rank model that shares similar behaviorally-relevant statistical structure in neural activity across sessions to improve decoding, and a multi-session state-space model that shares similar behavioral statistical structure across trials and sessions. Applied across 433 sessions spanning 270 brain regions in the International Brain Laboratory public mouse Neuropixels dataset, our decoders demonstrate improved decoding accuracy for four distinct behaviors compared to traditional approaches. Unlike existing deep learning approaches, our models are interpretable and efficient, uncovering latent behavioral dynamics that govern animal decision-making, quantifying single-neuron contributions to decoding behaviors, and identifying different activation timescales of neural activity across the brain. Code: https://github.com/yzhang511/neural_decoding.

Effects of connectivity hyperalignment (CHA) on estimated brain network properties: from coarse-scale to fine-scale

Recent gains in functional magnetic resonance imaging (fMRI) studies have been driven by increasingly sophisticated statistical and computational techniques and the ability to capture brain data at finer spatial and temporal resolution. These advances allow researchers to develop population-level models of the functional brain representations underlying behavior, performance, clinical status, and prognosis. However, even following conventional preprocessing pipelines, considerable inter-individual disparities in functional localization persist, posing a hurdle to performing compelling population-level inference. Persistent misalignment in functional topography after registration and spatial normalization will reduce power in developing predictive models and biomarkers, reduce the specificity of estimated brain responses and patterns, and provide misleading results on local neural representations and individual differences. This study aims to determine how connectivity hyperalignment (CHA)-an analytic approach for handling functional misalignment-can change estimated functional brain network topologies at various spatial scales from the coarsest set of parcels down to the vertex-level scale. The findings highlight the role of CHA in improving inter-subject similarities, while retaining individual-specific information and idiosyncrasies at finer spatial granularities. This highlights the potential for fine-grained connectivity analysis using this approach to reveal previously unexplored facets of brain structure and function.

Dissociation of Reliability, Heritability, and Predictivity in Coarse- and Fine-Scale Functional Connectomes during Development

The functional connectome supports information transmission through the brain at various spatial scales, from exchange between broad cortical regions to finer-scale, vertex-wise connections that underlie specific information processing mechanisms. In adults, while both the coarse- and fine-scale functional connectomes predict cognition, the fine scale can predict up to twice the variance as the coarse-scale functional connectome. Yet, past brain-wide association studies, particularly using large developmental samples, focus on the coarse connectome to understand the neural underpinnings of individual differences in cognition. Using a large cohort of children (age 9-10 years; n = 1,115 individuals; both sexes; 50% female, including 170 monozygotic and 219 dizygotic twin pairs and 337 unrelated individuals), we examine the reliability, heritability, and behavioral relevance of resting-state functional connectivity computed at different spatial scales. We use connectivity hyperalignment to improve access to reliable fine-scale (vertex-wise) connectivity information and compare the fine-scale connectome with the traditional parcel-wise (coarse scale) functional connectomes. Though individual differences in the fine-scale connectome are more reliable than those in the coarse-scale, they are less heritable. Further, the alignment and scale of connectomes influence their ability to predict behavior, whereby some cognitive traits are equally well predicted by both connectome scales, but other, less heritable cognitive traits are better predicted by the fine-scale connectome. Together, our findings suggest there are dissociable individual differences in information processing represented at different scales of the functional connectome which, in turn, have distinct implications for heritability and cognition.

Identifying and modulating neural signatures of stress susceptibility and resilience enables control of anhedonia

Anhedonia is a core aspect of major depressive disorder. Traditionally viewed as a blunted emotional state in which individuals are unable to experience joy, anhedonia also diminishes the drive to seek rewards and the ability to value and learn about them 1-4.The neural underpinnings of anhedonia and how this emotional state drives related behavioral changes remain unclear. Here, we investigated these questions by taking advantage of the fact that when mice are exposed to traumatic social stress, susceptible animals become socially withdrawn and anhedonic, where they cease to seek high-value rewards, while others remain resilient. By performing high density electrophysiological recordings and comparing neural activity patterns of these groups in the basolateral amygdala (BLA) and ventral CA1 (vCA1) of awake behaving animals, we identified neural signatures of susceptibility and resilience to anhedonia. When animals actively sought rewards, BLA activity in resilient mice showed stronger discrimination between upcoming reward choices. In contrast, susceptible mice displayed a rumination-like signature, where BLA neurons encoded the intention to switch or stay on a previously chosen reward. When animals were at rest, the spontaneous BLA activity of susceptible mice was higher dimensional than in controls, reflecting a greater number of distinct neural population states. Notably, this spontaneous activity allowed us to decode group identity and to infer if a mouse had a history of stress better than behavioral outcomes alone. Finally, targeted manipulation of vCA1 inputs to the BLA in susceptible mice rescued dysfunctional neural dynamics, amplified dynamics associated with resilience, and reversed their anhedonic behavior. This work reveals population-level neural signatures that explain individual differences in responses to traumatic stress, and suggests that modulating vCA1-BLA inputs can enhance resilience by regulating these dynamics.

Cross-movie prediction of individualized functional topography

Participant-specific, functionally defined brain areas are usually mapped with functional localizers and estimated by making contrasts between responses to single categories of input. Naturalistic stimuli engage multiple brain systems in parallel, provide more ecologically plausible estimates of real-world statistics, and are friendly to special populations. The current study shows that cortical functional topographies in individual participants can be estimated with high fidelity from naturalistic stimuli. Importantly, we demonstrate that robust, individualized estimates can be obtained even when participants watched different movies, were scanned with different parameters/scanners, and were sampled from different institutes across the world. Our results create a foundation for future studies that allow researchers to estimate a broad range of functional topographies based on naturalistic movies and a normative database, making it possible to integrate high-level cognitive functions across datasets from laboratories worldwide.

The individualized neural tuning model: Precise and generalizable cartography of functional architecture in individual brains

Quantifying how brain functional architecture differs from person to person is a key challenge in human neuroscience. Current individualized models of brain functional organization are based on brain regions and networks, limiting their use in studying fine-grained vertex-level differences. In this work, we present the individualized neural tuning (INT) model, a fine-grained individualized model of brain functional organization. The INT model is designed to have vertex-level granularity, to capture both representational and topographic differences, and to model stimulus-general neural tuning. Through a series of analyses, we demonstrate that (a) our INT model provides a reliable individualized measure of fine-grained brain functional organization, (b) it accurately predicts individualized brain response patterns to new stimuli, and (c) for many benchmarks, it requires only 10-20 minutes of data for good performance. The high reliability, specificity, precision, and generalizability of our INT model affords new opportunities for building brain-based biomarkers based on naturalistic neuroimaging paradigms.

Neural signatures of stress susceptibility and resilience in the amygdala-hippocampal network

The neural dynamics that underlie divergent anhedonic responses to stress remain unclear. Here, we identified neuronal dynamics in an amygdala-hippocampal circuit that distinguish stress resilience and susceptibility. In a reward-choice task, basolateral amygdala (BLA) activity in resilient mice showed enhanced discrimination of upcoming reward choices. In contrast, a rumination-like signature emerged in the BLA of susceptible mice; a linear decoder could classify the intention to switch or stay on a previously chosen reward. Spontaneous activity in the BLA of susceptible mice was higher dimensional than controls, reflecting the exploration of a larger number of distinct neural states. Manipulation of vCA1-BLA inputs rescued dysfunctional neural dynamics and anhedonia in susceptible mice, suggesting that targeting this pathway can enhance BLA circuit function and ameliorate of depression-related behaviors.

Person-specific and precision neuroimaging: Current methods and future directions

Most neuroimaging studies of brain function analyze data in normalized space to identify regions of common activation across participants. These studies treat interindividual differences in brain organization as noise, but this approach can obscure important information about the brain's functional architecture. Recently, a number of studies have adopted a person-specific approach that aims to characterize these individual differences and explore their reliability and implications for behavior. A subset of these studies has taken a precision imaging approach that collects multiple hours of data from each participant to map brain function on a finer scale. In this review, we provide a broad overview of how person-specific and precision imaging techniques have used resting-state measures to examine individual differences in the brain's organization and their impact on behavior, followed by how task-based activity continues to add detail to these discoveries. We argue that person-specific and precision approaches demonstrate substantial promise in uncovering new details of the brain's functional organization and its relationship to behavior in many areas of cognitive neuroscience. We also discuss some current limitations in this new field and some new directions it may take.