Evidence for a Third Visual Pathway Specialized for Social Perception

Emotion, Motivation, Reasoning, and How Their Brain Systems Are Related

A unified theory of emotion and motivation is updated in which motivational states are states in which instrumental goal-directed actions are performed to obtain anticipated rewards or avoid punishers, and emotional states are states that are elicited when the (conditioned or unconditioned) instrumental reward or punisher is or is not received. This advances our understanding of emotion and motivation, for the same set of genes and associated brain systems can define the primary or unlearned rewards and punishers such as a sweet taste or pain, and the brain systems that learn to expect rewards or punishers and that therefore produce motivational and emotional states. It is argued that instrumental actions under the control of the goal are important for emotion, because they require an intervening emotional state in which an action is learned or performed to obtain the goal, that is, the reward, or to avoid the punisher. The primate including human orbitofrontal cortex computes the reward value, and the anterior cingulate cortex is involved in learning the action to obtain the goal. In contrast, when the instrumental response is overlearned and becomes a habit with stimulus-response associations, emotional states may be less involved. In another route to output, the human orbitofrontal cortex has effective connectivity to the inferior frontal gyrus regions involved in language and provides a route for declarative reports about subjective emotional states to be produced. Reasoning brain systems provide alternative strategies to obtain rewards or avoid punishers and can provide different goals for action compared to emotional systems.

Spatiotemporal processing of real faces is modified by visual sensing

Live human faces, when engaged as visual stimuli, recruit unique and extensive patterns of neural activity. However, the underlying neural mechanisms that underly these live face-to-face processes are not known. We hypothesized that the neural correlates for live face processes are modulated by both spatial and temporal features of the live faces as well as visual sensing parameters. Hemodynamic signals detected by functional near infrared spectroscopy (fNIRS) were acquired concurrently with co-activated electroencephalographic (EEG) and eye-tracking signals during interactive gaze at a live human face or gaze at a human-like robot face. Regression of the fNIRS signals with two eye-gaze variables, fixation duration and dwell time, revealed separate regions of neural correlates, right supramarginal gyrus (lateral visual stream) and right inferior parietal sulcus (dorsal visual stream), respectively. These two areas served as the regions of interest for the EEG analysis. Standardized low-resolution brain electromagnetic tomography (sLORETA) was applied to determine theta (4 - 7 Hz) and alpha (8-13 Hz) oscillatory activity in these regions. Variations in oscillatory patterns corresponding to the neural correlates of the visual sensing parameters suggest an increase in spatial binding for the dorsal relative to the lateral regions of interest during live face-to-face visual stimulation.

Normative structural connectome constrains spreading transient brain activity in generalized epilepsy

BACKGROUND

Genetic generalized epilepsy is characterized by transient episodes of spontaneous abnormal neural activity in anatomically distributed brain regions that ultimately propagate to wider areas. However, the connectome-based mechanisms shaping these abnormalities remain largely unknown. We aimed to investigate how the normative structural connectome constrains abnormal brain activity spread in genetic generalized epilepsy with generalized tonic-clonic seizure (GGE-GTCS).

METHODS

Abnormal transient activity patterns between individuals with GGE-GTCS (n = 97) and healthy controls (n = 141) were estimated from the amplitude of low-frequency fluctuations measured by resting-state functional MRI. The normative structural connectome was derived from diffusion-weighted images acquired in an independent cohort of healthy adults (n = 326). Structural neighborhood analysis was applied to assess the degree of constraints between activity vulnerability and structural connectome. Dominance analysis was used to determine the potential molecular underpinnings of these constraints. Furthermore, a network-based diffusion model was utilized to simulate the spread of pathology and identify potential disease epicenters.

RESULTS

Brain activity abnormalities among patients with GGE-GTCS were primarily located in the temporal, cingulate, prefrontal, and parietal cortices. The collective abnormality of structurally connected neighbors significantly predicted regional activity abnormality, indicating that white matter network architecture constrains aberrant activity patterns. Molecular fingerprints, particularly laminar differentiation and neurotransmitter receptor profiles, constituted key predictors of these connectome-constrained activity abnormalities. Network-based diffusion modeling effectively replicated transient pathological activity spreading patterns, identifying the limbic-temporal, dorsolateral prefrontal, and occipital cortices as putative disease epicenters. These results were robust across different clinical factors and individual patients.

CONCLUSIONS

Our findings suggest that the structural connectome shapes the spatial patterning of brain activity abnormalities, advancing our understanding of the network-level mechanisms underlying vulnerability to abnormal brain activity onset and propagation in GGE-GTCS.

Shared and Unique Neural Codes for Biological Motion Perception in Humans and Macaque Monkeys

Throughout evolution, living organisms have honed the ability to swiftly recognize biological motion (BM) across species. However, how the brain processes within- and cross-species BM, and the evolutionary progression of these processes, remain unclear. To investigate these questions, the current study examined brain activity in the lateral temporal areas of humans and monkeys as they passively observed upright and inverted human and macaque BM stimuli. In humans, the middle temporal area (hMT+) responded to both human and macaque BM stimuli, while the right posterior superior temporal sulcus (hpSTS) exhibited selective responses to human BM stimuli. This selectivity is evidenced by an increased feedforward connection from hMT+ to hpSTS during the processing of human BM stimuli. In monkeys, the MT region processed BM stimuli from both species, but no subregion in the STS anterior to MT is specific to conspecific BM stimuli. A comparison of these findings suggests that upstream brain regions (i.e., MT) may retain homologous functions across species, while downstream brain regions (i.e., STS) may have undergone differentiation and specialization throughout evolution. These results provide insights into the commonalities and differences in the specialized visual pathway engaged in processing within- and cross-species BMs, as well as their functional divergence during evolution.

Dysregulated brain dynamics in the visualmotor network in type 2 diabetes patients and their relationship with cognitive impairment

OBJECTIVE

Type 2 diabetes mellitus (T2DM) is a significant risk factor for mild cognitive impairment (MCI). Here, we identified a T2DM-specific effective connectivity (EC) network, the dynamic features of which could be used to distinguish T2DM patients with MCI from healthy controls (HC) and correlation with cognitive performance.

METHODS

Local and multicentered T2DM patients and matched HC who underwent functional magnetic resonance imaging were recruited. Their static and dynamic effective connectivity were compared. The relationships between connectome characteristics and cognitive performance were also evaluated.

RESULTS

The nodes of the T2DM-related static causality network included the anterior central gyrus, tail of the parahippocampal gyrus, posterior superior temporal sulcus, posterior central parietal lobe, posterior central gyrus and V5 region of the occipital lobe. The V5 region of the visual cortex was the core node. In the multicentered dataset, compared with the HC group, the T2DM with MCI group had significantly greater fractional window and mean dwell time. Fractional windows of the state, which was dominated by the interaction of the nodes from SomMot_Network, Limbic_Network, Default_Network, in the T2DM-specific network increased with poorer cognitive performance in T2DM with MCI patients.

CONCLUSION

Our findings provide insights into the neurobiological mechanisms of the cognitive impairment of T2DM patients from a dynamic network perspective, which may ultimately inform more targeted and effective strategies to prevent MCI.

Visual cortical networks for "What" and "Where" to the human hippocampus revealed with dynamical graphs

Key questions for understanding hippocampal function in memory and navigation in humans are the type and source of visual information that reaches the human hippocampus. We measured bidirectional pairwise effective connectivity with functional magnetic resonance imaging between 360 cortical regions while 956 Human Connectome Project participants viewed scenes, faces, tools, or body parts. We developed a method using deterministic dynamical graphs to define whole cortical networks and the flow in both directions between their cortical regions over timesteps after signal is applied to V1. We revealed that a ventromedial cortical visual "Where" network from V1 via the retrosplenial and medial parahippocampal scene areas reaches the hippocampus when scenes are viewed. A ventrolateral "What" visual cortical network reaches the hippocampus from V1 via V2-V4, the fusiform face cortex, and lateral parahippocampal region TF when faces/objects are viewed. There are major implications for understanding the computations of the human vs rodent hippocampus in memory and navigation: primates with their fovea and highly developed cortical visual processing networks process information about the location of faces, objects, and landmarks in viewed scenes, whereas in rodents the representations in the hippocampal system are mainly about the place where the individual is located and self-motion between places.

Opposing effects of prior information on relational representation and visual cues in dynamic social interaction perception

Social interaction, as a crucial component of relational representation, is essential for understanding human social cognition. While visual cues play a pivotal role in perceiving interactions, little is known about how individuals utilize past visual and interaction-related relational judgments when making decisions under uncertainty. This study investigated how past visual information and interpersonal relational judgments influence the current interaction perception. Participants continuously evaluated the interaction state of two avatars presented at varying distances and facing orientations. The findings revealed a dissociation where the perception of the current interaction state tends to be biased toward past interaction states rather than past distance cues, and this only occurs when the prior interaction information comes from the same sensory modality and is consciously attended to. For the distance cues that contribute to interaction representation, the current distance perception deviates from past distance, even when distance was not explicitly processed. This opposite influence of past information on visual cues and interaction relational representation reflects two independent processing mechanisms of prior information. When dynamically perceiving interpersonal interactions, individuals integrate the repulsive effect of visual cues with the attractive effect of past interaction relations to form stable interaction perception.

From Multimodal Sensorimotor Integration to Semantic Networks: A Phylogenetic Perspective on Speech and Language Evolution

This integrative perspective article delves into the crucial role of the superior temporal sulcus (STS) and adjacent perisylvian regions in multimodal integration and semantic cognition. Drawing from a wide range of neuroscientific evidence, including studies on nonhuman primates and human brain evolution, the article highlights the significance of the STS in linking auditory and visual modalities, particularly in the establishment of associative links between auditory inputs and visual stimuli. Furthermore, it explores the expansion of the human temporal lobe and its implications for the amplification of multisensory regions, emphasizing the role of these regions in the development of word-related concepts and semantic networks. We propose a posteroanterior gradient organization in the human temporal lobe, from low-level sensorimotor integration in posterior regions to higher-order, transmodal semantic control in anterior portions, particularly in the anterior temporal lobe. Overall, this perspective provides a comprehensive overview of the functional and evolutionary aspects of the STS and adjacent regions in multimodal integration and semantic cognition, offering valuable insights for future research in this field.

Disrupted functional connectivity between visual and emotional networks in psychosis risk syndromes through representational similarity analysis

Schizophrenic individuals experience a prolonged prodrome before their first episode, often referred to as Psychosis Risk Syndromes (PRS). The PRS is characterized by non-specific symptoms, yet the underlying neural mechanisms remain unclear. Representational similarity analysis (RSA) has proven effective in elucidating the relationships between different data modalities. This approach could provide valuable insights into the functional coupling between sensory perception and emotion in PRS subjects. In this study, there were 27 PRS subjects and 33 control subjects. Neuropsychological assessments were conducted to evaluate the participants' recent mental states and their risk of mental illness. Each subject underwent task-based functional magnetic resonance imaging (fMRI), which included steady-state visual evoked potentials (SSVEP) and expression matching tasks. The areas of brain activity were defined as regions of interest (ROIs). RSA was used to calculate the relationships between the SSVEP and expression matching tasks. In the functional coupling between the SSVEP at 5 Hz and 10 Hz conditions, the PRS group showed lower functional coupling in the fusiform area compared to controls. Additionally, in the functional coupling between the SSVEP at 10 Hz and the emotion matching conditions, the PRS group demonstrated decreased activation in visual regions compared to controls. Overall, our findings suggest that PRS subjects exhibit diminished functional couplings between basic visual stimuli and vision-emotion matching tasks, indicating abnormal visual processing in both the primary visual cortex and more advanced stages of information processing.

Connectome modeling of discrimination exposure: Impact on your social brain and psychological symptoms

Discrimination is a social stressor that is associated with adverse health outcomes, but the underlying neural mechanisms remain unclear. The fusiform, including the fusiform face area (FFA) plays a critical role in face perception especially regarding hostile faces during discrimination exposure; and are key regions involved in social cognition. We compared resting-state spontaneous activity and connectivity of the fusiform and FFA, between 153 individuals (110 women) with high (N = 73) and low (N = 80) levels of discrimination (measured by the Everyday Discrimination Scale) and evaluated the relationships of these brain signatures with psychological outcomes and stress-related neurotransmitters. Discrimination-related group differences showed altered fusiform signal fluctuation dynamics (Hurst exponent) and connectivity. These alterations predicted discrimination experiences and correlated with anxiety, depression, and cognitive difficulties. A molecular architecture analysis using cross-modal spatial correlation of brain signatures and nuclear imaging derived estimates of stress-related neurotransmitters demonstrated overlap between discrimination-related connectivity and dopamine, serotonin, gamma-aminobutyric acid (GABA), and acetylcholine. Discrimination exposure associated with alterations in the fusiform and face processing area may reflect enhanced baseline preparedness and vigilance towards facial stimuli and decreased top-down regulation of potential threats. These brain alterations may contribute to increased vulnerability for the development of mental health symptoms, demonstrating clinical relevance of social cognition in stressful interpersonal relationships.

Dissociable neural networks for processing fearful bodily expressions at different spatial frequencies

The human brain processes visual input across various spatial frequency (SF) ranges to extract emotional cues. Prior studies have extensively explored SF processing in facial expressions, yielding partly conflicting results. However, bodily expressions, which provide complementary emotional and survival-relevant cues, remain unexplored. We investigated the neural mechanisms underlying the processing of low (LSF), high (HSF), and broad spatial frequency (BSF) components in fearful versus neutral bodily postures. Using functional Magnetic Resonance Imaging, we examined brain activity in 20 participants viewing SF-filtered images of bodily expressions in a semi-passive task. A multivariate "searchlight" analysis based on Multi-Voxel Pattern Analysis was employed to decode the non-linear activation patterns associated with each SF band. Our findings reveal that SF processing engages distinct neural networks in response to fearful bodily expressions. BSF stimuli activated a widespread network, including the amygdala, pulvinar, frontal, and temporal cortices. These findings suggest a general threat-detection system integrating information across all SFs. HSF stimuli engaged cortical regions associated with detailed emotional evaluation and motor planning, such as the orbitofrontal cortex, anterior cingulate cortex, and premotor areas, suggesting that processing fine-grained fear cues involves computationally demanding networks related to emotional resonance and action preparation. In contrast, LSF stimuli primarily activated motor-preparatory regions linked to rapid, action-oriented responses, highlighting the brain prioritization of quick readiness to low-detail threats. Notably, the amygdala showed no SF selectivity, supporting its role as a generalized "relevance detector" in emotional processing. The present study demonstrates that the brain flexibly adapts its SF processing strategy based on the visual details available in fearful bodily expressions, underscoring the complexity and adaptability of emotional processing from bodily signals.

The Bodily Appearance of a Virtual Partner Affects the Activity of the Action Observation and Action Monitoring Systems in a Minimally Interactive Task

One pending question in social neuroscience is whether interpersonal interactions are processed differently by the brain depending on the bodily characteristics of the interactor, i.e., their physical appearance. To address this issue, we engaged participants in a minimally interactive task with an avatar either showing bodily features or not while recording their brain activity using electroencephalography (EEG) in order to investigate indices of action observation and action monitoring processing. Multivariate results showed that bodily compared with nonbodily appearance modulated parieto-occipital neural patterns throughout the entire duration of the observed movement and that, importantly, such patterns differ from the ones related to initial shape processing. Furthermore, among the electrocortical indices of action monitoring, only the early observational positivity (oPe) was responsive to the bodily appearance of the observed agent under the specific task requirement to predict the partner movement. Taken together, these findings broaden the understanding of how bodily appearance shapes the spatiotemporal processing of an interactor's movements. This holds particular relevance in our modern society, where human-artificial (virtual or robotic) agent interactions are rapidly becoming ubiquitous.

Connectivity profile and function of uniquely human cortical areas

Determining the brain specializations unique to humans requires directly comparative anatomical information from other primates, especially our closest relatives. Human (Homo sapiens) (m/f), chimpanzee (Pan troglodytes) (f), and rhesus macaque (Macaca mulatta) (m/f) white matter atlases were used to create connectivity blueprints, i.e., descriptions of the cortical grey matter in terms of the connectivity with homologous white matter tracts. This allowed a quantitative comparative of cortical organization across the species. We identified human-unique connectivity profiles concentrated in temporal and parietal cortices, and hominid-unique organization in prefrontal cortex. Functional decoding revealed human-unique hotspots correlated with language processing and social cognition. Overall, our results counter models that assign primacy to prefrontal cortex for human uniqueness.Significance statement Understanding what makes the human brain unique requires direct comparisons with other primates, particularly our closest relatives. Using connectivity blueprints, we compared to cortical organization of the human to that of the macaque and, for the first time, the chimpanzee. This approach revealed human-specific connectivity patterns in the temporal and parietal lobes, regions linked to language and social cognition. These findings challenge traditional views that prioritize the prefrontal cortex in defining human cognitive uniqueness, emphasizing instead the importance of temporal and parietal cortical evolution in shaping our species' abilities.

Similarities in emotion perception from faces and voices: evidence from emotion sorting tasks

Emotions are expressed via many features including facial displays, vocal intonation, and touch, and perceivers can often interpret emotional displays across the different modalities with high accuracy. Here, we examine how emotion perception from faces and voices relates to one another, probing individual differences in emotion recognition abilities across visual and auditory modalities. We developed a novel emotion sorting task, in which participants were tasked with freely grouping different stimuli into perceived emotional categories, without requiring pre-defined emotion labels. Participants completed two emotion sorting tasks, one using silent videos of facial expressions, the other with audio recordings of vocal expressions. We furthermore manipulated the emotional intensity, contrasting more subtle, lower intensity vs higher intensity emotion portrayals. We find that participants' performance on the emotion sorting task was similar for face and voice stimuli. As expected, performance was lower when stimuli were of low emotional intensity. Consistent with previous reports, we find that task performance was positively correlated across the two modalities. Our findings show that emotion perception in the visual and auditory modalities may be underpinned by similar and/or shared processes, highlighting that emotion sorting tasks are powerful paradigms to investigate emotion recognition from voices, cross-modal and multimodal emotion recognition.

Facial gestures are enacted via a cortical hierarchy of dynamic and stable codes

Successful communication requires the generation and perception of a shared set of signals. Facial gestures are one fundamental set of communicative behaviors in primates, generated through the dynamic arrangement of dozens of fine muscles. While much progress has been made uncovering the neural mechanisms of face perception, little is known about those controlling facial gesture production. Commensurate with the importance of facial gestures in daily social life, anatomical work has shown that facial muscles are under direct control from multiple cortical regions, including primary and premotor in lateral frontal cortex, and cingulate in medial frontal cortex. Furthermore, neuropsychological evidence from focal lesion patients has suggested that lateral cortex controls voluntary movements, and medial emotional expressions. Here we show that lateral and medial cortical face motor regions encode both types of gestures. They do so through unique temporal activity patterns, distinguishable well-prior to movement onset. During gesture production, cortical regions encoded facial kinematics in a context-dependent manner. Our results show how cortical regions projecting in parallel downstream, but each situated at a different level of a posterior-anterior hierarchy form a continuum of gesture coding from dynamic to temporally stable, in order to produce context-related, coherent motor outputs during social communication.

Contrasting the organization of concrete and abstract word meanings

Concepts have traditionally been categorized as either concrete (e.g., ROSE) or abstract (e.g., ROMANCE) based on whether they have a direct connection to external sensory experience or not. However, there is growing consensus that these conceptual categories differ in their reliance on various additional sources of semantic information, such as motor, affective, social, and linguistic experiences, and this is reflected in systematic differences in the semantic properties that typically contribute to their informational content. However, it remains unclear whether concrete and abstract concepts also differ in how their constituent semantic properties relate to one another. To explore this, we compared the organization of 15 semantic dimensions underlying concrete and abstract concept knowledge using data-driven network analyses. We found striking differences in both (1) the centrality of conceptual properties and (2) their pairwise partial correlations. Distinct sensorimotor dimensions emerged as pivotal in organizing each concept type: haptic information for concrete concepts, and interoception and mouth action for abstract concepts. Social content was higher in abstract concepts. However, it played a more central role in structuring concrete meanings, suggesting distinct contributions of social experience to each concept type. Age of acquisition was related exclusively to dimensions quantifying sensorimotor and affective experiences, with sensorimotor properties supporting the acquisition of concrete concepts and affective properties contributing more to the acquisition of abstract concepts. Overall, our findings offer novel insights into the interplay between the diverse sources of semantic information proposed by multiple representation theories in shaping both abstract and concrete concept knowledge.

Tool Representations in Human Visual Cortex

Tools such as pens, forks, and scissors play an important role in many daily-life activities, an importance underscored by the presence in visual cortex of a set of tool-selective brain regions. This review synthesizes decades of neuroimaging research that investigated the representational spaces in the visual ventral stream for objects, such as tools, that are specifically characterized by action-related properties. Overall, results reveal a dissociation between representational spaces in ventral and lateral occipito-temporal cortex (OTC). While lateral OTC encodes both visual (shape) and action-related properties of objects, distinguishing between objects acting as end-effectors (e.g., tools, hands) versus similar noneffector manipulable objects (e.g., a glass), ventral OTC primarily represents objects' visual features such as their surface properties (e.g., material and texture). These areas act in concert with regions outside of OTC to support object interaction and tool use. The parallel investigation of the dimensions underlying object representations in artificial neural networks reveals both the possibilities and the difficulties in capturing the action-related dimensions that distinguish tools from other objects. Although artificial neural networks offer promise as models of visual cortex computations, challenges persist in replicating the action-related dimensions that go beyond mere visual features. Taken together, we propose that regions in OTC support the representation of tools based on a behaviorally relevant action code and suggest future paths to generate a computational model of this object space.

Resting-State Functional Interactions Between the Action Observation Network and the Mentalizing System

Human social functioning is thought to rely on the action observation network (AON) and the mentalizing system (MS). It is debated whether AON and MS are functionally separate or if they interact. To this end, we combined resting-state connectivity with task-based fMRI to characterize the functional connectome within and between these systems. In detail, we computed resting-state connectivity within and between the AON and MS using single subject-defined regions of interest (ROIs). Our results showed a positive coupling between ROIs within each system and negative coupling between the two systems, supporting the existence of two independent networks at rest. Still, two regions (pSTS, aIFG) showed hybrid coupling, connecting with regions of both systems, suggesting that they might mediate cross-network communication. This characterization of the interplay between MS and AON in the healthy brain might provide the starting point to further investigate aberrant "connectivity" fingerprints associated with neuropsychiatric disorders characterized by impairments in social cognition.

Youth Generalized Anxiety and Brain Activation States During Socioemotional Processing

Importance

The brain enters distinct activation states to support differential cognitive and emotional processes, but little is known about how brain activation states differ in youths with clinical anxiety.

Objective

To characterize brain activation states during socioemotional processing (movie stimuli) and assess associations between state characteristics and movie features and anxiety symptoms.

Design, Setting, and Participants

The Healthy Brain Network is an ongoing cross-sectional study of individuals aged 5 to 21 years experiencing difficulties in school, of whom approximately 45% met criteria for a lifetime anxiety disorder diagnosis. Data used in this study are from the first 9 releases (collected in a nonclinical research setting in the New York City metropolitan area from 2015 to 2020) and include 620 youths aged 5 to 15 years (53% of whom met criteria for a lifetime anxiety disorder diagnosis) who watched an emotional video during functional magnetic resonance imaging and completed questionnaires and clinical evaluation. Of those with functional magnetic resonance imaging data, 432 youths aged 7 to 15 years also self-reported on anxiety symptoms. Data were processed and analyzed between February 2020 and August 2024.

Main Outcomes and Measures

A hidden Markov model was trained to identify brain activation states across participants during video watching. Time spent in each state and the moment-to-moment probability of being in each state were extracted. Videos were annotated for emotion-specific and nonspecific information using the EmoCodes system. Self-reported anxiety symptoms were assessed using the Screen for Child Anxiety Related Disorders. Time spent in each state across the video and during and outside of peaks in negative content correlated with generalized and social anxiety scores.

Results

Among the 620 youths in the overall analysis, 369 were male and the mean (SD) age was 10.4 (2.8) years. In the anxiety symptom analysis, 263 of 432 youths were male and the mean (SD) age was 11.5 (2.2) years. Three brain activation states were identified: a high somatomotor activation state (state 1), a high cingulo-opercular network activation state (state 2), and a high ventral attention and default mode state (state 3). The probability of being in state 3 was correlated with video content that was more negative, quieter, and with less visual motion (ρ < 0.08; P < .001). Increased generalized anxiety was associated with greater time in state 3 (B, 0.10; 95% CI, 0.01 to 0.20; false discovery rate [FDR]-corrected P = .048) and less time in state 2 (B, -0.11; 95% CI, -0.21 to -0.02; FDR-corrected P = .048) when negative social cues were present.

Conclusions and Relevance

Youths entered 3 distinct brain activation states during movie watching, and youths with anxiety spent more time in a state with high ventral attention and default activation during negative socioemotional processing. Youths high in generalized anxiety may be more engaged in deeply processing negative emotional content, which may influence self-regulation. Interventions that focus on changing physiological and psychological state during negative social interactions in youths with anxiety should be considered.

Decoding the physics of observed actions in the human brain

Recognizing goal-directed actions is a computationally challenging task, requiring not only the visual analysis of body movements, but also analysis of how these movements causally impact, and thereby induce a change in, those objects targeted by an action. We tested the hypothesis that the analysis of body movements and the effects they induce relies on distinct neural representations in superior and anterior inferior parietal lobe (SPL and aIPL). In four fMRI sessions, participants observed videos of actions (e.g. breaking stick, squashing plastic bottle) along with corresponding point-light-display (PLD) stick figures, pantomimes, and abstract animations of agent-object interactions (e.g. dividing or compressing a circle). Cross-decoding between actions and animations revealed that aIPL encodes abstract representations of action effect structures independent of motion and object identity. By contrast, cross-decoding between actions and PLDs revealed that SPL is disproportionally tuned to body movements independent of visible interactions with objects. Lateral occipitotemporal cortex (LOTC) was sensitive to both action effects and body movements. These results demonstrate that parietal cortex and LOTC are tuned to physical action features, such as how body parts move in space relative to each other and how body parts interact with objects to induce a change (e.g. in position or shape/configuration). The high level of abstraction revealed by cross-decoding suggests a general neural code supporting mechanical reasoning about how entities interact with, and have effects on, each other.

How eccentricity modulates attention capture by direct face/gaze and sudden onset motion

We investigated how processing benefits for direct face/gaze and sudden onset motion depend on stimulus presentation location, specifically eccentricity from fixation. Participants responded to targets that were presented on one of four stimuli that displayed a direct or averted face and gaze either statically or suddenly. Between participants, stimuli were presented at different eccentricities relative to central fixation, spanning 3.3°, 4.3°, 5.5° or 6.5° of the visual field. Replicating previous studies, we found processing advantages for direct (vs. averted) face/gaze and motion onset (vs. static stimuli). Critically, while the motion-onset advantage increased with increasing distance to the center, the face/gaze direction advantage was not significantly modulated by target eccentricity. Results from a control experiment with eye tracking indicate that face/gaze direction could be accurately discriminated even at the largest eccentricity. These findings demonstrate a distinction between the processing of basic facial and gaze signals and exogenous motion cues, which may be based on functional differences between central and peripheral retinal regions. Moreover, the results highlight the importance of taking specific stimulus properties into account when studying perception and attention in the periphery.

What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

Origins of food selectivity in human visual cortex

Several recent studies, enabled by advances in neuroimaging methods and large-scale datasets, have identified areas in human ventral visual cortex that respond more strongly to food images than to images of many other categories, adding to our knowledge about the broad network of regions that are responsive to food. This finding raises important questions about the evolutionary and developmental origins of a possible food-selective neural population, as well as larger questions about the origins of category-selective neural populations more generally. Here, we propose a framework for how visual properties of food (particularly color) and nonvisual signals associated with multimodal reward processing, social cognition, and physical interactions with food may, in combination, contribute to the emergence of food selectivity. We discuss recent research that sheds light on each of these factors, alongside a broader account of category selectivity that incorporates both visual feature statistics and behavioral relevance.

Brain network alterations in anorexia Nervosa: A Multi-Center structural connectivity study

Anorexia nervosa (AN) is a severe eating disorder characterized by intense fear of weight gain, distorted body image, and extreme food restriction. This research employed advanced diffusion MRI techniques including single-shell 3-tissue constrained spherical deconvolution, anatomically constrained tractography, and spherical deconvolution informed filtering of tractograms to analyze brain network alterations in AN. Diffusion MRI data from 81 AN patients and 98 healthy controls were obtained. The structural brain connectome was constructed based on nodes set in 84 brain regions, and graph theory analysis was conducted. Results showed that AN patients exhibited significantly higher clustering coefficient and local efficiency in several brain regions, including the left fusiform gyrus, bilateral orbitofrontal cortex, right entorhinal cortex, right lateral occipital gyrus, right superior temporal gyrus, and right insula. A trend towards higher global efficiency and small-worldness was also observed in AN patients, although not statistically significant. These findings suggest increased local connectivity and efficiency within regions associated with behavioral rigidity, emotional regulation, and disturbed body image among AN patients. This study contributes to the understanding of the neurological basis of AN by highlighting structural connectivity alterations in specific brain regions.

A double dissociation between semantic and spatial cognition in visual to default network pathways

Processing pathways between sensory and default mode network (DMN) regions support recognition, navigation, and memory but their organisation is not well understood. We show that functional subdivisions of visual cortex and DMN sit at opposing ends of parallel streams of information processing that support visually mediated semantic and spatial cognition, providing convergent evidence from univariate and multivariate task responses, intrinsic functional and structural connectivity. Participants learned virtual environments consisting of buildings populated with objects, drawn from either a single semantic category or multiple categories. Later, they made semantic and spatial context decisions about these objects and buildings during functional magnetic resonance imaging. A lateral ventral occipital to fronto-temporal DMN pathway was primarily engaged by semantic judgements, while a medial visual to medial temporal DMN pathway supported spatial context judgements. These pathways had distinctive locations in functional connectivity space: the semantic pathway was both further from unimodal systems and more balanced between visual and auditory-motor regions compared with the spatial pathway. When semantic and spatial context information could be integrated (in buildings containing objects from a single category), regions at the intersection of these pathways responded, suggesting that parallel processing streams interact at multiple levels of the cortical hierarchy to produce coherent memory-guided cognition.

An intrinsic hierarchical, retinotopic organization of visual pulvinar connectivity in the human neonate

The thalamus plays a crucial role in the development of the neocortex, with the pulvinar being particularly important for visual development due to its involvement in various functions that emerge early in infancy. The development of connections between the pulvinar and the cortex constrains its role in infant visual processing and the maturation of associated cortical networks. However, the extent to which adult-like pulvino-cortical pathways are present at birth remains largely unknown, limiting our understanding of how the thalamus may support early vision. To address this gap, we investigated the organization of pulvino-cortical connections in human neonates using probabilistic tractography analyses on diffusion imaging data. Our analyses identified white matter pathways between the pulvinar and areas across occipital, ventral, lateral, and dorsal visual cortices at birth. These pathways exhibited specificity in their connections within the pulvinar, reflecting both an intra-areal retinotopic organization and a hierarchical structure across areas of visual cortical pathways. This organization suggests that even at birth, the pulvinar could facilitate detailed processing of sensory information and communication between distinct processing pathways. Comparative analyses revealed that while the large-scale organization of pulvino-cortical connectivity in neonates mirrored that of adults, connectivity with the ventral visual cortex was less mature than other cortical pathways, consistent with the protracted development of the visual recognition pathway. These findings advance our understanding of the developmental trajectory of thalamocortical connections and provide a framework for how subcortical structures may support early perceptual abilities and scaffold the development of cortex.

Shared and individual tuning curves for social vision

A stimulus with light is clearly visual; a stimulus with sound is clearly auditory. But what makes a stimulus "social", and how do judgments of socialness differ across people? Here, we characterize both group-level and individual thresholds for perceiving the presence and nature of a social interaction. We take advantage of the fact that humans are primed to see social interactions-e.g., chasing, playing, fighting-even in very un-lifelike stimuli such as animations of geometric shapes. Unlike prior work using these stimuli, we exploit their most advantageous property, which is that their visual features are fully parameterizable. We use this property to construct psychophysics-inspired "social tuning curves" for individual subjects. Social tuning curves are stable within individuals, unique across individuals, and show some relationship to socio-affective traits. Results support the view that social information processing begins early in the perceptual hierarchy. Further, our approach lays the foundation for a generative account of social perception in single subjects.

Connectional differences between humans and macaques in the MT+ complex

MT+ is pivotal in the dorsal visual stream, encoding tool-use characteristics such as motion speed and direction. Despite its conservation between humans and monkeys, differences in MT+ spatial location and organization may lead to divergent, yet unexplored, connectivity patterns and functional characteristics. Using diffusion tensor imaging, we examined the structural connectivity of MT+ subregions in macaques and humans. We also employed graph-theoretical analyses on the constructed homologous tool-use network to assess their functional roles. Our results revealed location-dependent connectivity in macaques, with MST, MT, and FST predominantly connected to dorsal, middle, and ventral surfaces, respectively. Humans showed similar connectivity across all subregions. Differences in connectivity between MST and FST are more pronounced in macaques. In humans, the entire MT+ region, especially MST, exhibited stronger information transmission capabilities. Our findings suggest that the differences in tool use between humans and macaques may originate earlier than previously thought, particularly within the MT+ region.

Visuospatial computations vary by category and stream and continue to develop in adolescence

Reading, face recognition, and navigation are supported by visuospatial computations in category-selective regions across ventral, lateral, and dorsal visual streams. However, the nature of visuospatial computations across streams and their development in adolescence remain unknown. Using fMRI and population receptive field (pRF) modeling in adolescents and adults, we estimate pRFs in high-level visual cortex and determine their development. Results reveal that pRF location, size, and visual field coverage vary across category, stream, and hemisphere in both adolescents and adults. While pRF location is mature by adolescence, pRF size and visual field coverage continue to develop - increasing in face-selective and decreasing in place-selective regions - alongside similar development of category selectivity. These findings provide a timeline for differential development of visual functions and suggest that visuospatial computations in high-level visual cortex continue to be optimized to accommodate both category and stream demands through adolescence.

An Eccentricity Gradient Reversal across High-Level Visual Cortex

Human visual cortex contains regions selectively involved in perceiving and recognizing ecologically important visual stimuli such as people and places. Located in the ventral temporal lobe, these regions are organized consistently relative to cortical folding, a phenomenon thought to be inherited from how centrally or peripherally these stimuli are viewed with the retina. While this eccentricity theory of visual cortex has been one of the best descriptions of its functional organization, whether or not it accurately describes visual processing in all category-selective regions is not yet clear. Through a combination of behavioral and functional MRI measurements in 27 participants (17 females), we demonstrate that a limb-selective region neighboring well-studied face-selective regions shows tuning for the visual periphery in a cortical region originally thought to be centrally biased. We demonstrate that the spatial computations performed by the limb-selective region are consistent with visual experience and in doing so, make the novel observation that there may in fact be two eccentricity gradients, forming an eccentricity reversal across high-level visual cortex. These data expand the current theory of cortical organization to provide a unifying principle that explains the broad functional features of many visual regions, showing that viewing experience interacts with innate wiring principles to drive the location of cortical specialization.

Brain Activation and Aberrant Effective Connectivity in the Mentalizing Network of Preadolescent Children at Familial High Risk of Schizophrenia or Bipolar Disorder

BACKGROUND

Schizophrenia and bipolar disorder are characterized by social cognitive impairments, and recent research has identified alterations of the social brain. However, it is unknown whether familial high risk (FHR) of these disorders is associated with neurobiological alterations already present in childhood.

METHODS

As part of the Danish High Risk and Resilience Study-VIA 11, we examined children at FHR of schizophrenia (n = 121, 50% female) or bipolar disorder (n = 75, 47% female) and population-based control children (PBCs) (n = 128, 48% female). Using functional magnetic resonance imaging and dynamic causal modeling, we investigated brain activation and effective connectivity during the social cognition paradigm from the Human Connectome Project.

RESULTS

We found similar activation of the mentalizing network across groups, including visual area V5, the dorsomedial prefrontal cortex, and the posterior superior temporal sulcus (pSTS). Nonetheless, both FHR groups showed aberrant brain connectivity in the form of increased feedforward connectivity from left V5 to pSTS compared with PBCs. Children at FHR of schizophrenia had reduced intrinsic connectivity in bilateral V5 compared with PBCs, whereas children at FHR of bipolar disorder showed increased reciprocal connectivity between the left dorsomedial prefrontal cortex and the pSTS, increased intrinsic connectivity in the right pSTS, and reduced feedforward connectivity from the right pSTS to the dorsomedial prefrontal cortex compared with PBCs.

CONCLUSIONS

Our results provide first-time evidence of aberrant brain connectivity in the mentalizing network of children at FHR of schizophrenia or FHR of bipolar disorder. Longitudinal research is warranted to clarify whether aberrant brain connectivity during mentalizing constitutes an endophenotype associated with the development of a mental disorder later in life.

Are there cortical somatotopic motor maps outside of the human precentral gyrus?

Human body movements are supported by a somatotopic map, primary motor cortex (M1), that is found along the precentral gyrus. Recent evidence has suggested two further motor maps that span the lateral occipitotemporal cortex (LOTC) and the precuneus. Confirmation of these maps is important, as they influence our understanding of the organization of motor behavior, for example by revealing how visual- and motor-related activity interact. However, evidence for these recently proposed maps is limited. We analyzed an open functional MRI (fMRI) dataset of 62 participants who performed 12 different body part movements. We analyzed the magnitude of responses evoked by movements with novel quantitative indices that test for maplike organization. We found strong evidence for bilateral somatotopic maps in precentral and postcentral gyri. In LOTC, we found much weaker responses to movement and little evidence of somatotopy. In the precuneus, we found only limited evidence for somatotopy. We also adopted a background connectivity approach to examine correlations between M1, LOTC, and the precuneus in the residual time series data. This revealed a ventral-posterior/dorsal-anterior distinction in the connectivity between precuneus and M1, favoring the head and arms, respectively. Posterior right hemisphere LOTC showed some evidence of preferential connectivity to arm-selective regions of M1. Overall, our results do not support the existence of a somatotopic motor map in LOTC but provide some support for a coarse map in the precuneus, especially as revealed in connectivity patterns. These findings help clarify the organization of human motor representations beyond the precentral gyrus.NEW & NOTEWORTHY We investigated previous claims about the existence of somatotopic motor maps in the human lateral occipitotemporal cortex (LOTC) and the precuneus, in comparison to known maps in the precentral and postcentral gyri. Consistent with previous findings, we identified clear somatotopic motor maps in the latter two regions. With multiple quantitative measures of activity and connectivity, however, we found no evidence for a map in the LOTC and limited evidence for a map in the precuneus.

Hippocampal Discoveries: Spatial View Cells, Connectivity, and Computations for Memory and Navigation, in Primates Including Humans

Two key series of discoveries about the hippocampus are described. One is the discovery of hippocampal spatial view cells in primates. This discovery opens the way to a much better understanding of human episodic memory, for episodic memory prototypically involves a memory of where people or objects or rewards have been seen in locations "out there" which could never be implemented by the place cells that encode the location of a rat or mouse. Further, spatial view cells are valuable for navigation using vision and viewed landmarks, and provide for much richer, vision-based, navigation than the place to place self-motion update performed by rats and mice who live in dark underground tunnels. Spatial view cells thus offer a revolution in our understanding of the functions of the hippocampus in memory and navigation in humans and other primates with well-developed foveate vision. The second discovery describes a computational theory of the hippocampal-neocortical memory system that includes the only quantitative theory of how information is recalled from the hippocampus to the neocortex. It is shown how foundations for this research were the discovery of reward neurons for food reward, and non-reward, in the primate orbitofrontal cortex, and representations of value including of monetary value in the human orbitofrontal cortex; and the discovery of face identity and face expression cells in the primate inferior temporal visual cortex and how they represent transform-invariant information. This research illustrates how in order to understand a brain computation, a whole series of integrated interdisciplinary discoveries is needed to build a theory of the operation of each neural system.

Early Neural Development of Social Interaction Perception: Evidence from Voxel-Wise Encoding in Young Children and Adults

From a young age, children have advanced social perceptual and reasoning abilities. However, the neural development of these abilities is still poorly understood. To address this gap, we used fMRI data collected while 122 3-12-year-old children (64 females) and 33 adults (20 females) watched an engaging and socially rich movie to investigate how the cortical basis of social processing changes throughout development. We labeled the movie with visual and social features, including motion energy, presence of a face and a social interaction, theory of mind (ToM) events, valence, and arousal. Using a voxel-wise encoding model trained on these features, we found that models based on visual (motion energy) and social (faces, social interaction, ToM, valence, and arousal) features can both predict brain activity in children as young as 3 years old across the cortex, with particularly high predictivity in motion-selective middle temporal region and the superior temporal sulcus (STS). Furthermore, models based on individual social features showed that while there may be some development throughout childhood, social interaction information in the STS is present in children as young as 3 years old and appears adult-like by age 7. The current study, for the first time, links neural activity in children to predefined social features in a narrative movie and suggests social interaction perception is supported by early developing neural responses in the STS.

COMMON AND DISTINCT NEURAL CORRELATES OF SOCIAL INTERACTION PERCEPTION AND THEORY OF MIND

Social cognition spans from perceiving agents and their interactions to making inferences based on theory of mind (ToM). Despite their frequent co-occurrence in real life, the commonality and distinction between social interaction perception and ToM at behavioral and neural levels remain unclear. Here, participants (N = 231) provided moment-by-moment ratings of four text and four audio narratives on social interactions and ToM engagement. Social interaction and ToM ratings were reliable (split-half r = .98 and .92, respectively) but only modestly correlated across time (r = .32). In a second sample (N = 90), we analyzed co-variation between normative social interaction and ToM ratings and functional magnetic resonance (fMRI) activity during narrative reading (text) and listening (audio). Social interaction perception and ToM activity maps generalized across text and audio presentation (r = .83 and .57 between unthresholded t maps, respectively). When ToM was held constant, merely perceiving social interactions activated all regions canonically associated with ToM under both modalities (FDR q < .01), including temporoparietal junction, superior temporal sulcus, medial prefrontal cortex, and precuneus. ToM activated these regions as well, indicating a shared, modality-general system for social interaction perception and ToM. Furthermore, ToM uniquely engaged lateral occipitotemporal cortex, left anterior intraparietal sulcus, and right premotor cortex. These results imply that perceiving social interactions automatically engages regions implicated in mental state inferences. In addition, ToM is distinct from social interaction perception in its recruitment of regions associated with higher-level cognitive processes, including action understanding and executive functions.

Paranoid and teleological thinking give rise to distinct social hallucinations in vision

Paranoia (believing others intend harm) and excess teleological thinking (ascribing too much purpose) are non-consensual beliefs about agents. Human vision rapidly detects agents and their intentions. Might paranoia and teleology have roots in visual perception? Using displays that evoke the impression that one disc ('wolf') is chasing another ('sheep'), we find that paranoia and teleology involve perceiving chasing when there is none (studies 1 and 2) - errors we characterize as social hallucinations. When asked to identify the wolf or the sheep (studies 3, 4a, and 4b), we find high-paranoia participants struggled to identify sheep, while high-teleology participants were impaired at identifying wolves - both despite high-confidence. Both types of errors correlated with hallucinatory percepts in the real world. Although paranoia and teleology both involve excess perception of agency, the current results collectively suggest a perceptual distinction between the two, perhaps with clinical import.

Neurosurgical and BCI approaches to visual rehabilitation in occipital lobe tumor patients

This study investigates the effects of occipital lobe tumors on visual processing and the role of brain-computer interface (BCI) technologies in post-surgical visual rehabilitation. Through a combination of pre-surgical functional magnetic resonance imaging (fMRI) and Diffusion Tensor Imaging (DTI), intra-operative direct cortical stimulation (DCS) and Electrocorticography (ECoG), and post-surgical BCI interventions, we provide insight into the complex dynamics between occipital lobe tumors and visual function. Our results highlight a discrepancy between clinical assessments of visual field damage and the patient's reported visual experiences, suggesting a residual functional capacity within the damaged occipital regions. Additionally, the absence of expected visual phenomena during surgery and the promising outcomes from BCI-driven rehabilitation underscore the complexity of visual processing and the potential of technology-enhanced rehabilitation strategies. This work emphasizes the need for an interdisciplinary approach in developing effective treatments for visual impairments related to brain tumors, illustrating the significant implications for neurosurgical practices and the advancement of rehabilitation sciences.

A gradient of hemisphere-specific dorsal to ventral processing routes in parieto-premotor networks

Networks in the parietal and premotor cortices enable essential human abilities regarding motor processing, including attention and tool use. Even though our knowledge on its topography has steadily increased, a detailed picture of hemisphere-specific integrating pathways is still lacking. With the help of multishell diffusion magnetic resonance imaging, probabilistic tractography, and the Graph Theory Analysis, we investigated connectivity patterns between frontal premotor and posterior parietal brain areas in healthy individuals. With a two-stage node characterization approach, we defined the network role of precisely mapped cortical regions from the Julich-Brain atlas. We found evidence for a third, left-sided, medio-dorsal subpathway in a successively graded dorsal stream, referencing more specialized motor processing on the left. Supplementary motor areas had a strongly lateralized connectivity to either left dorsal or right ventral parietal domains, representing an action-attention dichotomy between hemispheres. The left sulcal parietal regions primarily coupled with areas 44 and 45, mirrored by the inferior frontal junction (IFJ) on the right, a structural lateralization we termed as "Broca's-IFJ switch." We were able to deepen knowledge on gyral and sulcal pathways as well as domain-specific contributions in parieto-premotor networks. Our study sheds new light on the complex lateralization of cortical routes for motor activity in the human brain.

Exploring neural architectures for simultaneously recognizing multiple visual attributes

Much experimental evidence in neuroscience has suggested a division of higher visual processing into a ventral pathway specialized for object recognition and a dorsal pathway specialized for spatial recognition. Previous computational studies have suggested that neural networks with two segregated pathways (branches) have better performance in visual recognition tasks than neural networks with a single pathway (branch). One previously proposed possibility is that two pathways increase the learning efficiency of a network by allowing separate networks to process information about different visual attributes separately. However, most of these previous studies were limited, considering recognition of only two visual attributes, identity and location, simultaneously with a restricted number of classes in each attribute. We investigate whether it is always advantageous to use two-pathway networks when recognizing other visual attributes as well as examine whether the advantage of using two-pathway networks would be different when there are a different number of classes in each attribute. We find that it is always advantageous to use segregated pathways to process different visual attributes separately, with this advantage increasing with a greater number of classes. Thus, using a computational approach, we demonstrate that it is computationally advantageous to have separate pathways if the amount of variations of a given visual attribute is high or that attribute needs to be finely discriminated. Hence, when the size of the computer vision model is limited, designing a segregated pathway (branch) for a given visual attribute should only be used when it is computationally advantageous to do so.

The Development of Socially Directed Attention: A Functional Magnetic Resonance Imaging Study in Infant Monkeys

Socially guided visual attention, such as gaze following and joint attention, represents the building block of higher-level social cognition in primates, although their neurodevelopmental processes are still poorly understood. Atypical development of these social skills has served as early marker of autism spectrum disorder and Williams syndrome. In this study, we trace the developmental trajectories of four neural networks underlying visual and attentional social engagement in the translational rhesus monkey model. Resting-state fMRI (rs-fMRI) data and gaze following skills were collected in infant rhesus macaques from birth through 6 months of age. Developmental trajectories from subjects with both resting-state fMRI and eye-tracking data were used to explore brain-behavior relationships. Our findings indicate robust increases in functional connectivity (FC) between primary visual areas (primary visual cortex [V1] - extrastriate area 3 [V3] and V3 - middle temporal area [MT], MT and anterior superior temporal sulcus area [AST], as well as between anterior temporal area [TE]) and amygdala (AMY) as infants mature. Significant FC decreases were found in more rostral areas of the pathways, such as between temporal area occipital part - TE in the ventral object pathway, V3 - lateral intraparietal (LIP) of the dorsal visual attention pathway and V3 - temporo-parietal area of the ventral attention pathway. No changes in FC were found between cortical areas LIP-FEF and temporo-parietal area - Area 12 of the dorsal and ventral attention pathways or between Anterior Superior Temporal sulcus area (AST)-AMY and AMY-insula. Developmental trajectory of gaze following revealed a period of dynamic changes with gradual increases from 1 to 2 months, followed by slight decreases from 3 to 6 months. Exploratory association findings across the 6-month period showed that infants with higher gaze following had lower FC between primary visual areas V1-V3, but higher FC in the dorsal attention areas V3-LIP, both in the right hemisphere. Together, the first 6 months of life in rhesus macaques represent a critical period for the emergence of gaze following skills associated with maturational changes in FC of socially guided attention pathways.

What Is a Visual Stream?

The dual stream model of the human and non-human primate visual systems remains Leslie Ungerleider's (1946-2020) most indelible contribution to visual neuroscience. In this model, a dorsal "where" stream specialized for visuospatial representation extends through occipitoparietal cortex, whereas a ventral "what" stream specialized for representing object qualities extends through occipitotemporal cortex. Over time, this model underwent a number of revisions and expansions. In one of her last scientific contributions, Leslie proposed a third visual stream specialized for representing dynamic signals related to social perception. This alteration invites the question: What is a visual stream, and how are different visual streams individuated? In this article, we first consider and reject a simple answer to this question based on a common idealizing visualization of the model, which conflicts with the complexities of the visual system that the model was intended to capture. Next, we propose a taxonomic answer that takes inspiration from the philosophy of science and Leslie's body of work, which distinguishes between neural mechanisms, pathways, and streams. In this taxonomy, visual streams are superordinate to pathways and mechanisms and provide individuation conditions for determining whether collections of cortical connections delineate different visual streams. Given this characterization, we suggest that the proposed third visual stream does not yet meet these conditions, although the tripartite model still suggests important revisions to how we think about the organization of the human and non-human primate visual systems.

Understanding Cortical Streams from a Computational Perspective

The two visual cortical streams hypothesis, which suggests object properties (what) are processed separately from spatial properties (where), has a longstanding history, and much evidence has accumulated to support its conjectures. Nevertheless, in the last few decades, conflicting evidence has mounted that demands some explanation and modification. For example, existence of (1) shape activities (fMRI) or shape selectivities (physiology) in dorsal stream, similar to ventral stream; likewise, spatial activations (fMRI) or spatial selectivities (physiology) in ventral stream, similar to dorsal stream; (2) multiple segregated subpathways within a stream. In addition, the idea of segregation of various aspects of multiple objects in a scene raises questions about how these properties of multiple objects are then properly re-associated or bound back together to accurately perceive, remember, or make decisions. We will briefly review the history of the two-stream hypothesis, discuss competing accounts that challenge current thinking, and propose ideas on why the brain has segregated pathways. We will present ideas based on our own data using artificial neural networks (1) to reveal encoding differences for what and where that arise in a two-pathway neural network, (2) to show how these encoding differences can clarify previous conflicting findings, and (3) to elucidate the computational advantages of segregated pathways. Furthermore, we will discuss whether neural networks need to have multiple subpathways for different visual attributes. We will also discuss the binding problem (how to correctly associate the different attributes of each object together when there are multiple objects each with multiple attributes in a scene) and possible solutions to the binding problem. Finally, we will briefly discuss problems and limitations with existing models and potential fruitful future directions.

From Motion to Emotion: Visual Pathways and Potential Interconnections

The two visual pathway description of Ungerleider and Mishkin changed the course of late 20th century systems and cognitive neuroscience. Here, I try to reexamine our laboratory's work through the lens of the Pitcher and Ungerleider new third visual pathway. I also briefly review the literature related to brain responses to static and dynamic visual displays, visual stimulation involving multiple individuals, and compare existing models of social information processing for the face and body. In this context, I examine how the posterior STS might generate unique social information relative to other brain regions that also respond to social stimuli. I discuss some of the existing challenges we face with assessing how information flow progresses between structures in the proposed functional pathways and how some stimulus types and experimental designs may have complicated our data interpretation and model generation. I also note a series of outstanding questions for the field. Finally, I examine the idea of a potential expansion of the third visual pathway, to include aspects of previously proposed "lateral" visual pathways. Doing this would yield a more general entity for processing motion/action (i.e., "[inter]action") that deals with interactions between people, as well as people and objects. In this framework, a brief discussion of potential hemispheric biases for function, and different forms of neuropsychological impairments created by focal lesions in the posterior brain is highlighted to help situate various brain regions into an expanded [inter]action pathway.

Moving and Static Faces, Bodies, Objects, and Scenes Are Differentially Represented across the Three Visual Pathways

Models of human cortex propose the existence of neuroanatomical pathways specialized for different behavioral functions. These pathways include a ventral pathway for object recognition, a dorsal pathway for performing visually guided physical actions, and a recently proposed third pathway for social perception. In the current study, we tested the hypothesis that different categories of moving stimuli are differentially processed across the dorsal and third pathways according to their behavioral implications. Human participants (n = 30) were scanned with fMRI while viewing moving and static stimuli from four categories (faces, bodies, scenes, and objects). A whole-brain group analysis showed that moving bodies and moving objects increased neural responses in the bilateral posterior parietal cortex, parts of the dorsal pathway. By contrast, moving faces and moving bodies increased neural responses, the superior temporal sulcus, part of the third pathway. This pattern of results was also supported by a separate ROI analysis showing that moving stimuli produced more robust neural responses for all visual object categories, particularly in lateral and dorsal brain areas. Our results suggest that dynamic naturalistic stimuli from different categories are routed in specific visual pathways that process dissociable behavioral functions.

Real Face Value: The Processing of Naturalistic Facial Expressions in the Macaque Inferior Temporal Cortex

For primates, expressions of fear are thought to be powerful social signals. In laboratory settings, faces with fearful expressions have reliably evoked valence effects in inferior temporal cortex. However, because macaques use so called "fear grins" in a variety of different contexts, the deeper question is whether the macaque inferior temporal cortex is tuned to the prototypical fear grin, or to conspecifics signaling fear? In this study, we combined neuroimaging with the results of a behavioral task to investigate how macaques encode a wide variety of fearful facial expressions. In Experiment 1, we identified two sets of macaque face stimuli using different approaches; we selected faces based on the emotional context (i.e., calm vs. fearful), and we selected faces based on the engagement of action units (i.e., neutral vs. fear grins). We also included human faces in Experiment 1. Then, using fMRI, we found that the faces selected based on context elicited a larger valence effect in the inferior temporal cortex than faces selected based on visual appearance. Furthermore, human facial expressions only elicited weak valence effects. These observations were further supported by the results of a two-alternative, forced-choice task (Experiment 2), suggesting that fear grins vary in their perceived pleasantness. Collectively, these findings indicate that the macaque inferior temporal cortex is more involved in social intelligence than commonly assumed, encoding emergent properties in naturalistic face stimuli that transcend basic visual features. These results demand a rethinking of theories surrounding the function and operationalization of primate inferior temporal cortex.

On the Role of Sensorimotor Experience in Facial Expression Perception

Humans recognize the facial expressions of others rapidly and effortlessly. Although much is known about how we perceive expressions, the role of facial experience in shaping this remarkable ability remains unclear. Is our perception of expressions linked to how we ourselves make facial expressions? Are we better at recognizing other's facial expressions if we are experts at making the same expressions ourselves? And if we could not make facial expressions at all, would it impact our ability to recognize others' facial expressions? The current article aims to examine these questions by explicating the link between facial experience and facial expression recognition. It includes a comprehensive appraisal of the related literature and examines three main theories that posit a connection between making and recognizing facial expressions. First, recent studies in individuals with Moebius syndrome support the role of facial ability (i.e., the ability to move one's face to make facial expressions) in facial expression recognition. Second, motor simulation theory suggests that humans recognize others' facial expressions by covertly mimicking the observed expression (without overt motor action) and that this facial mimicry helps us identify and feel the associated emotion. Finally, the facial feedback hypothesis provides a framework for enhanced emotional experience via proprioceptive feedback from facial muscles when mimicking a viewed facial expression. Evidence for and against these theories is presented as well as some considerations and outstanding questions for future research studies investigating the role of facial experience in facial expression perception.

Anatomical Connectivity Constrains Dynamic Functional Connectivity among Neural Systems: Implications for Cognition and Behavior

Leslie Ungerleider had a tremendous impact across many different areas of cognitive neuroscience. Her ideas and her approach, as well as her findings, will continue to impact the field for generations to come. One of the most impactful aspects of her approach was her focus on the ways that anatomical connections constrain functional communications among brain regions. Furthermore, she emphasized that changes in these functional communications, whether from lesions to the anatomical connections or temporary modulations of the efficacy of information transmission resulting from selective attention, have consequences for cognition and behavior. By necessity, this short review cannot cover the vast amount of research that contributed to or benefited from Leslie's work. Rather, we focus on one line of research that grew directly from some of Leslie's early work and her mentoring on these important concepts. This research and the many other lines of research that arose from these same origins has helped develop our understanding of the visual system, and cognitive systems more generally, as collections of highly organized, specialized, dynamic, and interacting subsystems.

Insights from the Evolving Model of Two Cortical Visual Pathways

The two cortical visual pathways framework has had a profound influence on theories and empirical studies of the visual system for over 40 years. By grounding physiological responses and behavior in neuroanatomy, the framework provided a critical guide for understanding vision. Although the framework has evolved over time, as our understanding of the physiology and neuroanatomy expanded, cortical visual processing is still often conceptualized as two separate pathways emerging from the primary visual cortex that support distinct behaviors ("what" vs. "where/how"). Here, we take a historical perspective and review the continuing evolution of the framework, discussing key and often overlooked insights. Rather than a functional and neuroanatomical bifurcation into two independent serial, hierarchical pathways, the current evidence points to two highly recurrent heterarchies with heterogeneous connections to cortical regions and subcortical structures that flexibly support a wide variety of behaviors. Although many of the simplifying assumptions of the framework are belied by the evidence gathered since its initial proposal, the core insight of grounding function and behavior in neuroanatomy remains fundamental. Given this perspective, we highlight critical open questions and the need for a better understanding of neuroanatomy, particularly in the human.

The Spiraling Cognitive-Emotional Brain: Combinatorial, Reciprocal, and Reentrant Macro-organization

This article proposes a framework for understanding the macro-scale organization of anatomical pathways in the mammalian brain. The architecture supports flexible behavioral decisions across a spectrum of spatiotemporal scales. The proposal emphasizes the combinatorial, reciprocal, and reentrant connectivity-called CRR neuroarchitecture-between cortical, BG, thalamic, amygdala, hypothalamic, and brainstem circuits. Thalamic nuclei, especially midline/intralaminar nuclei, are proposed to act as hubs routing the flow of signals between noncortical areas and pFC. The hypothalamus also participates in multiregion circuits via its connections with cortex and thalamus. At slower timescales, long-range behaviors integrate signals across levels of the neuroaxis. At fast timescales, parallel engagement of pathways allows urgent behaviors while retaining flexibility. Overall, the proposed architecture enables context-dependent, adaptive behaviors spanning proximate to distant spatiotemporal scales. The framework promotes an integrative perspective and a distributed, heterarchical view of brain function.

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test a number of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral judgments of similarity better reflect neural representational geometry than semantic or visual models throughout much of cortex. The sociality and transitivity models in particular captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, our findings expand the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.