Functional subdomains within human FFA

Quantitative modelling demonstrates format-invariant representations of mathematical problems in the brain

Mathematical problems can be described in either symbolic form or natural language. Previous studies have reported that activation overlaps exist for these two types of mathematical problems, but it is unclear whether they are based on similar brain representations. Furthermore, quantitative modelling of mathematical problem solving has yet to be attempted. In the present study, subjects underwent 3 h of functional magnetic resonance experiments involving math word and math expression problems, and a read word condition without any calculations was used as a control. To evaluate the brain representations of mathematical problems quantitatively, we constructed voxel-wise encoding models. Both intra- and cross-format encoding modelling significantly predicted brain activity predominantly in the left intraparietal sulcus (IPS), even after subtraction of the control condition. Representational similarity analysis and principal component analysis revealed that mathematical problems with different formats had similar cortical organization in the IPS. These findings support the idea that mathematical problems are represented in the brain in a format-invariant manner.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Functionally and structurally distinct fusiform face area(s) in over 1000 participants

The fusiform face area (FFA) is a widely studied region causally involved in face perception. Even though cognitive neuroscientists have been studying the FFA for over two decades, answers to foundational questions regarding the function, architecture, and connectivity of the FFA from a large (N>1000) group of participants are still lacking. To fill this gap in knowledge, we quantified these multimodal features of fusiform face-selective regions in 1053 participants in the Human Connectome Project. After manually defining over 4,000 fusiform face-selective regions, we report five main findings. First, 68.76% of hemispheres have two cortically separate regions (pFus-faces/FFA-1 and mFus-faces/FFA-2). Second, in 26.69% of hemispheres, pFus-faces/FFA-1 and mFus-faces/FFA-2 are spatially contiguous, yet are distinct based on functional, architectural, and connectivity metrics. Third, pFus-faces/FFA-1 is more face-selective than mFus-faces/FFA-2, and the two regions have distinct functional connectivity fingerprints. Fourth, pFus-faces/FFA-1 is cortically thinner and more heavily myelinated than mFus-faces/FFA-2. Fifth, face-selective patterns and functional connectivity fingerprints of each region are more similar in monozygotic than dizygotic twins and more so than architectural gradients. As we share our areal definitions with the field, future studies can explore how structural and functional features of these regions will inform theories regarding how visual categories are represented in the brain.

Abnormal Functional Connectivity Between Cerebral Hemispheres in Patients With High Myopia: A Resting FMRI Study Based on Voxel-Mirrored Homotopic Connectivity

Purpose

To study the changes in functional connections between the left and right hemispheres of patients with high myopia (HM) and healthy controls (HCs) by resting functional magnetic resonance imaging (fMRI) based on voxel-mirrored homotopic connectivity (VMHC). To study the changes in resting-state functional connectivity (rsFC) between the left and right hemispheres of patients with HM and healthy controls (HCS) at rest by using resting functional magnetic resonance imaging (fMRI) based on voxel-mirror homotopy connectivity (VMHC).

Patients and Methods

A total of 89 patients with HM (41 men and 48 women) and 59 HCs (24 men and 35 women) were collected and matched according to gender, age, and education level. The VMHC method was used to evaluate the changes in rsFC between cerebral hemispheres, and a correlation analysis was carried out to understand the differences in brain functional activities between the patients with HM and the HCs.

Results

Compared with the HCs, the VMHC values of the putamen and fusiform in the HM group were significantly lower (voxel-level p < 0.01, Gaussian random field correction cluster level p < 0.05).

Conclusion

This study preliminarily confirmed the destruction of interhemispheric functional connection in some brain regions of the patients with HM and provided effective information for clarifying the neural mechanism of patients with HM.

Brain Morphological Modifications in Congenital and Acquired Auditory Deprivation: A Systematic Review and Coordinate-Based Meta-Analysis

Neuroplasticity following deafness has been widely demonstrated in both humans and animals, but the anatomical substrate of these changes is not yet clear in human brain. However, it is of high importance since hearing loss is a growing problem due to aging population. Moreover, knowing these brain changes could help to understand some disappointing results with cochlear implant, and therefore could improve hearing rehabilitation. A systematic review and a coordinate-based meta-analysis were realized about the morphological brain changes highlighted by MRI in severe to profound hearing loss, congenital and acquired before or after language onset. 25 papers were included in our review, concerning more than 400 deaf subjects, most of them presenting prelingual deafness. The most consistent finding is a volumetric decrease in gray matter around bilateral auditory cortex. This change was confirmed by the coordinate-based meta-analysis which shows three converging clusters in this region. The visual areas of deaf children is also significantly impacted, with a decrease of the volume of both gray and white matters. Finally, deafness is responsible of a gray matter increase within the cerebellum, especially at the right side. These results are largely discussed and compared with those from deaf animal models and blind humans, which demonstrate for example a much more consistent gray matter decrease along their respective primary sensory pathway. In human deafness, a lot of other factors than deafness could interact on the brain plasticity. One of the most important is the use of sign language and its age of acquisition, which induce among others changes within the hand motor region and the visual cortex. But other confounding factors exist which have been too little considered in the current literature, such as the etiology of the hearing impairment, the speech-reading ability, the hearing aid use, the frequent associated vestibular dysfunction or neurocognitive impairment. Another important weakness highlighted by this review concern the lack of papers about postlingual deafness, whereas it represents most of the deaf population. Further studies are needed to better understand these issues, and finally try to improve deafness rehabilitation.

Cortical networks of dynamic scene category representation in the human brain

Humans have an impressive ability to rapidly process global information in natural scenes to infer their category. Yet, it remains unclear whether and how scene categories observed dynamically in the natural world are represented in cerebral cortex beyond few canonical scene-selective areas. To address this question, here we examined the representation of dynamic visual scenes by recording whole-brain blood oxygenation level-dependent (BOLD) responses while subjects viewed natural movies. We fit voxelwise encoding models to estimate tuning for scene categories that reflect statistical ensembles of objects and actions in the natural world. We find that this scene-category model explains a significant portion of the response variance broadly across cerebral cortex. Cluster analysis of scene-category tuning profiles across cortex reveals nine spatially-segregated networks of brain regions consistently across subjects. These networks show heterogeneous tuning for a diverse set of dynamic scene categories related to navigation, human activity, social interaction, civilization, natural environment, non-human animals, motion-energy, and texture, suggesting that the organization of scene category representation is quite complex.

High-Level Representations in Human Occipito-Temporal Cortex Are Indexed by Distal Connectivity

Human object recognition is dependent on occipito-temporal cortex (OTC), but a complete understanding of the complex functional architecture of this area must account for how it is connected to the wider brain. Converging functional magnetic resonance imaging evidence shows that univariate responses to different categories of information (e.g., faces, bodies, and nonhuman objects) are strongly related to, and potentially shaped by, functional and structural connectivity to the wider brain. However, to date, there have been no systematic attempts to determine how distal connectivity and complex local high-level responses in occipito-temporal cortex (i.e., multivoxel response patterns) are related. Here, we show that distal functional connectivity is related to, and can reliably index, high-level representations for several visual categories (i.e., tools, faces, and places) within occipito-temporal cortex; that is, voxel sets that are strongly connected to distal brain areas show higher pattern discriminability than less well-connected sets do. We further show that in several cases, pattern discriminability is higher in sets of well-connected voxels than sets defined by local activation (e.g., strong amplitude responses to faces in fusiform face area). Together, these findings demonstrate the important relationship between the complex functional organization of occipito-temporal cortex and wider brain connectivity.SIGNIFICANCE STATEMENT Human object recognition relies strongly on OTC, yet responses in this broad area are often considered in relative isolation to the rest of the brain. We employ a novel connectivity-guided voxel selection approach with functional magnetic resonance imaging data to show higher sensitivity to information (i.e., higher multivoxel pattern discriminability) in voxel sets that share strong connectivity to distal brain areas, relative to (1) voxel sets that are less strongly connected, and in several cases, (2) voxel sets that are defined by strong local response amplitude. These findings underscore the importance of distal contributions to local processing in OTC.

Convergence of Modality Invariance and Attention Selectivity in the Cortical Semantic Circuit

The human linguistic system is characterized by modality invariance and attention selectivity. Previous studies have examined these properties independently and reported perisylvian region involvement for both; however, their relationship and the linguistic information they harbor remain unknown. Participants were assessed by functional magnetic resonance imaging, while spoken narratives (auditory) and written texts (visual) were presented, either separately or simultaneously. Participants were asked to attend to one stimulus when both were presented. We extracted phonemic and semantic features from these auditory and visual modalities, to train multiple, voxel-wise encoding models. Cross-modal examinations of the trained models revealed that perisylvian regions were associated with modality-invariant semantic representations. Attentional selectivity was quantified by examining the modeling performance for attended and unattended conditions. We have determined that perisylvian regions exhibited attention selectivity. Both modality invariance and attention selectivity are both prominent in models that use semantic but not phonemic features. Modality invariance was significantly correlated with attention selectivity in some brain regions; however, we also identified cortical regions associated with only modality invariance or only attention selectivity. Thus, paying selective attention to a specific sensory input modality may regulate the semantic information that is partly processed in brain networks that are shared across modalities.

Voxel-Based State Space Modeling Recovers Task-Related Cognitive States in Naturalistic fMRI Experiments

Complex natural tasks likely recruit many different functional brain networks, but it is difficult to predict how such tasks will be represented across cortical areas and networks. Previous electrophysiology studies suggest that task variables are represented in a low-dimensional subspace within the activity space of neural populations. Here we develop a voxel-based state space modeling method for recovering task-related state spaces from human fMRI data. We apply this method to data acquired in a controlled visual attention task and a video game task. We find that each task induces distinct brain states that can be embedded in a low-dimensional state space that reflects task parameters, and that attention increases state separation in the task-related subspace. Our results demonstrate that the state space framework offers a powerful approach for modeling human brain activity elicited by complex natural tasks.

Informed feature regularization in voxelwise modeling for naturalistic fMRI experiments

Voxelwise modeling is a powerful framework to predict single-voxel functional selectivity for the stimulus features that exist in complex natural stimuli. Yet, because VM disregards potential correlations across stimulus features or neighboring voxels, it may yield suboptimal sensitivity in measuring functional selectivity in the presence of high levels of measurement noise. Here, we introduce a novel voxelwise modeling approach that simultaneously utilizes stimulus correlations in model features and response correlations among voxel neighborhoods. The proposed method performs feature and spatial regularization while still generating single-voxel response predictions. We demonstrated the performance of our approach on a functional magnetic resonance imaging dataset from a natural vision experiment. Compared to VM, the proposed method yields clear improvements in prediction performance, together with increased feature coherence and spatial coherence of voxelwise models. Overall, the proposed method can offer improved sensitivity in modeling of single voxels in naturalistic functional magnetic resonance imaging experiments.

Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies

Information that is shared across brains is encoded in idiosyncratic fine-scale functional topographies. Hyperalignment captures shared information by projecting pattern vectors for neural responses and connectivities into a common, high-dimensional information space, rather than by aligning topographies in a canonical anatomical space. Individual transformation matrices project information from individual anatomical spaces into the common model information space, preserving the geometry of pairwise dissimilarities between pattern vectors, and model cortical topography as mixtures of overlapping, individual-specific topographic basis functions, rather than as contiguous functional areas. The fundamental property of brain function that is preserved across brains is information content, rather than the functional properties of local features that support that content. In this Perspective, we present the conceptual framework that motivates hyperalignment, its computational underpinnings for joint modeling of a common information space and idiosyncratic cortical topographies, and discuss implications for understanding the structure of cortical functional architecture.

Predicting individual face-selective topography using naturalistic stimuli

Subject-specific, functionally defined areas are conventionally estimated with functional localizers and a simple contrast analysis between responses to different stimulus categories. Compared with functional localizers, naturalistic stimuli provide several advantages such as stronger and widespread brain activation, greater engagement, and increased subject compliance. In this study we demonstrate that a subject's idiosyncratic functional topography can be estimated with high fidelity from that subject's fMRI data obtained while watching a naturalistic movie using hyperalignment to project other subjects' localizer data into that subject's idiosyncratic cortical anatomy. These findings lay the foundation for developing an efficient tool for mapping functional topographies for a wide range of perceptual and cognitive functions in new subjects based only on fMRI data collected while watching an engaging, naturalistic stimulus and other subjects' localizer data from a normative sample.

Similarity judgments and cortical visual responses reflect different properties of object and scene categories in naturalistic images

Numerous factors have been reported to underlie the representation of complex images in high-level human visual cortex, including categories (e.g. faces, objects, scenes), animacy, and real-world size, but the extent to which this organization reflects behavioral judgments of real-world stimuli is unclear. Here, we compared representations derived from explicit behavioral similarity judgments and ultra-high field (7T) fMRI of human visual cortex for multiple exemplars of a diverse set of naturalistic images from 48 object and scene categories. While there was a significant correlation between similarity judgments and fMRI responses, there were striking differences between the two representational spaces. Behavioral judgements primarily revealed a coarse division between man-made (including humans) and natural (including animals) images, with clear groupings of conceptually-related categories (e.g. transportation, animals), while these conceptual groupings were largely absent in the fMRI representations. Instead, fMRI responses primarily seemed to reflect a separation of both human and non-human faces/bodies from all other categories. Further, comparison of the behavioral and fMRI representational spaces with those derived from the layers of a deep neural network (DNN) showed a strong correspondence with behavior in the top-most layer and with fMRI in the mid-level layers. These results suggest a complex relationship between localized responses in high-level visual cortex and behavioral similarity judgments - each domain reflects different properties of the images, and responses in high-level visual cortex may correspond to intermediate stages of processing between basic visual features and the conceptual categories that dominate the behavioral response.

Characterizing functional pathways of the human olfactory system

The central processing pathways of the human olfactory system are not fully understood. The olfactory bulb projects directly to a number of cortical brain structures, but the distinct networks formed by projections from each of these structures to the rest of the brain have not been well-defined. Here, we used functional magnetic resonance imaging and k-means clustering to parcellate human primary olfactory cortex into clusters based on whole-brain functional connectivity patterns. Resulting clusters accurately corresponded to anterior olfactory nucleus, olfactory tubercle, and frontal and temporal piriform cortices, suggesting dissociable whole-brain networks formed by the subregions of primary olfactory cortex. This result was replicated in an independent data set. We then characterized the unique functional connectivity profiles of each subregion, producing a map of the large-scale processing pathways of the human olfactory system. These results provide insight into the functional and anatomical organization of the human olfactory system.

Differential sustained and transient temporal processing across visual streams

How do high-level visual regions process the temporal aspects of our visual experience? While the temporal sensitivity of early visual cortex has been studied with fMRI in humans, temporal processing in high-level visual cortex is largely unknown. By modeling neural responses with millisecond precision in separate sustained and transient channels, and introducing a flexible encoding framework that captures differences in neural temporal integration time windows and response nonlinearities, we predict fMRI responses across visual cortex for stimuli ranging from 33 ms to 20 s. Using this innovative approach, we discovered that lateral category-selective regions respond to visual transients associated with stimulus onsets and offsets but not sustained visual information. Thus, lateral category-selective regions compute moment-to-moment visual transitions, but not stable features of the visual input. In contrast, ventral category-selective regions process both sustained and transient components of the visual input. Our model revealed that sustained channel responses to prolonged stimuli exhibit adaptation, whereas transient channel responses to stimulus offsets are surprisingly larger than for stimulus onsets. This large offset transient response may reflect a memory trace of the stimulus when it is no longer visible, whereas the onset transient response may reflect rapid processing of new items. Together, these findings reveal previously unconsidered, fundamental temporal mechanisms that distinguish visual streams in the human brain. Importantly, our results underscore the promise of modeling brain responses with millisecond precision to understand the underlying neural computations.

How does the brain encode the timing of our visual experience? Using functional magnetic resonance imaging (fMRI) and a generative temporal model with millisecond resolution, we discovered that visual regions in the lateral and ventral processing streams fundamentally differ in their temporal processing of the visual input. Regions in lateral temporal cortex process visual transients associated with the beginning and ending of the stimulus, but not its stable aspects. That is, lateral regions appear to compute moment-to-moment changes in the visual input. In contrast, regions in ventral temporal cortex process both stable and transient components of the visual input, even as the response to the former exhibits adaptation. Surprisingly, the model predicts that in ventral regions responses to stimulus endings are larger than beginnings. We suggest that ending responses may reflect a memory trace of the stimulus, when it is no longer visible, and the beginning responses may reflect processing of new inputs. Together, these findings (i) reveal a fundamental temporal mechanism that distinguishes visual streams and (ii) highlight both the importance and utility of modeling brain responses with millisecond precision to understand the temporal dynamics of neural computations in the human brain.

The Neural Dynamics of Novel Scene Imagery

Retrieval of long-term episodic memories is characterized by synchronized neural activity between hippocampus and ventromedial prefrontal cortex (vmPFC), with additional evidence that vmPFC activity leads that of the hippocampus. It has been proposed that the mental generation of scene imagery is a crucial component of episodic memory processing. If this is the case, then a comparable interaction between the two brain regions should exist during the construction of novel scene imagery. To address this question, we leveraged the high temporal resolution of MEG to investigate the construction of novel mental imagery. We tasked male and female humans with imagining scenes and single isolated objects in response to one-word cues. We performed source-level power, coherence, and causality analyses to characterize the underlying interregional interactions. Both scene and object imagination resulted in theta power changes in the anterior hippocampus. However, higher theta coherence was observed between the hippocampus and vmPFC in the scene compared with the object condition. This interregional theta coherence also predicted whether imagined scenes were subsequently remembered. Dynamic causal modeling of this interaction revealed that vmPFC drove activity in hippocampus during novel scene construction. Additionally, theta power changes in the vmPFC preceded those observed in the hippocampus. These results constitute the first evidence in humans that episodic memory retrieval and scene imagination rely on similar vmPFC–hippocampus neural dynamics. Furthermore, they provide support for theories emphasizing similarities between both cognitive processes and perspectives that propose the vmPFC guides the construction of context-relevant representations in the hippocampus.

SIGNIFICANCE STATEMENT Episodic memory retrieval is characterized by a dialog between hippocampus and ventromedial prefrontal cortex (vmPFC). It has been proposed that the mental generation of scene imagery is a crucial component of episodic memory processing. An ensuing prediction would be of a comparable interaction between the two brain regions during the construction of novel scene imagery. Here, we leveraged the high temporal resolution of MEG and combined it with a scene imagination task. We found that a hippocampal–vmPFC dialog existed and that it took the form of vmPFC driving the hippocampus. We conclude that episodic memory and scene imagination share fundamental neural dynamics and the process of constructing vivid, spatially coherent, contextually appropriate scene imagery is strongly modulated by vmPFC.

Spatially informed voxelwise modeling for naturalistic fMRI experiments

Voxelwise modeling (VM) is a powerful framework to predict single voxel responses evoked by a rich set of stimulus features present in complex natural stimuli. However, because VM disregards correlations across neighboring voxels, its sensitivity in detecting functional selectivity can be diminished in the presence of high levels of measurement noise. Here, we introduce spatially-informed voxelwise modeling (SPIN-VM) to take advantage of response correlations in spatial neighborhoods of voxels. To optimally utilize shared information, SPIN-VM performs regularization across spatial neighborhoods in addition to model features, while still generating single-voxel response predictions. We demonstrated the performance of SPIN-VM on a rich dataset from a natural vision experiment. Compared to VM, SPIN-VM yields higher prediction accuracies and better capture locally congruent information representations across cortex. These results suggest that SPIN-VM offers improved performance in predicting single-voxel responses and recovering coherent information representations.

The functional neuroanatomy of face perception: from brain measurements to deep neural networks

A central goal in neuroscience is to understand how processing within the ventral visual stream enables rapid and robust perception and recognition. Recent neuroscientific discoveries have significantly advanced understanding of the function, structure and computations along the ventral visual stream that serve as the infrastructure supporting this behaviour. In parallel, significant advances in computational models, such as hierarchical deep neural networks (DNNs), have brought machine performance to a level that is commensurate with human performance. Here, we propose a new framework using the ventral face network as a model system to illustrate how increasing the neural accuracy of present DNNs may allow researchers to test the computational benefits of the functional architecture of the human brain. Thus, the review (i) considers specific neural implementational features of the ventral face network, (ii) describes similarities and differences between the functional architecture of the brain and DNNs, and (iii) provides a hypothesis for the computational value of implementational features within the brain that may improve DNN performance. Importantly, this new framework promotes the incorporation of neuroscientific findings into DNNs in order to test the computational benefits of fundamental organizational features of the visual system.

Structural and Functional Brain Abnormalities Associated With Exposure to Different Childhood Trauma Subtypes: A Systematic Review of Neuroimaging Findings

Background: Childhood trauma subtypes sexual abuse, physical abuse, emotional maltreatment, and neglect may have differential effects on the brain that persist into adulthood. A systematic review of neuroimaging findings supporting these differential effects is as yet lacking. Objectives: The present systematic review aims to summarize the findings of controlled neuroimaging trials regarding long-term differential effects of trauma subtypes on the human brain. Methods: A systematic literature search was performed using the PubMed and PsycINFO databases from January 2017 up to and including January 2018. Additional papers were identified by a manual search in the reference lists of selected papers and of relevant review articles retrieved by the initial database search. Studies were then assessed for eligibility by the first author. Only original human studies directly comparing neuroimaging findings of exposed and unexposed individuals to well-defined emotional, physical or sexual childhood maltreatment while controlling for the effects of other subtypes were included. A visual summary of extracted data was made for neuroimaging modalities for which comparison between trauma subtypes was feasible, taking the studies' numbers and sample sizes into account. Results: The systematic literature search yielded 25 publications. Sexual abuse was associated with structural deficits in the reward circuit and genitosensory cortex and amygdalar hyperreactivity during sad autobiographic memory recall. Emotional maltreatment correlated with abnormalities in fronto-limbic socioemotional networks. In neglected individuals, white matter integrity and connectivity were disturbed in several brain networks involved in a variety of functions. Other abnormalities, such as reduced frontal cortical volume, were common to all maltreatment types. Conclusions: There is some evidence for long-term differential effects of trauma subtypes on the human brain. The observed alterations may result from both protective adaptation of and damage to the brain following exposure to threatening life events. Though promising, the current evidence is incomplete, with few brain regions and neuroimaging modalities having been investigated in all subtypes. The comparability of the available evidence is further limited by the heterogeneity of study populations regarding gender, age and comorbid psychopathology. Future neuroimaging studies should take this potentially differential role of childhood trauma subtypes into account.

Encoding model of temporal processing in human visual cortex

How is temporal information processed in human visual cortex? Visual input is relayed to V1 through segregated transient and sustained channels in the retina and lateral geniculate nucleus (LGN). However, there is intense debate as to how sustained and transient temporal channels contribute to visual processing beyond V1. The prevailing view associates transient processing predominately with motion-sensitive regions and sustained processing with ventral stream regions, while the opposing view suggests that both temporal channels contribute to neural processing beyond V1. Using fMRI, we measured cortical responses to time-varying stimuli and then implemented a two temporal channel-encoding model to evaluate the contributions of each channel. Different from the general linear model of fMRI that predicts responses directly from the stimulus, the encoding approach first models neural responses to the stimulus from which fMRI responses are derived. This encoding approach not only predicts cortical responses to time-varying stimuli from milliseconds to seconds but also, reveals differential contributions of temporal channels across visual cortex. Consistent with the prevailing view, motion-sensitive regions and adjacent lateral occipitotemporal regions are dominated by transient responses. However, ventral occipitotemporal regions are driven by both sustained and transient channels, with transient responses exceeding the sustained. These findings propose a rethinking of temporal processing in the ventral stream and suggest that transient processing may contribute to rapid extraction of the content of the visual input. Importantly, our encoding approach has vast implications, because it can be applied with fMRI to decipher neural computations in millisecond resolution in any part of the brain.

Facephenes and rainbows: Causal evidence for functional and anatomical specificity of face and color processing in the human brain

Neuroscientists have long debated whether some regions of the human brain are exclusively engaged in a single specific mental process. Consistent with this view, fMRI has revealed cortical regions that respond selectively to certain stimulus classes such as faces. However, results from multivoxel pattern analyses (MVPA) challenge this view by demonstrating that category-selective regions often contain information about "nonpreferred" stimulus dimensions. But is this nonpreferred information causally relevant to behavior? Here we report a rare opportunity to test this question in a neurosurgical patient implanted for clinical reasons with strips of electrodes along his fusiform gyri. Broadband gamma electrocorticographic responses in multiple adjacent electrodes showed strong selectivity for faces in a region corresponding to the fusiform face area (FFA), and preferential responses to color in a nearby site, replicating earlier reports. To test the causal role of these regions in the perception of nonpreferred dimensions, we then electrically stimulated individual sites while the patient viewed various objects. When stimulated in the FFA, the patient reported seeing an illusory face (or "facephene"), independent of the object viewed. Similarly, stimulation of color-preferring sites produced illusory "rainbows." Crucially, the patient reported no change in the object viewed, apart from the facephenes and rainbows apparently superimposed on them. The functional and anatomical specificity of these effects indicate that some cortical regions are exclusively causally engaged in a single specific mental process, and prompt caution about the widespread assumption that any information scientists can decode from the brain is causally relevant to behavior.

Improving fMRI in signal drop-out regions at 7 T by using tailored radio-frequency pulses: application to the ventral occipito-temporal cortex

OBJECTIVE

Signal drop-off occurs in echo-planar imaging in inferior brain areas due to field gradients from susceptibility differences between air and tissue. Tailored-RF pulses based on a hyperbolic secant (HS) have been shown to partially recover signal at 3 T, but have not been tested at higher fields.

MATERIALS AND METHODS

The aim of this study was to compare the performance of an optimized tailored-RF gradient-echo echo-planar imaging (TRF GRE-EPI) sequence with standard GRE-EPI at 7 T, in a passive viewing of faces or objects fMRI paradigm in healthy subjects.

RESULTS

Increased temporal-SNR (tSNR) was observed in the middle and inferior temporal lobes and orbitofrontal cortex of all subjects scanned, but elsewhere tSNR decreased relative to the standard acquisition. In the TRF GRE-EPI, increased functional signal was observed in the fusiform, lateral occipital cortex, and occipital pole, regions known to be part of the visual pathway involved in face-object perception.

CONCLUSION

This work highlights the potential of TRF approaches at 7 T. Paired with a reversed-gradient distortion correction to compensate for in-plane susceptibility gradients, it provides an improved acquisition strategy for future neurocognitive studies at ultra-high field imaging in areas suffering from static magnetic field inhomogeneities.

The Functional Neuroanatomy of Human Face Perception

Face perception is critical for normal social functioning and is mediated by a network of regions in the ventral visual stream. In this review, we describe recent neuroimaging findings regarding the macro- and microscopic anatomical features of the ventral face network, the characteristics of white matter connections, and basic computations performed by population receptive fields within face-selective regions composing this network. We emphasize the importance of the neural tissue properties and white matter connections of each region, as these anatomical properties may be tightly linked to the functional characteristics of the ventral face network. We end by considering how empirical investigations of the neural architecture of the face network may inform the development of computational models and shed light on how computations in the face network enable efficient face perception.

Functional Subdomains within Scene-Selective Cortex: Parahippocampal Place Area, Retrosplenial Complex, and Occipital Place Area

Functional MRI studies suggest that at least three brain regions in human visual cortex-the parahippocampal place area (PPA), retrosplenial complex (RSC), and occipital place area (OPA; often called the transverse occipital sulcus)-represent large-scale information in natural scenes. Tuning of voxels within each region is often assumed to be functionally homogeneous. To test this assumption, we recorded blood oxygenation level-dependent responses during passive viewing of complex natural movies. We then used a voxelwise modeling framework to estimate voxelwise category tuning profiles within each scene-selective region. In all three regions, cluster analysis of the voxelwise tuning profiles reveals two functional subdomains that differ primarily in their responses to animals, man-made objects, social communication, and movement. Thus, the conventional functional definitions of the PPA, RSC, and OPA appear to be too coarse. One attractive hypothesis is that this consistent functional subdivision of scene-selective regions is a reflection of an underlying anatomical organization into two separate processing streams, one selectively biased toward static stimuli and one biased toward dynamic stimuli.

SIGNIFICANCE STATEMENT

Visual scene perception is a critical ability to survive in the real world. It is therefore reasonable to assume that the human brain contains neural circuitry selective for visual scenes. Here we show that responses in three scene-selective areas-identified in previous studies-carry information about many object and action categories encountered in daily life. We identify two subregions in each area: one that is selective for categories of man-made objects, and another that is selective for vehicles and locomotion-related action categories that appear in dynamic scenes. This consistent functional subdivision may reflect an anatomical organization into two processing streams, one biased toward static stimuli and one biased toward dynamic stimuli.

Multivoxel Object Representations in Adult Human Visual Cortex Are Flexible: An Associative Learning Study

Learning associations between co-occurring events enables us to extract structure from our environment. Medial-temporal lobe structures are critical for associative learning. However, the role of the ventral visual pathway (VVP) in associative learning is not clear. Do multivoxel object representations in the VVP reflect newly formed associations? We show that VVP multivoxel representations become more similar to each other after human participants learn arbitrary new associations between pairs of unrelated objects (faces, houses, cars, chairs). Participants were scanned before and after 15 days of associative learning. To evaluate how object representations changed, a classifier was trained on discriminating two nonassociated categories (e.g., faces/houses) and tested on discriminating their paired associates (e.g., cars/chairs). Because the associations were arbitrary and counterbalanced across participants, there was initially no particular reason for this cross-classification decision to tend toward either alternative. Nonetheless, after learning, cross-classification performance increased in the VVP (but not hippocampus), on average by 3.3%, with some voxels showing increases of up to 10%. For example, a chair multivoxel representation that initially resembled neither face nor house representations was, after learning, classified as more similar to that of faces for participants who associated chairs with faces and to that of houses for participants who associated chairs with houses. Additionally, learning produced long-lasting perceptual consequences. In a behavioral priming experiment performed several months later, the change in cross-classification performance was correlated with the degree of priming. Thus, VVP multivoxel representations are not static but become more similar to each other after associative learning.

Feature-based face representations and image reconstruction from behavioral and neural data

The reconstruction of images from neural data can provide a unique window into the content of human perceptual representations. Although recent efforts have established the viability of this enterprise using functional magnetic resonance imaging (MRI) patterns, these efforts have relied on a variety of prespecified image features. Here, we take on the twofold task of deriving features directly from empirical data and of using these features for facial image reconstruction. First, we use a method akin to reverse correlation to derive visual features from functional MRI patterns elicited by a large set of homogeneous face exemplars. Then, we combine these features to reconstruct novel face images from the corresponding neural patterns. This approach allows us to estimate collections of features associated with different cortical areas as well as to successfully match image reconstructions to corresponding face exemplars. Furthermore, we establish the robustness and the utility of this approach by reconstructing images from patterns of behavioral data. From a theoretical perspective, the current results provide key insights into the nature of high-level visual representations, and from a practical perspective, these findings make possible a broad range of image-reconstruction applications via a straightforward methodological approach.

The anatomical and functional specialization of the fusiform gyrus

The fusiform gyrus (FG) is commonly included in anatomical atlases and is considered a key structure for functionally-specialized computations of high-level vision such as face perception, object recognition, and reading. However, it is not widely known that the FG has a contentious history. In this review, we first provide a historical analysis of the discovery of the FG and why certain features, such as the mid-fusiform sulcus, were discovered and then forgotten. We then discuss how observer-independent methods for identifying cytoarchitectonical boundaries of the cortex revolutionized our understanding of cytoarchitecture and the correspondence between those boundaries and cortical folding patterns of the FG. We further explain that the co-occurrence between cortical folding patterns and cytoarchitectonical boundaries are more common than classically thought and also, are functionally meaningful especially on the FG and probably in high-level visual cortex more generally. We conclude by proposing a series of alternatives for how the anatomical organization of the FG can accommodate seemingly different theoretical aspects of functional processing, such as domain specificity and perceptual expertise.

Face processing in autism spectrum disorders: From brain regions to brain networks

Autism spectrum disorder (ASD) is characterized by reduced attention to social stimuli including the human face. This hypo-responsiveness to stimuli that are engaging to typically developing individuals may result from dysfunctioning motivation, reward, and attention systems in the brain. Here we review an emerging neuroimaging literature that emphasizes a shift from focusing on hypo-activation of isolated brain regions such as the fusiform gyrus, amygdala, and superior temporal sulcus in ASD to a more holistic approach to understanding face perception as a process supported by distributed cortical and subcortical brain networks. We summarize evidence for atypical activation patterns within brain networks that may contribute to social deficits characteristic of the disorder. We conclude by pointing to gaps in the literature and future directions that will continue to shed light on aspects of face processing in autism that are still under-examined. In particular, we highlight the need for more developmental studies and studies examining ecologically valid and naturalistic social stimuli.

Successful decoding of famous faces in the fusiform face area

What are the neural mechanisms of face recognition? It is believed that the network of face-selective areas, which spans the occipital, temporal, and frontal cortices, is important in face recognition. A number of previous studies indeed reported that face identity could be discriminated based on patterns of multivoxel activity in the fusiform face area and the anterior temporal lobe. However, given the difficulty in localizing the face-selective area in the anterior temporal lobe, its role in face recognition is still unknown. Furthermore, previous studies limited their analysis to occipito-temporal regions without testing identity decoding in more anterior face-selective regions, such as the amygdala and prefrontal cortex. In the current high-resolution functional Magnetic Resonance Imaging study, we systematically examined the decoding of the identity of famous faces in the temporo-frontal network of face-selective and adjacent non-face-selective regions. A special focus has been put on the face-area in the anterior temporal lobe, which was reliably localized using an optimized scanning protocol. We found that face-identity could be discriminated above chance level only in the fusiform face area. Our results corroborate the role of the fusiform face area in face recognition. Future studies are needed to further explore the role of the more recently discovered anterior face-selective areas in face recognition.

A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes

Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain-machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.

Discriminable spatial patterns of activation for faces and bodies in the fusiform gyrus

Functional neuroimaging studies consistently report that the visual perception of faces and bodies strongly activates regions within ventral occipitotemporal cortex (VOTC) and, in particular, within the mid-lateral fusiform gyrus. One unresolved issue is the degree to which faces and bodies activate discrete or overlapping cortical regions within this region. Here, we examined VOTC activity to faces and bodies at high spatial resolution, using univariate and multivariate analysis approaches sensitive to differences in both the strength and spatial pattern of activation. Faces and bodies evoked substantially overlapping activations in the fusiform gyrus when each was compared to the control category of houses. No discrete regions of activation for faces and bodies in the fusiform gyrus survived a direct statistical comparison using standard univariate statistics. However, multi-voxel pattern analysis differentiated faces and bodies in regions where univariate analysis found no significant difference in the strength of activation. Using a whole-brain multivariate searchlight approach, we also found that extensive regions in VOTC beyond those defined as fusiform face and body areas using standard criteria where the spatial pattern of activation discriminated faces and bodies. These findings provide insights into the spatial distribution of face- and body-specific activations in VOTC and the identification of functionally specialized regions.

The neural mechanisms for the recognition of face identity in humans

Every day we encounter dozens of people, and in order to interact with them appropriately we need to recognize their identity. The face is a crucial source of information to recognize a person’s identity. However, recognizing the identity of a face is challenging because it requires distinguishing between very similar images (e.g., the front views of two different faces) while categorizing very different images (e.g., a front view and a profile) as the same person. Neuroimaging has the whole-brain coverage needed to investigate where representations of face identity are encoded, but it is limited in terms of spatial and temporal resolution. In this article, we review recent neuroimaging research that attempted to investigate the representation of face identity, the challenges it faces, and the proposed solutions, to conclude that given the current state of the evidence the right anterior temporal lobe is the most promising candidate region for the representation of face identity.

The functional architecture of the ventral temporal cortex and its role in categorization

Visual categorization is thought to occur in the human ventral temporal cortex (VTC), but how this categorization is achieved is still largely unknown. In this Review, we consider the computations and representations that are necessary for categorization and examine how the microanatomical and macroanatomical layout of the VTC might optimize them to achieve rapid and flexible visual categorization. We propose that efficient categorization is achieved by organizing representations in a nested spatial hierarchy in the VTC. This spatial hierarchy serves as a neural infrastructure for the representational hierarchy of visual information in the VTC and thereby enables flexible access to category information at several levels of abstraction.

Neural networks related to dysfunctional face processing in autism spectrum disorder

One of the most consistent neuropsychological findings in autism spectrum disorders (ASD) is a reduced interest in and impaired processing of human faces. We conducted an activation likelihood estimation meta-analysis on 14 functional imaging studies on neural correlates of face processing enrolling a total of 164 ASD patients. Subsequently, normative whole-brain functional connectivity maps for the identified regions of significant convergence were computed for the task-independent (resting-state) and task-dependent (co-activations) state in healthy subjects. Quantitative functional decoding was performed by reference to the BrainMap database. Finally, we examined the overlap of the delineated network with the results of a previous meta-analysis on structural abnormalities in ASD as well as with brain regions involved in human action observation/imitation. We found a single cluster in the left fusiform gyrus showing significantly reduced activation during face processing in ASD across all studies. Both task-dependent and task-independent analyses indicated significant functional connectivity of this region with the temporo-occipital and lateral occipital cortex, the inferior frontal and parietal cortices, the thalamus and the amygdala. Quantitative reverse inference then indicated an association of these regions mainly with face processing, affective processing, and language-related tasks. Moreover, we found that the cortex in the region of right area V5 displaying structural changes in ASD patients showed consistent connectivity with the region showing aberrant responses in the context of face processing. Finally, this network was also implicated in the human action observation/imitation network. In summary, our findings thus suggest a functionally and structurally disturbed network of occipital regions related primarily to face (but potentially also language) processing, which interact with inferior frontal as well as limbic regions and may be the core of aberrant face processing and reduced interest in faces in ASD.

Heterogeneity of FFA responses or multiplexing?

Recent work using cluster analysis of brain activity during movies revealed distinct clusters that respond to faces and different non-face categories in the fusiform face area (FFA). Because of the limited heterogeneity observed, these results could mean that the FFA contains one population of cells capable of representing multiple categories.