Network dynamics of human face perception

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Event-Related Phase Synchronization Propagates Rapidly across Human Ventral Visual Cortex

Visual inputs to early visual cortex integrate with semantic, linguistic and memory inputs in higher visual cortex, in a manner that is rapid and accurate, and enables complex computations such as face recognition and word reading. This implies the existence of fundamental organizational principles that enable such efficiency. To elaborate on this, we performed intracranial recordings in 82 individuals while they performed tasks of varying visual and cognitive complexity. We discovered that visual inputs induce highly organized posterior-to-anterior propagating patterns of phase modulation across the ventral occipitotemporal cortex. At individual electrodes there was a stereotyped temporal pattern of phase progression following both stimulus onset and offset, consistent across trials and tasks. The phase of low frequency activity in anterior regions was predicted by the prior phase in posterior cortical regions. This spatiotemporal propagation of phase likely serves as a feed-forward organizational influence enabling the integration of information across the ventral visual stream. This phase modulation manifests as the early components of the event related potential; one of the most commonly used measures in human electrophysiology. These findings illuminate fundamental organizational principles of the higher order visual system that enable the rapid recognition and characterization of a variety of inputs.

NetDI: Methodology Elucidating the Role of Power and Dynamical Brain Network Features That Underpin Word Production

Canonical language models describe eloquent function as the product of a series of cognitive processes, typically characterized by the independent activation profiles of focal brain regions. In contrast, more recent work has suggested that the interactions between these regions, the cortical networks of language, are critical for understanding speech production. We investigated the cortical basis of picture naming (PN) with human intracranial electrocorticography (ECoG) recordings and direct cortical stimulation (DCS), adjudicating between two competing hypotheses: are task-specific cognitive functions discretely computed within well-localized brain regions or rather by distributed networks? The time resolution of ECoG allows direct comparison of intraregional activation measures [high gamma (h_γ) power] with graph theoretic measures of interregional dynamics. We developed an analysis framework, network dynamics using directed information (NetDI), using information and graph theoretic tools to reveal spatiotemporal dynamics at multiple scales: coarse, intermediate, and fine. Our analysis found novel relationships between the power profiles and network measures during the task. Furthermore, validation using DCS indicates that such network parameters combined with h_γ power are more predictive than h_γ power alone, for identifying critical language regions in the brain. NetDI reveals a high-dimensional space of network dynamics supporting cortical language function, and to account for disruptions to language function observed after neurosurgical resection, traumatic injury, and degenerative disease.

Social Perception of Faces: Brain Imaging and Subjective Ratings

The aim of this study was to investigate how a female face is perceived in terms of its attractiveness, dominance, health, femininity-masculinity, and maturity in direct relation to the body fat percentage (BFP) conveyed by the face. To compare how young adults (ages 18 to 35) respond to different levels of body fat percentage both subjectively and objectively we collected survey ratings and electroencephalography (EEG) data across five different levels of BFP from 40 participants. We adapted the experimental design from a prior behavioral study and used calibrated and morphed female face images of five different BFP levels. The results of the survey are in consensus with the previous study and assessed to be a successful replication. From the EEG data, event-related potentials (ERPs) were extracted from one electrode location (right occipitotemporal brain region) known to be particularly sensitive to face-stimuli. We found statistically significant differences in the amplitudes of the P200 component (194 ms post stimulus onset) between the thickest face and all four other BFP conditions, and in the amplitudes of the N300 component (274 ms post stimulus onset) between the average face and three other BFP conditions. As expected, there were no significant differences among the N170 amplitudes of all five BFP conditions since this ERP component simply reflects the processing of faces in general. From these results, we can infer that holistic face encoding characterized by the N170 component in the right occipitotemporal area is followed by serial evaluative processes, whose categorical and qualitative matrix and spatiotemporal dynamics should be further explored in future studies, especially in relation to the social constructs that were focused on in this study.

The occipital face area is causally involved in identity-related visual-semantic associations

Faces are processed in a network of areas within regions of the ventral visual stream. However, familiar faces typically are characterized by additional associated information, such as episodic memories or semantic biographical information as well. The acquisition of such non-sensory, identity-specific knowledge plays a crucial role in our ability to recognize and identify someone we know. The occipital face area (OFA), an early part of the core face-processing network, is recently found to be involved in the formation of identity-specific memory traces but it is currently unclear if this role is limited to unimodal visual information. The current experiments used transcranial magnetic stimulation (TMS) to test whether the OFA is involved in the association of a face with identity-specific semantic information, such as the name or job title of a person. We applied an identity-learning task where unfamiliar faces were presented together with a name and a job title in the first encoding phase. Simultaneously, TMS pulses were applied either to the left or right OFA or to Cz, as a control. In the subsequent retrieval phase, the previously seen faces were presented either with two names or with two job titles and the task of the participants was to select the semantic information previously learned. We found that the stimulation of the right or left OFA reduced subsequent retrieval performance for the face-associated job titles. This suggests a causal role of the OFA in the association of faces and related semantic information. Furthermore, in contrast to prior findings, we did not observe hemispherical differences of the TMS intervention, suggesting a similar role of the left and right OFAs in the formation of the visual-semantic associations. Our results suggest the necessity to reconsider the hierarchical face-perception models and support the distributed and recurrent models.

The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Although face processing has been studied extensively, the dynamics of how face-selective cortical areas are engaged remains unclear. Here, we uncovered the timing of activation in core face-selective regions using functional Magnetic Resonance Imaging and Magnetoencephalography in humans. Processing of normal faces started in the posterior occipital areas and then proceeded to anterior regions. This bottom-up processing sequence was also observed even when internal facial features were misarranged. However, processing of two-tone Mooney faces lacking explicit prototypical facial features engaged top-down projection from the right posterior fusiform face area to right occipital face area. Further, face-specific responses elicited by contextual cues alone emerged simultaneously in the right ventral face-selective regions, suggesting parallel contextual facilitation. Together, our findings chronicle the precise timing of bottom-up, top-down, as well as context-facilitated processing sequences in the occipital-temporal face network, highlighting the importance of the top-down operations especially when faced with incomplete or ambiguous input.

Network dynamics of Broca's area during word selection

Current models of word-production in Broca’s area (i.e. left ventro-lateral prefrontal cortex, VLPFC) posit that sequential and staggered semantic, lexical, phonological and articulatory processes precede articulation. Using millisecond-resolution intra-cranial recordings, we evaluated spatiotemporal dynamics and high frequency functional interconnectivity between left VLPFC regions during single-word production. Through the systematic variation of retrieval, selection, and phonological loads, we identified specific activation profiles and functional coupling patterns between these regions that fit within current psycholinguistic theories of word production. However, network interactions underpinning these processes activate in parallel (not sequentially), while the processes themselves are indexed by specific changes in network state. We found evidence that suggests that pars orbitalis is coupled with pars triangularis during lexical retrieval, while lexical selection is terminated via coupled activity with M1 at articulation onset. Taken together, this work reveals that speech production relies on very specific inter-regional couplings in rapid sequence in the language dominant hemisphere.

A lexical semantic hub for heteromodal naming in middle fusiform gyrus

Semantic memory underpins our understanding of objects, people, places, and ideas. Anomia, a disruption of semantic memory access, is the most common residual language disturbance and is seen in dementia and following injury to temporal cortex. While such anomia has been well characterized by lesion symptom mapping studies, its pathophysiology is not well understood. We hypothesize that inputs to the semantic memory system engage a specific heteromodal network hub that integrates lexical retrieval with the appropriate semantic content. Such a network hub has been proposed by others, but has thus far eluded precise spatiotemporal delineation. This limitation in our understanding of semantic memory has impeded progress in the treatment of anomia. We evaluated the cortical structure and dynamics of the lexical semantic network in driving speech production in a large cohort of patients with epilepsy using electrocorticography (n = 64), functional MRI (n = 36), and direct cortical stimulation (n = 30) during two generative language processes that rely on semantic knowledge: visual picture naming and auditory naming to definition. Each task also featured a non-semantic control condition: scrambled pictures and reversed speech, respectively. These large-scale data of the left, language-dominant hemisphere uniquely enable convergent, high-resolution analyses of neural mechanisms characterized by rapid, transient dynamics with strong interactions between distributed cortical substrates. We observed three stages of activity during both visual picture naming and auditory naming to definition that were serially organized: sensory processing, lexical semantic processing, and articulation. Critically, the second stage was absent in both the visual and auditory control conditions. Group activity maps from both electrocorticography and functional MRI identified heteromodal responses in middle fusiform gyrus, intraparietal sulcus, and inferior frontal gyrus; furthermore, the spectrotemporal profiles of these three regions revealed coincident activity preceding articulation. Only in the middle fusiform gyrus did direct cortical stimulation disrupt both naming tasks while still preserving the ability to repeat sentences. These convergent data strongly support a model in which a distinct neuroanatomical substrate in middle fusiform gyrus provides access to object semantic information. This under-appreciated locus of semantic processing is at risk in resections for temporal lobe epilepsy as well as in trauma and strokes that affect the inferior temporal cortex-it may explain the range of anomic states seen in these conditions. Further characterization of brain network behaviour engaging this region in both healthy and diseased states will expand our understanding of semantic memory and further development of therapies directed at anomia.

Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG

Humans recognize faces with ease, despite the complexity of the task and of the visual system which underlies it. Different spatial regions, including both the core and extended face processing networks, and distinct temporal stages of processing have been implicated in face recognition, but there is ongoing controversy regarding the extent to which the mechanisms for recognizing a familiar face differ from those for an unfamiliar face. Here, we used electroencephalogram (EEG) and flicker SSVEP, a high signal-to-noise approach, and searchlight decoding methods to elucidate the mechanisms mediating the processing of familiar and unfamiliar faces in the time domain. Familiar and unfamiliar faces were presented periodically at 15 Hz, 6 Hz and 3.75 Hz either upright or inverted in separate blocks, with the rationale that faster frequencies require shorter processing times per image and tap into fundamentally different levels of visual processing. The 15 Hz trials, likely to reflect early visual processing, exhibited enhanced neural responses for familiar over unfamiliar face trials, but only when the faces were upright. In contrast, decoding methods revealed similar classification accuracies for upright and inverted faces for both familiar and unfamiliar faces. For the 6 Hz frequency, familiar faces had lower amplitude responses than unfamiliar faces, and decoding familiarity was more accurate for upright compared with inverted faces. Finally, the 3.75 Hz frequency revealed no main effects of familiarity, but decoding showed significant correlations with behavioral ratings of face familiarity, suggesting that activity evoked by this slow presentation frequency reflected higher-level, cognitive aspects of familiarity processing. This three-way dissociation between frequencies reveals that fundamentally different stages of the visual hierarchy are modulated by face familiarity. The combination of experimental and analytical approaches used here represent a novel method for elucidating spatio-temporal characteristics within the visual system.