Non-conscious recognition of affect in the absence of striate cortex

Conscious perception of fear in faces: Insights from high-density EEG and perceptual awareness scale with threshold stimuli

Contrary to the extensive research on processing subliminal and/or unattended emotional facial expressions, only a minority of studies have investigated the neural correlates of consciousness (NCCs) of emotions conveyed by faces. In the present high-density electroencephalography (EEG) study, we first employed a staircase procedure to identify each participant's perceptual threshold of the emotion expressed by the face and then compared the EEG signals elicited in trials where the participants were aware with the activity elicited in trials where participants were unaware of the emotions expressed by these, otherwise identical, faces. Drawing on existing knowledge of the neural mechanisms of face processing and NCCs, we hypothesized that activity in frontal electrodes would be modulated in relation to participants' awareness of facial emotional content. More specifically, we hypothesized that the NCC of fear seen on someone else's face could be detected as a modulation of a later and more anterior (i.e., at frontal sites) event-related potential (ERP) than the face-sensitive N170. By adopting a data-driven approach and cluster-based statistics to the analysis of EEG signals, the results were clear-cut in showing that visual awareness of fear was associated with the modulation of a frontal ERP component in a 150-300 msec interval. These insights are dissected and contextualized in relation to prevailing theories of visual consciousness and their proposed NCC benchmarks.

Amygdala and cortical gamma-band responses to emotional faces are modulated by attention to valence

The amygdala might support an attentional bias for emotional faces. However, whether and how selective attention toward a specific valence modulates this bias is not fully understood. Likewise, it is unclear whether amygdala and cortical signals respond to emotion and attention in a similar way. We recorded gamma-band activity (GBA, > 30 Hz) intracranially in the amygdalae of 11 patients with epilepsy and collected scalp recordings from 19 healthy participants. We presented angry, neutral, and happy faces randomly, and we denoted one valence as the target. Participants detected happy targets most quickly and accurately. In the amygdala, during attention to negative faces, low gamma-band activity (LGBA, < 90 Hz) increased for angry compared with happy faces from 160 ms. From 220 ms onward, amygdala high gamma-band activity (HGBA, > 90 Hz) was higher for angry and neutral faces than for happy ones. Monitoring neutral faces increased amygdala HGBA for emotions compared with neutral faces from 40 ms. Expressions were not differentiated in GBA while monitoring positive faces. On the scalp, only threat monitoring resulted in expression differentiation. Here, posterior LGBA was increased selectively for angry targets from 60 ms. The data show that GBA differentiation of emotional expressions is modulated by attention to valence: Top-down-controlled threat vigilance coordinates widespread GBA in favor of angry faces. Stimulus-driven emotion differentiation in amygdala GBA occurs during a neutral attentional focus. These findings align with a multi-pathway model of emotion processing and specify the role of GBA in this process.

Unconscious face processing shows a different lateralisation pattern

Faces contain rich information and play a pivotal role in human daily social interactions. Previous studies have provided evidence for the left-visual-field-advantage in face perception. The current study investigated lateralisation when face stimuli were unconsciously presented using interocular suppression methods. The results showed that when an unconscious face stimulus was presented in one side of the visual field without other face stimuli, the unconscious face in the right visual field could break through interocular suppression faster than that in the left (Experiments 1 and 2). However, this right visual field processing advantage reversed to the left when a face stimulus either with (Experiment 2) or without (Experiment 3) visual awareness was presented concurrently with or prior to (Experiment 4) the unconscious face stimuli. These results showed that the unconscious face might have a different processing pattern compared with the conscious face. The relationship between this novel behavioural observation and known functional lateralisation of the fusiform face areas is discussed, suggesting a dynamic interaction between the two cortical hemispheres may underlie the formation of visual awareness for faces. The current study expands our understanding of face processing lateralisation and provided evidence on the asymmetric inter-hemisphere interaction patterns in forming visual awareness.

Influence of stimulus manipulation on conscious awareness of emotional facial expressions in the match-to-sample paradigm

The conscious perception of emotional facial expressions plays an indispensable role in social interaction. However, previous psychological studies have reported inconsistent findings regarding whether conscious awareness is greater for emotional expressions than for neutral expressions. Furthermore, whether this phenomenon is attributable to emotional or visual factors remains unknown. To investigate these issues, we conducted five psychological experiments to test the conscious perception of emotional and neutral facial expressions using the match-to-sample paradigm. Facial stimuli were momentarily presented in the peripheral visual fields while participants read simultaneously presented letters in the central visual fields. The participants selected a perceived face from nine samples. The results of all experiments demonstrated that emotional expressions were more accurately identified than neutral expressions. Furthermore, Experiment 4 showed that angry expressions were identified more accurately than anti-angry expressions, which expressed neutral emotions with comparable physical changes to angry expressions. Experiment 5, testing the interaction between emotional expression and face direction, showed that angry expressions looking toward participants were more accurately identified than those looking away from participants, even though they were physically identical. These results suggest that the conscious awareness of emotional facial expressions is enhanced by their emotional significance.

Fixation-related electrical potentials during a free visual search task reveal the timing of visual awareness

It has been repeatedly claimed that emotional faces readily capture attention, and that they may be processed without awareness. Yet some observations cast doubt on these assertions. Part of the problem may lie in the experimental paradigms employed. Here, we used a free viewing visual search task during electroencephalographic recordings, where participants searched for either fearful or neutral facial expressions among distractor expressions. Fixation-related potentials were computed for fearful and neutral targets and the response compared for stimuli consciously reported or not. We showed that awareness was associated with an electrophysiological negativity starting at around 110 ms, while emotional expressions were distinguished on the N170 and early posterior negativity only when stimuli were consciously reported. These results suggest that during unconstrained visual search, the earliest electrical correlate of awareness may emerge as early as 110 ms, and fixating at an emotional face without reporting it may not produce any unconscious processing.

Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

Facial expression and emotion

Human facial expressions are unique in their ability to express our emotions and communicate them to others. The mimic expression of basic emotions is very similar across different cultures and has also many features in common with other mammals. This suggests a common genetic origin of the association between facial expressions and emotion. However, recent studies also show cultural influences and differences. The recognition of emotions from facial expressions, as well as the process of expressing one's emotions facially, occurs within an extremely complex cerebral network. Due to the complexity of the cerebral processing system, there are a variety of neurological and psychiatric disorders that can significantly disrupt the coupling of facial expressions and emotions. Wearing masks also limits our ability to convey and recognize emotions through facial expressions. Through facial expressions, however, not only "real" emotions can be expressed, but also acted ones. Thus, facial expressions open up the possibility of faking socially desired expressions and also of consciously faking emotions. However, these pretenses are mostly imperfect and can be accompanied by short-term facial movements that indicate the emotions that are actually present (microexpressions). These microexpressions are of very short duration and often barely perceptible by humans, but they are the ideal application area for computer-aided analysis. This automatic identification of microexpressions has not only received scientific attention in recent years, but its use is also being tested in security-related areas. This article summarizes the current state of knowledge of facial expressions and emotions.

Effects of spatially filtered fearful faces and awareness on amygdala activity in adults with autism spectrum disorder: A magnetoencephalography study

BACKGROUND

The amygdala is pivotal in emotional face processing. Spatial frequencies (SFs) of visual images are divided and processed via two visual pathways: low spatial frequency (LSF) information is conveyed by the magnocellular pathway, while the parvocellular pathway carries high spatial frequency information. We hypothesized that altered amygdala activity might underlie atypical social communication caused by changes in both conscious and non-conscious emotional face processing in the brain in individuals with autism spectrum disorder (ASD).

METHOD

Eighteen adults with ASD and 18 typically developing (TD) peers participated in this study. Spatially filtered fearful- and neutral-expression faces and object stimuli were presented under supraliminal or subliminal conditions, and neuromagnetic responses in the amygdala were measured using 306-channel whole-head magnetoencephalography.

RESULTS

The latency of the evoked responses at approximately 200 ms to unfiltered neutral face stimuli and object stimuli in the ASD group was shorter than that in the TD group in the unaware condition. Regarding emotional face processing, the evoked responses in the ASD group were larger than those in the TD group under the aware condition. The later positive shift during 200-500 ms (ARV) was larger than that in the TD group, regardless of awareness. Moreover, ARV to HSF face stimuli was larger than that to the other spatial filtered face stimuli in the aware condition.

CONCLUSION

Regardless of awareness, ARV might reflect atypical face information processing in the ASD brain.

A generative adversarial model of intrusive imagery in the human brain

The mechanisms underlying the subjective experiences of mental disorders remain poorly understood. This is partly due to long-standing over-emphasis on behavioral and physiological symptoms and a de-emphasis of the patient's subjective experiences when searching for treatments. Here, we provide a new perspective on the subjective experience of mental disorders based on findings in neuroscience and artificial intelligence (AI). Specifically, we propose the subjective experience that occurs in visual imagination depends on mechanisms similar to generative adversarial networks that have recently been developed in AI. The basic idea is that a generator network fabricates a prediction of the world, and a discriminator network determines whether it is likely real or not. Given that similar adversarial interactions occur in the two major visual pathways of perception in people, we explored whether we could leverage this AI-inspired approach to better understand the intrusive imagery experiences of patients suffering from mental illnesses such as post-traumatic stress disorder (PTSD) and acute stress disorder. In our model, a nonconscious visual pathway generates predictions of the environment that influence the parallel but interacting conscious pathway. We propose that in some patients, an imbalance in these adversarial interactions leads to an overrepresentation of disturbing content relative to current reality, and results in debilitating flashbacks. By situating the subjective experience of intrusive visual imagery in the adversarial interaction of these visual pathways, we propose testable hypotheses on novel mechanisms and clinical applications for controlling and possibly preventing symptoms resulting from intrusive imagery.

A deep neural network model of the primate superior colliculus for emotion recognition

Although sensory processing is pivotal to nearly every theory of emotion, the evaluation of the visual input as ‘emotional’ (e.g. a smile as signalling happiness) has been traditionally assumed to take place in supramodal ‘limbic’ brain regions. Accordingly, subcortical structures of ancient evolutionary origin that receive direct input from the retina, such as the superior colliculus (SC), are traditionally conceptualized as passive relay centres. However, mounting evidence suggests that the SC is endowed with the necessary infrastructure and computational capabilities for the innate recognition and initial categorization of emotionally salient features from retinal information. Here, we built a neurobiologically inspired convolutional deep neural network (DNN) model that approximates physiological, anatomical and connectional properties of the retino-collicular circuit. This enabled us to characterize and isolate the initial computations and discriminations that the DNN model of the SC can perform on facial expressions, based uniquely on the information it directly receives from the virtual retina. Trained to discriminate facial expressions of basic emotions, our model matches human error patterns and above chance, yet suboptimal, classification accuracy analogous to that reported in patients with V1 damage, who rely on retino-collicular pathways for non-conscious vision of emotional attributes. When presented with gratings of different spatial frequencies and orientations never ‘seen’ before, the SC model exhibits spontaneous tuning to low spatial frequencies and reduced orientation discrimination, as can be expected from the prevalence of the magnocellular (M) over parvocellular (P) projections. Likewise, face manipulation that biases processing towards the M or P pathway affects expression recognition in the SC model accordingly, an effect that dovetails with variations of activity in the human SC purposely measured with ultra-high field functional magnetic resonance imaging. Lastly, the DNN generates saliency maps and extracts visual features, demonstrating that certain face parts, like the mouth or the eyes, provide higher discriminative information than other parts as a function of emotional expressions like happiness and sadness. The present findings support the contention that the SC possesses the necessary infrastructure to analyse the visual features that define facial emotional stimuli also without additional processing stages in the visual cortex or in ‘limbic’ areas.

This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Neural activities during the Processing of unattended and unseen emotional faces: a voxel-wise Meta-analysis

Voxel-wise meta-analyses of task-evoked regional activity were conducted for healthy individuals during the unconscious processing of emotional and neutral faces with an aim to examine whether and how different experimental paradigms influenced brain activation patterns. Studies were categorized into sensory and attentional unawareness paradigms. Thirty-four fMRI studies including 883 healthy participants were identified. Across experimental paradigms, unaware emotional faces elicited stronger activation of the limbic system, striatum, inferior frontal gyrus, insula and the temporal lobe, compared to unaware neutral faces. Crucially, in attentional unawareness paradigms, unattended emotional faces elicited a right-lateralized increased activation (i.e., right amygdala, right temporal pole), suggesting a right hemisphere dominance for processing emotional faces during inattention. By contrast, in sensory unawareness paradigms, unseen emotional faces elicited increased activation of the left striatum, the left amygdala and the right middle temporal gyrus. Additionally, across paradigms, unconsciously processed positive emotions were found associated with more activation in temporal and parietal cortices whereas unconsciously processed negative emotions elicited stronger activation in subcortical regions, compared to neutral faces.

Cautionary Observations Concerning the Introduction of Psychophysiological Biomarkers into Neuropsychiatric Practice

The combination of statistical learning technologies with large databases of psychophysiological data has appropriately generated enthusiastic interest in future clinical applicability. It is argued here that this enthusiasm should be tempered with the understanding that significant obstacles must be overcome before the systematic introduction of psychophysiological measures into neuropsychiatric practice becomes possible. The objective of this study is to identify challenges to this effort. The nonspecificity of psychophysiological measures complicates their use in diagnosis. Low test-retest reliability complicates use in longitudinal assessment, and quantitative psychophysiological measures can normalize in response to placebo intervention. Ten cautionary observations are introduced and, in some instances, possible directions for remediation are suggested.

Do Subliminal Fearful Facial Expressions Capture Attention?

In two experiments, we tested whether fearful facial expressions capture attention in an awareness-independent fashion. In Experiment 1, participants searched for a visible neutral face presented at one of two positions. Prior to the target, a backward-masked and, thus, invisible emotional (fearful/disgusted) or neutral face was presented as a cue, either at target position or away from the target position. If negative emotional faces capture attention in a stimulus-driven way, we would have expected a cueing effect: better performance where fearful or disgusted facial cues were presented at target position than away from the target. However, no evidence of capture of attention was found, neither in behavior (response times or error rates), nor in event-related lateralizations (N2pc). In Experiment 2, we went one step further and used fearful faces as visible targets, too. Thereby, we sought to boost awareness-independent capture of attention by fearful faces. However, still, we found no significant attention-capture effect. Our results show that fearful facial expressions do not capture attention in an awareness-independent way. Results are discussed in light of existing theories.

Rapid processing of threatening faces in the amygdala of nonhuman primates: subcortical inputs and dual roles

A subcortical pathway through the superior colliculus and pulvinar has been proposed to provide the amygdala with rapid but coarse visual information about emotional faces. However, evidence for short-latency, facial expression-discriminating responses from individual amygdala neurons is lacking; even if such a response exists, how it might contribute to stimulus detection is unclear. Also, no definitive anatomical evidence is available for the assumed pathway. Here we showed that ensemble responses of amygdala neurons in monkeys carried robust information about open-mouthed, presumably threatening, faces within 50 ms after stimulus onset. This short-latency signal was not found in the visual cortex, suggesting a subcortical origin. Temporal analysis revealed that the early response contained excitatory and suppressive components. The excitatory component may be useful for sending rapid signals downstream, while the sharpening of the rising phase of later-arriving inputs (presumably from the cortex) by the suppressive component might improve the processing of facial expressions over time. Injection of a retrograde trans-synaptic tracer into the amygdala revealed presumed monosynaptic labeling in the pulvinar and disynaptic labeling in the superior colliculus, including the retinorecipient layers. We suggest that the early amygdala responses originating from the colliculo-pulvino-amygdalar pathway play dual roles in threat detection.

Rehabilitation of visual perception in cortical blindness

Blindness is a common sequela after stroke affecting the primary visual cortex, presenting as a contralesional, homonymous, visual field cut. This can occur unilaterally or, less commonly, bilaterally. While it has been widely assumed that after a brief period of spontaneous improvement, vision loss becomes stable and permanent, accumulating data show that visual training can recover some of the vision loss, even long after the stroke. Here, we review the different approaches to rehabilitation employed in adult-onset cortical blindness (CB), focusing on visual restoration methods. Most of this work was conducted in chronic stroke patients, partially restoring visual discrimination and luminance detection. However, to achieve this, patients had to train for extended periods (usually many months), and the vision restored was not entirely normal. Several adjuvants to training such as noninvasive, transcranial brain stimulation, and pharmacology are starting to be investigated for their potential to increase the efficacy of training in CB patients. However, these approaches are still exploratory and require considerably more research before being adopted. Nonetheless, having established that the adult visual system retains the capacity for restorative plasticity, attention recently turned toward the subacute poststroke period. Drawing inspiration from sensorimotor stroke rehabilitation, visual training was recently attempted for the first time in subacute poststroke patients. It improved vision faster, over larger portions of the blind field, and for a larger number of visual discrimination abilities than identical training initiated more than 6 months poststroke (i.e., in the chronic period). In conclusion, evidence now suggests that visual neuroplasticity after occipital stroke can be reliably recruited by a range of visual training approaches. In addition, it appears that poststroke visual plasticity is dynamic, with a critical window of opportunity in the early postdamage period to attain more rapid, more extensive recovery of a larger set of visual perceptual abilities.

Amygdala function in emotion, cognition, and behavior

The amygdala is a core structure in the anterior medial temporal lobe, with an important role in several brain functions involving memory, emotion, perception, social cognition, and even awareness. As a key brain structure for saliency detection, it triggers and controls widespread modulatory signals onto multiple areas of the brain, with a great impact on numerous aspects of adaptive behavior. Here we discuss the neural mechanisms underlying these functions, as established by animal and human research, including insights provided in both healthy and pathological conditions.

Right Hemisphere Dominance for Unconscious Emotionally Salient Stimuli

The present review will focus on evidence demonstrating the prioritization in visual processing of fear-related signals in the absence of awareness. Evidence in hemianopic patients without any form of blindsight or affective blindsight in classical terms will be presented, demonstrating that fearful faces, via a subcortical colliculo-pulvinar-amygdala pathway, have a privileged unconscious visual processing and facilitate responses towards visual stimuli in the intact visual field. Interestingly, this fear-specific implicit visual processing in hemianopics has only been observed after lesions to the visual cortices in the left hemisphere, while no effect was found in patients with damage to the right hemisphere. This suggests that the subcortical route for emotional processing in the right hemisphere might provide a pivotal contribution to the implicit processing of fear, in line with evidence showing enhanced right amygdala activity and increased connectivity in the right colliculo-pulvinar-amygdala pathway for unconscious fear-conditioned stimuli and subliminal fearful faces. These findings will be discussed within a theoretical framework that considers the amygdala as an integral component of a constant and continuous vigilance system, which is preferentially invoked with stimuli signaling ambiguous environmental situations of biological relevance, such as fearful faces.

Neural Mechanism of Blindsight in a Macaque Model

Some patients with damage to the primary visual cortex (V1) exhibit visuomotor ability, despite loss of visual awareness, a phenomenon termed "blindsight". We review a series of studies conducted mainly in our laboratory on macaque monkeys with unilateral V1 lesioning to reveal the neural pathways underlying visuomotor transformation and the cognitive capabilities retained in blindsight. After lesioning, it takes several weeks for the recovery of visually guided saccades toward the lesion-affected visual field. In addition to the lateral geniculate nucleus, the pathway from the superior colliculus to the pulvinar participates in visuomotor processing in blindsight. At the cortical level, bilateral lateral intraparietal regions become critically involved in the saccade control. These results suggest that the visual circuits experience drastic changes while the monkey acquires blindsight. In these animals, analysis based on signal detection theory adapted to behavior in the "Yes-No" task indicates reduced sensitivity to visual targets, suggesting that visual awareness is impaired. Saccades become less accurate, decisions become less deliberate, and some forms of bottom-up attention are impaired. However, a variety of cognitive functions are retained such as saliency detection during free viewing, top-down attention, short-term spatial memory, and associative learning. These observations indicate that blindsight is not a low-level sensory-motor response, but the residual visual inputs can access these cognitive capabilities. Based on these results we suggest that the macaque model of blindsight replicates type II blindsight patients who experience some "feeling" of objects, which guides cognitive capabilities that we naïvely think are not possible without phenomenal consciousness.

Is the primary visual cortex necessary for blindsight-like behavior? Review of transcranial magnetic stimulation studies in neurologically healthy individuals

The visual pathways that bypass the primary visual cortex (V1) are often assumed to support visually guided behavior in humans in the absence of conscious vision. This conclusion is largely based on findings on patients: V1 lesions cause blindness but sometimes leave some visually guided behaviors intact-this is known as blindsight. With the aim of examining how well the findings on blindsight patients generalize to neurologically healthy individuals, we review studies which have tried to uncover transcranial magnetic stimulation (TMS) induced blindsight. In general, these studies have failed to demonstrate a completely unconscious blindsight-like capacity in neurologically healthy individuals. A possible exception to this is TMS-induced blindsight of stimulus presence or location. Because blindsight in patients is often associated with some form of introspective access to the visual stimulus, and blindsight may be associated with neural reorganization, we suggest that rather than revealing a dissociation between visually guided behavior and conscious seeing, blindsight may reflect preservation or partial recovery of conscious visual perception after the lesion.

The impact of facemasks on emotion recognition, trust attribution and re-identification

Covid-19 pandemics has fostered a pervasive use of facemasks all around the world. While they help in preventing infection, there are concerns related to the possible impact of facemasks on social communication. The present study investigates how emotion recognition, trust attribution and re-identification of faces differ when faces are seen without mask, with a standard medical facemask, and with a transparent facemask restoring visual access to the mouth region. Our results show that, in contrast to standard medical facemasks, transparent masks significantly spare the capability to recognize emotional expressions. Moreover, transparent masks spare the capability to infer trustworthiness from faces with respect to standard medical facemasks which, in turn, dampen the perceived untrustworthiness of faces. Remarkably, while transparent masks (unlike standard masks) do not impair emotion recognition and trust attribution, they seemingly do impair the subsequent re-identification of the same, unmasked, face (like standard masks). Taken together, this evidence supports a dissociation between mechanisms sustaining emotion and identity processing. This study represents a pivotal step in the much-needed analysis of face reading when the lower portion of the face is occluded by a facemask.

EEG Decoding of Dynamic Facial Expressions of Emotion: Evidence from SSVEP and Causal Cortical Network Dynamics

The neural cognitive mechanism in processing static facial expressions (FEs) has been well documented, whereas the one underlying perceiving dynamic faces remains unclear. In this study, Fourier transformation and time-frequency analysis of Electroencephalography (EEG) data were carried out to detect the brain activation underlying dynamic or static FEs while twenty-one participants were viewing dynamic or static faces flicking at 10 Hz. In particular, steady-state visual evoked potentials (SSVEPs) were quantified through spectral power analysis of EEG recordings. Besides, Granger causality (GC) analysis (GCA) was also performed to capture the causal cortical network dynamics during dynamic or static FEs of emotion. It was discovered that the dynamic (from neural to happy (N2H) or vice versa (H2N)) FEs elicited larger SSVEPs than the static ones. Additionally, GCA demonstrated that the H2N case, in which happy FEs were being gradually changed into neutral ones, exhibited larger GC measure during the late processing stage than that from the early stage. Consequently, enhanced SSVEPs and effective brain connectivity for dynamic FEs illustrated that participants might need consume more attentional resources to process the dynamic faces, particularly for the change from happy to neutral faces. The new neural index might facilitate us to better understand the cognitive processing of dynamic and static FEs.

Rapid processing of fearful faces relies on the right amygdala: evidence from individuals undergoing unilateral temporal lobectomy

Facial expressions of emotions have been shown to modulate early ERP components, in particular the N170. The underlying anatomical structure producing these early effects are unclear. In this study, we examined the N170 enhancement for fearful expressions in healthy controls as well as epileptic patients after unilateral left or right amygdala resection. We observed a greater N170 for fearful faces in healthy participants as well as in individuals with left amygdala resections. By contrast, the effect was not observed in patients who had undergone surgery in which the right amygdala had been removed. This result demonstrates that the amygdala produces an early brain response to fearful faces. This early response relies specifically on the right amygdala and occurs at around 170 ms. It is likely that such increases are due to a heightened response of the extrastriate cortex that occurs through rapid amygdalofugal projections to the visual areas.

Blindsight

For over a century, research has demonstrated that damage to primary visual cortex does not eliminate all capacity for visual processing in the brain. From Riddoch's (1917) early demonstration of intact motion processing for blind field stimuli, to the iconic work of Weiskrantz et al. (1974) showing reliable spatial localization, it is clear that secondary visual pathways that bypass V1 carry information to the visual brain that in turn influences behavior. In this chapter, we briefly outline the history and phenomena associated with blindsight, before discussing the nature of the secondary visual pathways that support residual visual processing in the absence of V1. We finish with some speculation as to the functional characteristics of these secondary pathways.

Unconscious processing of emotions and the right hemisphere

Many influential authors maintain that, even if emotions are conscious experiences, the processing of information that produces emotions is usually unconscious. This chapter discusses the nonconscious aspects of emotional processing and the critical role played in them by the right hemisphere. This chapter first reviews the studies that have demonstrated the existence of unconscious or subconscious forms of emotional processing and then discusses the data supporting the hypothesis that the right hemisphere is dominant for the processing of emotions. After these topics are reviewed, the chapter will discuss investigations that have shown that the right and left amygdala have different roles in the processing of emotional stimuli, the former being involved in nonconscious and the latter in conscious forms of emotional learning. This chapter will also address the distinction proposed by Freud between "removed" and "nonremoved" forms of unconscious processing and will consider whether "nonremoved preverbal implicit memories" have a preferential link with the right hemisphere. The possibility that the right hemisphere may play a critical role not only in the formation of nonremoved subconscious memories but also in the development of denial phenomena, resulting from dynamic processes of unconscious repression, will also be discussed.

Subliminal affective priming effect: Dissociated processes for intense versus normal facial expressions

Positive vs. negative intense-facial expressions are difficult to explicitly distinguish; yet, whether they dissociate when subliminally presented remains unclear. Through three experiments using affective priming paradigms, we assessed how intense facial expressions, when presented briefly (17 ms) and masked, influenced following neutral ambiguous words (Experiment 1) or visible facial expressions (Experiments 2&3). We also compared these results with those of using normal facial expressions as primes in each experiment. All experiments indicated masked affective priming effects (biasing valence judgement of neutral words or facilitating reaction time to faces with the same valence as the prime) in normal facial expression, but not those intense ones. Experiment 3 using event related potentials (ERPs) further revealed that two ERP components N250 and LPP were consistent with behavioral changes in the normal condition (larger when valences of primes and targets were different), but inconsistent in the intense condition. Taken together, our results provided behavioral and neural evidence for distinctive processing between normal and intense facial expressions under masked condition.

Where does fear originate in the brain? A coordinate-based meta-analysis of explicit and implicit fear processing

Processing of fear is of crucial importance for human survival and it can generally occur at explicit and implicit conditions. It is worth noting that explicit and implicit fear processing produces different behavioral and neurophysiological outcomes. The present study capitalizes on the Activation Likelihood Estimation (ALE) method of meta-analysis to identify: (a) the "core" network of fear processing in healthy individuals; (b) common and specific neural activations associated with explicit and implicit processing of fear. Following PRISMA guidelines, a total of 92 fMRI and PET studies were included in the meta-analysis. The overall analysis show that the core fear network comprises the amygdala, pulvinar, and fronto-occipital regions. Both implicit and explicit fear processing activated amygdala, declive, fusiform gyrus, and middle frontal gyrus, suggesting that these two types of fear processing share a common neural substrate. Explicit fear processing elicited more activations at the pulvinar and parahippocampal gyrus, suggesting visual attention/orientation and contextual association play important roles during explicit fear processing. In contrast, implicit fear processing elicited more activations at the cerebellum-amygdala-cortical pathway, indicating an 'alarm' system underlying implicit fear processing. These findings have shed light on the neural mechanism underlying fear processing at different levels of awareness.

Fear-related signals are prioritised in visual, somatosensory and spatial systems

The human brain has evolved a multifaceted fear system, allowing threat detection to enable rapid adaptive responses crucial for survival. Although many cortical and subcortical brain areas are believed to be involved in the survival circuits detecting and responding to threat, the amygdala has reportedly a crucial role in the fear system. Here, we review evidence demonstrating that fearful faces, a specific category of salient stimuli indicating the presence of threat in the surrounding, are preferentially processed in the fear system and in the connected sensory cortices, even when they are presented outside of awareness or are irrelevant to the task. In the visual domain, we discuss evidence showing in hemianopic patients that fearful faces, via a subcortical colliculo-pulvinar-amygdala pathway, have a privileged visual processing even in the absence of awareness and facilitate responses towards visual stimuli in the intact visual field. Moreover, evidence showing that somatosensory cortices prioritise fearful-related signals, to the extent that tactile processing is enhanced in the presence of fearful faces, will be also reported. Finally, we will review evidence revealing that fearful faces have a pivotal role in modulating responses in peripersonal space, in line with the defensive functional definition of PPS.

Disrupted dynamic functional connectivity in right amygdalar subregions differentiates bipolar disorder from major depressive disorder

Notwithstanding being the object of a growing field of clinical research, the investigation of the dynamic resting-state functional connectivity alterations in psychiatric illnesses is still in its early days. Current research on major depressive disorder (MDD) and bipolar disorder (BD) has evidenced abnormal resting-state functional connectivity (rsFC), especially in regions subserving emotional processing and regulation such as the amygdala. However, dynamic changes in functional connectivity within the amygdalar subregions in distinguishing BD and MDD has not yet been fully understood. In this paper, we aim at analyzing the patterns characterizing dynamic FC (dFC) in the right amygdala to investigate the differences between similarly depressed BD and MDD. A number of 40 BD patients, 61 MDD patients and 63 healthy controls (HCs) underwent functional magnetic resonance imaging (fMRI) at rest. Using the right-amygdala as seed region, we compared the dFC within three subdivisions, namely, laterobasal (LB), centromedial (CM) and superficial (SF) between all the groups. To do so, one-way ANOVA followed by post-hoc t-tests were employed. Compared to HCs, patients with BD had a decreased dFC between right LB and the left postcentral gyrus as well as an increased dFC between right CM and the right cerebellum.Compared to BD patients, patients with MDD showed a decreased dFC between right CM and the cerebellum and an increased dFC between right LB and the left postcentral gyrus. These findings present initial evidence that abnormal patterns of the right-amygdalar subregions shared by BD and MDD supports the differential pathophysiology of these disorders.

Reflections on Eriksen's seminal essay on discrimination, performance and learning without awareness

Early in his career C.W. Eriksen published in Psychological Review what turned out to be a highly impactful critique on methods and findings on the topic of unconscious influences on discrimination and awareness. His incisive commentary on extant methodology employed at that time - especially the heavy dependence on subjective reports - clearly was heard by others moving forward, as evidenced by the subsequent, lively discussions within the literature concerning the very definition of the notion of unconscious processing. Of equal importance, Eriksen's paper provided an impetus for the development of more refined techniques for manipulating perceptual awareness and for measuring the consequences of those manipulations. My purpose in this essay is to ensure that Eriksen's seminal contributions concerning unconscious phenomena remain within the awareness of the many current investigators working on this popular topic.

The Superior Colliculus and Amygdala Support Evaluation of Face Trait in Blindsight

Humans can respond rapidly to viewed expressions of fear, even in the absence of conscious awareness. This is demonstrated using visual masking paradigms in healthy individuals and in patients with cortical blindness due to damage to the primary visual cortex (V1) - so called affective blindsight. Humans have also been shown to implicitly process facial expressions representing important social dimensions. Two major axes, dominance and trustworthiness, are proposed to characterize the social dimensions of face evaluation. The processing of both types of implicit stimuli is believed to occur via similar subcortical pathways involving the amygdala. However, we do not know whether unconscious processing of more subtle expressions of facial traits can occur in blindsight, and if so, how. To test this, we studied 13 patients with unilateral V1 damage and visual field loss. We assessed their ability to detect and discriminate faces that had been manipulated along two orthogonal axes of trustworthiness and dominance to generate five trait levels inside the blind visual field: dominant, submissive, trustworthy, untrustworthy, and neutral. We compared neural activity and functional connectivity in patients classified as blindsight positive or negative for these stimuli. We found that dominant faces were most likely to be detected above chance, with individuals demonstrating unique interactions between performance and face trait. Only patients with blindsight (n = 8) showed significant preference in the superior colliculus and amygdala for face traits in the blind visual field, and a critical functional connection between the amygdala and superior colliculus in the damaged hemisphere. We also found a significant correlation between behavioral performance and fMRI activity in the amygdala and lateral geniculate nucleus across all participants. Our findings confirm that affective blindsight involving the superior colliculus and amygdala extends to the processing of socially salient but emotionally neutral facial expressions when V1 is damaged. This pathway is distinct from that which supports motion blindsight, as both types of blindsight can exist in the absence of the other with corresponding patterns of residual connectivity.

Danger is in the eyes of the beholder: The effect of visible and invisible affective faces on the judgment of social interactions

Previous studies demonstrated that observation of facial expressions can modulate threat detection while looking at neutral or emotion-related scenes. Similarly, stimuli presented outside conscious awareness could influence social judgments of neutral novel stimuli. The two-fold aim of this study was: i) to evaluate whether observation of seen emotional faces could affect the judgment of social interactions without contextual cues (visible prime condition), and ii) whether this effect could also emerge when the emotional faces were made not visible by means of continuous flash suppression (invisible prime condition). We found that both seen and unseen faces are able to affect the judgment of ambiguous social interactions although this effect was particularly evident when affective faces were clearly visible. The present findings supported the idea that both conscious and unconscious processing of emotional faces have an important role in modulating perceivers' affective state and their judgment of social interactions.

The influence of facial expression at perceptual threshold on electrodermal activity and social comfort distance

Interpersonal distance, an essential component of social interaction, is modulated by the emotion conveyed by others and associated physiological response. However, in modern societies with overcrowded and hyperstimulating environments, we can only surreptitiously glimpse the faces of others in order to quickly make behavioral adjustments. How this impacts social interactions is not yet well understood. In the present study, we investigated this issue by testing whether facial expressions that are difficult to identify modify the physiological response (Electrodermal Activity, EDA) and subsequent judgment of interpersonal comfort distance. We recorded participants' EDA while they provided comfort judgments to interpersonal distances with a Point-Light Walker (PLW). The PLW, with an emotionally neutral gait, moved toward and crossed participants at various distances after the latter were exposed to a negative (anger), positive (happiness) or neutral facial expression presented at the perceptual threshold. Bayesian analyses of the data revealed an increase versus decrease of interpersonal comfort distance with the PLW depending on the negative versus positive emotional valence of the facial expression. They also showed an increase in EDA when the approaching PLW violated interpersonal comfort distance after participants were exposed to an angry facial expression. These effects correlated with the subjective assessment of the arousal of facial expressions. Thus, previous exposure to barely visible facial expressions can alter the representation of social comfort space and the physiological response associated with a violation of interpersonal comfort distances, depending on the valence and arousal of the emotional social stimuli.

Changes in peri-calcarine cortical thickness in blindsight

Blindsight is the ability of patients with primary visual cortex (V1) damage to process information in their clinically blind visual field in the absence of conscious awareness. In addition to those with localized V1 lesions, some patients exhibiting this phenomenon have had a cerebral hemisphere removed or disconnected from the rest of the brain for the treatment of drug-resistant epilepsy (hemispherectomy). Research into the underlying neural substrates of blindsight has long implicated the intact visual cortex in maintaining residual vision and supporting visuo-guided responses to stimuli presented ipsilaterally within the blind visual field while operating outside the geniculo-striate pathway. A recent study demonstrated functional reorganization in the dorsal visual areas of the intact hemisphere, thereby supporting its compensatory role in non-conscious vision. In this study, we used cortical thickness analysis to examine anatomical differences in the visual cortex of the intact hemisphere of three subjects with varying degrees of cortical damage and well documented blindsight: two with a right hemispherectomy (complete and partial), and one with a left V1 lesion. T1-weighted MRI data were obtained for the subjects while control data were chosen from publicly available NKI-dataset to match closely the acquisition parameters of our blindsight cases. Our results show significant increases in cortical thickness in the visual cortex of all blindsight subjects compared to healthy controls, irrespective of age-onset, etiology, and extent of the damage. Our findings add to accumulating evidence from behavioral, functional imaging, and tractography studies of cerebral compensation and reorganization.

•

Examined anatomical changes in the intact visual cortex of rare blindsight patients.

•

First comparison of hemispherectomy and lesion patients to a large control sample.

•

Blindsight subjects show significant increases in peri-calcarine cortical thickness.

•

Similar changes observed despite differences in etiology and age at time of lesion.

•

Increases are possible morphological signs of compensation underlying blindsight.

The Age-Dependent Neural Substrates of Blindsight

Some patients who are considered cortically blind due to the loss of their primary visual cortex (V1) show a remarkable ability to act upon or discriminate between visual stimuli presented to their blind field, without any awareness of those stimuli. This phenomenon is often referred to as blindsight. Despite the range of spared visual abilities, the identification of the pathways mediating blindsight remains an active and contentious topic in the field. In this review, we discuss recent findings of the candidate pathways and their relative contributions to different forms of blindsight across the lifespan to illustrate the varied nature of unconscious visual processing.

A Prototypical Template for Rapid Face Detection Is Embedded in the Monkey Superior Colliculus

Human babies respond preferentially to faces or face-like images. It has been proposed that an innate and rapid face detection system is present at birth before the cortical visual pathway is developed in many species, including primates. However, in primates, the visual area responsible for this process is yet to be unraveled. We hypothesized that the superior colliculus (SC) that receives direct and indirect retinal visual inputs may serve as an innate rapid face-detection system in primates. To test this hypothesis, we examined the responsiveness of monkey SC neurons to first-order information of faces required for face detection (basic spatial layout of facial features including eyes, nose, and mouth), by analyzing neuronal responses to line drawing images of: (1) face-like patterns with contours and properly placed facial features; (2) non-face patterns including face contours only; and (3) nonface random patterns with contours and randomly placed face features. Here, we show that SC neurons respond stronger and faster to upright and inverted face-like patterns compared to the responses to nonface patterns, regardless of contrast polarity and contour shapes. Furthermore, SC neurons with central receptive fields (RFs) were more selective to face-like patterns. In addition, the population activity of SC neurons with central RFs can discriminate face-like patterns from nonface patterns as early as 50 ms after the stimulus onset. Our results provide strong neurophysiological evidence for the involvement of the primate SC in face detection and suggest the existence of a broadly tuned template for face detection in the subcortical visual pathway.

Towards a comparative science of emotion: Affect and consciousness in humans and animals

The componential view of human emotion recognises that affective states comprise conscious, behavioural, physiological, neural and cognitive elements. Although many animals display bodily and behavioural changes consistent with the occurrence of affective states similar to those seen in humans, the question of whether and in which species these are accompanied by conscious experiences remains controversial. Finding scientifically valid methods for investigating markers for the subjective component of affect in both humans and animals is central to developing a comparative understanding of the processes and mechanisms of affect and its evolution and distribution across taxonomic groups, to our understanding of animal welfare, and to the development of animal models of affective disorders. Here, contemporary evidence indicating potential markers of conscious processing in animals is reviewed, with a view to extending this search to include markers of conscious affective processing. We do this by combining animal-focused approaches with investigations of the components of conscious and non-conscious emotional processing in humans, and neuropsychological research into the structure and functions of conscious emotions.

Superior Neuronal Detection of Snakes and Conspecific Faces in the Macaque Medial Prefrontal Cortex

Snakes and conspecific faces are quickly and efficiently detected in primates. Because the medial prefrontal cortex (mPFC) has been implicated in attentional allocation to biologically relevant stimuli, we hypothesized that it might also be highly responsive to snakes and conspecific faces. In this study, neuronal responses in the monkey mPFC were recorded, while monkeys discriminated 8 categories of visual stimuli. Here, we show that the monkey mPFC neuronal responses to snakes and conspecific faces were unique. First, the ratios of the neurons that responded strongly to snakes and monkey faces were greater than those of the neurons that responded strongly to the other stimuli. Second, mPFC neurons responded stronger and faster to snakes and monkey faces than the other categories of stimuli. Third, neuronal responses to snakes were unaffected by low-pass filtering of the images. Finally, activity patterns of responsive mPFC neurons discriminated snakes from the other stimuli in the second 50 ms period and monkey faces in the third period after stimulus onset. These response features indicate that the mPFC processes fast and coarse visual information of snakes and monkey faces, and support the hypothesis that snakes and social environments have shaped the primate visual system over evolutionary time.

Impaired visual search with paradoxically increased facilitation by emotional features after unilateral pulvinar damage

Posterior thalamic pulvinar nuclei have been implicated in different aspects of spatial attention, but their exact role in humans remain unclear. Most neuropsychological studies of attention deficits after pulvinar lesion have concerned single patients or small samples. Here we examined a group of 13 patients with focal damage to posterior thalamus on a visual search task with faces, allowing us to test several hypotheses concerning pulvinar function in controlling attention to visually salient or emotionally significant stimuli. Our results identified two subgroups of thalamic patients with distinct patterns of attentional responsiveness to emotional and colour features in face targets. One group with lesions located in anterior and ventral portions of thalamus showed intact performance, with a normal facilitation of visual search for faces with emotional (fearful or happy) expressions on both side of space, similar to healthy controls. By contrast, a second group showed a slower and poorer detection of face targets, most severe for neutral faces, but with a paradoxically enhanced facilitation by both colour and emotional features. This second group had lesions centred on the pulvinar, involving mainly the dorso-medial sectors in patients showing enhanced effects of colour features, but extending to more dorso-lateral sectors in those with enhanced effects of emotional features. These findings reveal that pulvinar nuclei are not critical for orienting attention to emotionally or visually salient features, but instead provide new evidence in support of previous hypotheses suggesting an important role in controlling attention in visual scenes with distracting information.

What drives prioritized visual processing? A motivational relevance account

Emotion is fundamental to our being, and an essential aspect guiding behavior when rapid responding is required. This includes whether we approach or avoid a stimulus, and the accompanying physiological responses. A common tenet is that threat-related content drives stimulus processing and biases visual attention, so that rapid responding can be initiated. In this paper, it will be argued instead that prioritization of threatening stimuli should be encompassed within a motivational relevance framework. To more fully understand what is, or is not, prioritized for visual processing one must, however, additionally consider: (i) stimulus ambiguity and perceptual saliency; (ii) task demands, including both perceptual load and cognitive load; and (iii) endogenous/affective states of the individual. Combined with motivational relevance, this then leads to a multifactorial approach to understanding the drivers of prioritized visual processing. This accords with current recognition that the brain basis allowing for visual prioritization is also multifactorial, including transient, dynamic and overlapping networks. Taken together, the paper provides a reconceptualization of how "emotional" information prioritizes visual processing.

Unconscious processing of coarse visual information during anticipatory threat

Rapid detection of threats has been proposed to rely on automatic processing of their coarse visual features. However, it remains unclear whether such a mechanism is restricted to detection of threat cues, or whether it reflects a broader sensitivity to even neutral coarse visual information features during states of threat. We used a backward masking task in which participants discriminated the orientation of subliminally presented low (3 cpd) and high (6 cpd) spatial frequency gratings, under threat (of shock) and safe conditions. Visual awareness of the gratings was assessed objectively using an additional localization task. When participants were unaware of the gratings, above chance and improved discrimination of low-spatial frequency gratings was observed under threat compared to safe trials. These findings demonstrate unconscious processing of neutral coarse visual information during threat state, supporting the view that automatic threat detection may rely on a general facilitation of coarse features irrespective of threat content.

Neuronal mechanisms of motion detection underlying blindsight assessed by functional magnetic resonance imaging (fMRI)

Brain imaging offers a valuable tool to observe functional brain plasticity by showing how sensory inputs reshape cortical activations after a visual impairment. Following a unilateral post-chiasmatic lesion affecting the visual cortex, patients may suffer a contralateral visual loss referred to homonymous hemianopia. Nevertheless, these patients preserve the ability to unconsciously detect, localize and discriminate visual stimuli presented in their impaired visual field. To investigate this paradox, known as blindsight, we conducted a study using functional magnetic resonance imaging (fMRI) to evaluate the structural and functional impact of such lesion in a 33-year old patient (ML), who suffers a complete right hemianopia without macular sparing and showing strong evidences of blindsight. We thus performed whole brain and sliced thalamic fMRI scan sequences during an event-related motion detection task. We provided evidence of the neuronal fingerprint of blindsight by acquiring and associating neural correlates, specific structures and functional networks of the midbrain during blindsight performances which may help to better understand this condition. Accurate performance demonstrated the presence of residual vision and the ability to unconsciously perceive motion presented in the blind hemifield, although her reaction time was significantly higher in her blind-field. When the normal hemifield was stimulated, we observed significant contralateral activations in primary and secondary visual areas as well as motion specific areas, such as the supramarginal gyrus and middle temporal area. We also demonstrated sub-thalamic activations within the superior colliculi (SC) and the pulvinar. These results suggest a role of secondary subcortical structures in normal spontaneous motion detection. In a similar way, when the lesioned hemifield was stimulated, we observed contralateral activity in extrastriate areas with no activation of the primary lesioned visual cortex. Moreover, we observed activations within the SC when the blind hemifield was stimulated. However, we observed unexpected ipsilateral activations within the same motion specific areas, as well as bilateral frontal activations. These results highlight the importance of abnormal secondary pathways bypassing the primary visual area (V1) in residual vision. This reorganization in the structure and function of the visual pathways correlates with behavioral changes, thus offering a plausible explanation for the blindsight phenomenon. Our results may potentially impact the development of rehabilitation strategies to target subcortical pathways.