Fear-Related Signals in the Primary Visual Cortex

Understanding human amygdala function with artificial neural networks

The amygdala is a cluster of subcortical nuclei that receives diverse sensory inputs and projects to the cortex, midbrain, and other subcortical structures. Numerous accounts of amygdalar contributions to social and emotional behavior have been offered, yet an overarching description of amygdala function remains elusive. Here we adopt a computationally explicit framework that aims to develop a model of amygdala function based on the types of sensory inputs it receives, rather than individual constructs such as threat, arousal, or valence. Characterizing human fMRI signal acquired as male and female participants viewed a full-length film, we developed encoding models that predict both patterns of amygdala activity and self-reported valence evoked by naturalistic images. We use deep image synthesis to generate artificial stimuli that distinctly engage encoding models of amygdala subregions that systematically differ from one another in terms of their low-level visual properties. These findings characterize how the amygdala compresses high-dimensional sensory inputs into low-dimensional representations relevant for behavior.Significance Statement The amygdala is a cluster of subcortical nuclei critical for motivation, emotion, and social behavior. Characterizing the contribution of the amygdala to behavior has been challenging due to its structural complexity, broad connectivity, and functional heterogeneity. Here we use a combination of human neuroimaging and computational modeling to investigate how visual inputs relate to low-dimensional representations encoded in the amygdala. We find that the amygdala encodes an array of visual features, which systematically vary across specific nuclei and relate to the affective properties of the sensory environment.

Quantifying population-level neural tuning functions using Ricker wavelets and the Bayesian bootstrap

BACKGROUND

Experience changes visuo-cortical tuning. In humans, re-tuning has been studied during aversive generalization learning, in which the similarity of generalization stimuli (GSs) with a conditioned threat cue (CS+) is used to quantify tuning functions. Previous work utilized pre-defined tuning shapes (generalization and sharpening patterns). This approach may constrain the ways in which re-tuning can be characterized since the tuning patterns may not match the prototypical functions.

NEW METHOD

The present study proposes a flexible and data-driven method for precisely quantifying changes in tuning based on the Ricker wavelet function and the Bayesian bootstrap. This method was applied to EEG and psychophysics data from an aversive generalization learning paradigm.

RESULTS

The Ricker wavelet model fitted the steady-state visual event potentials (ssVEP), alpha-band power, and detection accuracy data well. A Morlet wavelet function was used for comparison and fit the data better in some situations, but was more challenging to interpret. The pattern of re-tuning in the EEG data, predicted by the Ricker model, resembled the shapes of the best fitting a-priori patterns.

COMPARISON WITH EXISTING METHODS

Although the re-tuning shape modeled by the Ricker function resembled the pre-defined shapes, the Ricker approach led to greater Bayes factors and more interpretable results compared to a-priori models. The Ricker approach was more easily fit and led to more interpretable results than a Morlet wavelet model.

CONCLUSION

This work highlights the promise of the current method for capturing the precise nature of visuo-cortical tuning, unconstrained by the implementation of a-priori models.

Understanding human amygdala function with artificial neural networks

The amygdala is a cluster of subcortical nuclei that receives diverse sensory inputs and projects to the cortex, midbrain and other subcortical structures. Numerous accounts of amygdalar contributions to social and emotional behavior have been offered, yet an overarching description of amygdala function remains elusive. Here we adopt a computationally explicit framework that aims to develop a model of amygdala function based on the types of sensory inputs it receives, rather than individual constructs such as threat, arousal, or valence. Characterizing human fMRI signal acquired as participants viewed a full-length film, we developed encoding models that predict both patterns of amygdala activity and self-reported valence evoked by naturalistic images. We use deep image synthesis to generate artificial stimuli that distinctly engage encoding models of amygdala subregions that systematically differ from one another in terms of their low-level visual properties. These findings characterize how the amygdala compresses high-dimensional sensory inputs into low-dimensional representations relevant for behavior.

Quantifying Population-level Neural Tuning Functions Using Ricker Wavelets and the Bayesian Bootstrap

Experience changes the tuning of sensory neurons, including neurons in retinotopic visual cortex, as evident from work in humans and non-human animals. In human observers, visuo-cortical re-tuning has been studied during aversive generalization learning paradigms, in which the similarity of generalization stimuli (GSs) with a conditioned threat cue (CS+) is used to quantify tuning functions. This work utilized pre-defined tuning shapes reflecting prototypical generalization (Gaussian) and sharpening (Difference-of-Gaussians) patterns. This approach may constrain the ways in which re-tuning can be characterized, for example if tuning patterns do not match the prototypical functions or represent a mixture of functions. The present study proposes a flexible and data-driven method for precisely quantifying changes in neural tuning based on the Ricker wavelet function and the Bayesian bootstrap. The method is illustrated using data from a study in which university students (n = 31) performed an aversive generalization learning task. Oriented gray-scale gratings served as CS+ and GSs and a white noise served as the unconditioned stimulus (US). Acquisition and extinction of the aversive contingencies were examined, while steady-state visual event potentials (ssVEP) and alpha-band (8-13 Hz) power were measured from scalp EEG. Results showed that the Ricker wavelet model fitted the ssVEP and alpha-band data well. The pattern of re-tuning in ssVEP amplitude across the stimulus gradient resembled a generalization (Gaussian) shape in acquisition and a sharpening (Difference-of-Gaussian) shape in an extinction phase. As expected, the pattern of re-tuning in alpha-power took the form of a generalization shape in both phases. The Ricker-based approach led to greater Bayes factors and more interpretable results compared to prototypical tuning models. The results highlight the promise of the current method for capturing the precise nature of visuo-cortical tuning functions, unconstrained by the exact implementation of prototypical a-priori models.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that (1) in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and (2) lesioning these neurons by setting their output to 0 or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Dynamic threat-reward neural processing under semi-naturalistic ecologically relevant scenarios

Studies of affective neuroscience have typically employed highly controlled, static experimental paradigms to investigate the neural underpinnings of threat and reward processing in the brain. Yet our knowledge of affective processing in more naturalistic settings remains limited. Specifically, affective studies generally examine threat and reward features separately and under brief time periods, despite the fact that in nature organisms are often exposed to the simultaneous presence of threat and reward features for extended periods. To study the neural mechanisms of threat and reward processing under distinct temporal profiles, we created a modified version of the PACMAN game that included these environmental features. We also conducted two automated meta-analyses to compare the findings from our semi-naturalistic paradigm to those from more constrained experiments. Overall, our results revealed a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. Additionally, these systems broadly overlapped with corresponding meta-analyses, with the notable absence of the amygdala in our findings. Together, these findings suggest a shared system for salience processing that reveals a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The broad correspondence of our findings to meta-analyses, consisting of more tightly controlled paradigms, illustrates how semi-naturalistic studies can corroborate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.

Emergence of Emotion Selectivity in Deep Neural Networks Trained to Recognize Visual Objects

Recent neuroimaging studies have shown that the visual cortex plays an important role in representing the affective significance of visual input. The origin of these affect-specific visual representations is debated: they are intrinsic to the visual system versus they arise through reentry from frontal emotion processing structures such as the amygdala. We examined this problem by combining convolutional neural network (CNN) models of the human ventral visual cortex pre-trained on ImageNet with two datasets of affective images. Our results show that in all layers of the CNN models, there were artificial neurons that responded consistently and selectively to neutral, pleasant, or unpleasant images and lesioning these neurons by setting their output to zero or enhancing these neurons by increasing their gain led to decreased or increased emotion recognition performance respectively. These results support the idea that the visual system may have the intrinsic ability to represent the affective significance of visual input and suggest that CNNs offer a fruitful platform for testing neuroscientific theories.

Emotion category-modulated interpretation bias in perceiving ambiguous facial expressions

In contrast to prototypical facial expressions, we show less perceptual tolerance in perceiving vague expressions by demonstrating an interpretation bias, such as more frequent perception of anger or happiness when categorizing ambiguous expressions of angry and happy faces that are morphed in different proportions and displayed under high- or low-quality conditions. However, it remains unclear whether this interpretation bias is specific to emotion categories or reflects a general negativity versus positivity bias and whether the degree of this bias is affected by the valence or category of two morphed expressions. These questions were examined in two eye-tracking experiments by systematically manipulating expression ambiguity and image quality in fear- and sad-happiness faces (Experiment 1) and by directly comparing anger-, fear-, sadness-, and disgust-happiness expressions (Experiment 2). We found that increasing expression ambiguity and degrading image quality induced a general negativity versus positivity bias in expression categorization. The degree of negativity bias, the associated reaction time and face-viewing gaze allocation were further manipulated by different expression combinations. It seems that although we show a viewing condition-dependent bias in interpreting vague facial expressions that display valence-contradicting expressive cues, it appears that the perception of these ambiguous expressions is guided by a categorical process similar to that involved in perceiving prototypical expressions.

Decoding Emotion From High-frequency Steady State Visual Evoked Potential (SSVEP)

BACKGROUND

Steady-state visual evoked potential (SSVEP) by flickering sensory stimuli has been widely applied in the brain-machine interface (BMI). Yet, it remains largely unexplored whether affective information could be decoded from the signal of SSVEP, especially from the frequencies higher than the critical flicker frequency (an upper-frequency limit one can see the flicker).

NEW METHOD

Participants fixated on visual stimuli presented at 60Hz above the critical flicker frequency. The stimuli were pictures with different affective valance (positive, neutral, negative) in distinctive semantic categories (human, animal, scene). SSVEP entrainment in the brain evoked by the flickering stimuli at 60Hz was used to decode the affective and semantic information.

RESULTS

During the presentation of stimuli (1s), the affective valance could be decoded from the SSVEP signals at 60Hz, while the semantic categories could not. In contrast, neither the affective nor the semantic information could be decoded from the brain signal 1second before the onset of stimuli.

COMPARISON WITH EXISTING METHOD(S)

Previous studies focused mainly on EEG activity tagged at frequencies lower than the critical flickering frequency and investigated whether the affective valence of stimuli drew participants' attention. The current study was the first to use SSVEP signals from high-frequency (60Hz) above the critical flickering frequency to decode affective information from stimuli. The high-frequency flickering was invisible and thus substantially reduced the fatigue of participants.

CONCLUSIONS

We found that affective information could be decoded from high-frequency SSVEP and the current finding could be added to designing affective BMI in the future.

Sensing fear: fast and precise threat evaluation in human sensory cortex

Animal models of threat processing have evolved beyond the amygdala to incorporate a distributed neural network. In human research, evidence has intensified in recent years to challenge the canonical threat circuitry centered on the amygdala, urging revision of threat conceptualization. A strong surge of research into threat processing in the sensory cortex in the past decade has generated particularly useful insights to inform the reconceptualization. Here, synthesizing findings from both animal and human research, we highlight sensitive, specific, and adaptable threat representations in the sensory cortex, arising from experience-based sculpting of sensory coding networks. We thus propose that the human sensory cortex can drive smart (fast and precise) threat evaluation, producing threat-imbued sensory afferents to elicit network-wide threat responses.

Contextual Fear Learning and Extinction in the Primary Visual Cortex of Mice

Fear memory contextualization is critical for selecting adaptive behavior to survive. Contextual fear conditioning (CFC) is a classical model for elucidating related underlying neuronal circuits. The primary visual cortex (V1) is the primary cortical region for contextual visual inputs, but its role in CFC is poorly understood. Here, our experiments demonstrated that bilateral inactivation of V1 in mice impaired CFC retrieval, and both CFC learning and extinction increased the turnover rate of axonal boutons in V1. The frequency of neuronal Ca2+ activity decreased after CFC learning, while CFC extinction reversed the decrease and raised it to the naïve level. Contrary to control mice, the frequency of neuronal Ca2+ activity increased after CFC learning in microglia-depleted mice and was maintained after CFC extinction, indicating that microglial depletion alters CFC learning and the frequency response pattern of extinction-induced Ca2+ activity. These findings reveal a critical role of microglia in neocortical information processing in V1, and suggest potential approaches for cellular-based manipulation of acquired fear memory.

Layer-specific, retinotopically-diffuse modulation in human visual cortex in response to viewing emotionally expressive faces

Viewing faces that are perceived as emotionally expressive evokes enhanced neural responses in multiple brain regions, a phenomenon thought to depend critically on the amygdala. This emotion-related modulation is evident even in primary visual cortex (V1), providing a potential neural substrate by which emotionally salient stimuli can affect perception. How does emotional valence information, computed in the amygdala, reach V1? Here we use high-resolution functional MRI to investigate the layer profile and retinotopic distribution of neural activity specific to emotional facial expressions. Across three experiments, human participants viewed centrally presented face stimuli varying in emotional expression and performed a gender judgment task. We found that facial valence sensitivity was evident only in superficial cortical layers and was not restricted to the retinotopic location of the stimuli, consistent with diffuse feedback-like projections from the amygdala. Together, our results provide a feedback mechanism by which the amygdala directly modulates activity at the earliest stage of visual processing.

Limited Pairings of Electrical Micro-stimulation of the Ventral Tegmental Area and a Visual Stimulus Enhance Visual Cortical Responses

Previous studies demonstrated that pairing a visual stimulus and electrical micro-stimulation of the ventral tegmental area (VTA-EM) for multiple days is sufficient to induce visual cortical plasticity and changes perception. However, a brief epoch of VTA-EM-stimulus pairing within a single day has been shown to result in a behavioral preference for the paired stimulus. Here, we investigated whether a brief single-day session of VTA-EM-stimulus pairings is sufficient to induce changes in visual cortical responses. We examined macaque posterior inferior temporal (PIT) cortex because previous studies demonstrated response changes after VTA-EM stimulus pairing in that area. Multi-unit recordings in PIT were interleaved with VTA-EM-stimulus pairing epochs. During the short VTA-EM-stimulus pairing epochs (60 pairings), one image (fractal) was paired with VTA-EM (STIM) whereas another, unpaired fractal was presented as control. Two other fractals (dummies) were presented only during the recordings. The difference in response between the STIM and control fractals already increased after the first VTA-EM-stimulus pairing epoch, reflecting a relative increase of the response to the STIM fractal. However, the response to the STIM fractal did not increase further with more VTA-EM-stimulus pairing epochs. The relative increase in firing rate for the paired fractal was present early in the response, in line with a local/ bottom-up origin. These effects were absent when comparing the responses to the dummies pre- and post-VTA-EM. This study shows that pairing a visual image and VTA-EM in a brief single-day session is sufficient to increase the response for the paired image in macaque PIT.

A Model of the Early Visual System Based on Parallel Spike-Sequence Detection, Showing Orientation Selectivity

Simple Summary

A computational model of primates’ early visual processing, showing orientation selectivity, is presented. The system importantly integrates two key elements: (1) a neuromorphic spike-decoding structure that considerably resembles the circuitry between layers IV and II/III of the primary visual cortex, both in topology and operation; (2) the plasticity of intrinsic excitability, to embed recent findings about the operation of the same area. The model is proposed as a tool for the analysis and reproduction of the orientation selectivity phenomenon, whose underlying neuronal-level computational mechanisms are today the subject of intense scrutiny. In response to rotated Gabor patches the model is able to exhibit realistic orientation tuning curves and to reproduce responses similar to those found in neurophysiological recordings from the primary visual cortex obtained under the same task, considering different stages of the network. This demonstrates its aptness to capture the mechanisms underlying the evoked response in the primary visual cortex. Our tool is available online, and can be expanded to other experiments using a dedicated software library developed by the authors, to elucidate the computational mechanisms underlying orientation selectivity.

Abstract

Since the first half of the twentieth century, numerous studies have been conducted on how the visual cortex encodes basic image features. One of the hallmarks of basic feature extraction is the phenomenon of orientation selectivity, of which the underlying neuronal-level computational mechanisms remain partially unclear despite being intensively investigated. In this work we present a reduced visual system model (RVSM) of the first level of scene analysis, involving the retina, the lateral geniculate nucleus and the primary visual cortex (V1), showing orientation selectivity. The detection core of the RVSM is the neuromorphic spike-decoding structure MNSD, which is able to learn and recognize parallel spike sequences and considerably resembles the neuronal microcircuits of V1 in both topology and operation. This structure is equipped with plasticity of intrinsic excitability to embed recent findings about V1 operation. The RVSM, which embeds 81 groups of MNSD arranged in 4 oriented columns, is tested using sets of rotated Gabor patches as input. Finally, synthetic visual evoked activity generated by the RVSM is compared with real neurophysiological signal from V1 area: (1) postsynaptic activity of human subjects obtained by magnetoencephalography and (2) spiking activity of macaques obtained by multi-tetrode arrays. The system is implemented using the NEST simulator. The results attest to a good level of resemblance between the model response and real neurophysiological recordings. As the RVSM is available online, and the model parameters can be customized by the user, we propose it as a tool to elucidate the computational mechanisms underlying orientation selectivity.

Fear conditioning prompts sparser representations of conditioned threat in primary visual cortex

Repeated exposure to threatening stimuli alters sensory responses. We investigated the underlying neural mechanism by re-analyzing previously published simultaneous electroencephalogram-functional magnetic resonance imaging (EEG-fMRI) data from humans viewing oriented gratings during Pavlovian fear conditioning. In acquisition, one grating (CS+) was paired with a noxious noise, the unconditioned stimulus (US). The other grating (CS-) was never paired with the US. In habituation, which preceded acquisition, and in extinction, the same two gratings were presented without US. Using fMRI multivoxel patterns in primary visual cortex during habituation as reference, we found that during acquisition, aversive learning selectively prompted systematic changes in multivoxel patterns evoked by CS+. Specifically, CS+ evoked voxel patterns in V1 became sparser as aversive learning progressed, and the sparsified pattern appeared to be preserved in extinction. Concomitant with the voxel pattern changes, occipital alpha oscillations were increasingly more desynchronized during CS+ (but not CS-) trials. Across acquisition trials, the rate of change in CS+-related alpha desynchronization was correlated with the rate of change in multivoxel pattern representations of CS+. Furthermore, alpha oscillations co-varied with blood-oxygen-level-dependent (BOLD) data in the ventral attention network, but not with BOLD in the amygdala. Thus, fear conditioning prompts persistent sparsification of voxel patterns evoked by threat, likely mediated by attention-related mechanisms

Aversive Conditioning of Spatial Position Sharpens Neural Population-Level Tuning in Visual Cortex and Selectively Alters Alpha-Band Activity

Processing capabilities for many low-level visual features are experientially malleable, aiding sighted organisms in adapting to dynamic environments. Explicit instructions to attend a specific visual field location influence retinotopic visuocortical activity, amplifying responses to stimuli appearing at cued spatial positions. It remains undetermined both how such prioritization affects surrounding nonprioritized locations, and if a given retinotopic spatial position can attain enhanced cortical representation through experience rather than instruction. The current report examined visuocortical response changes as human observers (N = 51, 19 male) learned, through differential classical conditioning, to associate specific screen locations with aversive outcomes. Using dense-array EEG and pupillometry, we tested the preregistered hypotheses of either sharpening or generalization around an aversively associated location following a single conditioning session. Competing hypotheses tested whether mean response changes would take the form of a Gaussian (generalization) or difference-of-Gaussian (sharpening) distribution over spatial positions, peaking at the viewing location paired with a noxious noise. Occipital 15 Hz steady-state visual evoked potential responses were selectively heightened when viewing aversively paired locations and displayed a nonlinear, difference-of-Gaussian profile across neighboring locations, consistent with suppressive surround modulation of nonprioritized positions. Measures of alpha-band (8-12 Hz) activity were differentially altered in anterior versus posterior locations, while pupil diameter exhibited selectively heightened responses to noise-paired locations but did not evince differences across the nonpaired locations. These results indicate that visuocortical spatial representations are sharpened in response to location-specific aversive conditioning, while top-down influences indexed by alpha-power reduction exhibit posterior generalization and anterior sharpening.SIGNIFICANCE STATEMENT It is increasingly recognized that early visual cortex is not a static processor of physical features, but is instead constantly shaped by perceptual experience. It remains unclear, however, to what extent the cortical representation of many fundamental features, including visual field location, is malleable by experience. Using EEG and an aversive classical conditioning paradigm, we observed sharpening of visuocortical responses to stimuli appearing at aversively associated locations along with location-selective facilitation of response systems indexed by pupil diameter and EEG alpha power. These findings highlight the experience-dependent flexibility of retinotopic spatial representations in visual cortex, opening avenues toward novel treatment targets in disorders of attention and spatial cognition.

NMDAR-Dependent Emergence of Behavioral Representation in Primary Visual Cortex

Although neocortical sensory areas are generally thought to faithfully represent external stimuli, cortical networks exhibit considerable functional plasticity, allowing them to modify their output to reflect ongoing behavioral demands. We apply longitudinal 2-photon imaging of activity in the primary visual cortex (V1) of mice learning a conditioned eyeblink task to investigate the dynamic representations of task-relevant information. We find that, although all V1 neurons robustly and stably encode visual input, pyramidal cells and parvalbumin-expressing interneurons exhibit experience-dependent emergence of accurate behavioral representations during learning. The functional plasticity driving performance-predictive activity requires cell-autonomous expression of NMDA-type glutamate receptors. Our findings demonstrate that accurate encoding of behavioral output is not inherent to V1 but develops during learning to support visual task performance.

Human Sensory Cortex Contributes to the Long-Term Storage of Aversive Conditioning

Growing animal data evince a critical role of the sensory cortex in the long-term storage of aversive conditioning, following acquisition and consolidation in the amygdala. Whether and how this function is conserved in the human sensory cortex is nonetheless unclear. We interrogated this question in a human aversive conditioning study using multidimensional assessments of conditioning and long-term (15 d) retention. Conditioned stimuli (CSs; Gabor patches) were calibrated to differentially activate the parvocellular (P) and magnocellular (M) visual pathways, further elucidating cortical versus subcortical mechanisms. Full-blown conditioning and long-term retention emerged for M-biased CS (vs limited effects for P-biased CS), especially among anxious individuals, in all four dimensions assessed: threat appraisal (threat ratings), physiological arousal (skin conductance response), perceptual learning [discrimination sensitivity (d') and response speed], and cortical plasticity [visual evoked potentials (VEPs) and cortical current density]. Interestingly, while behavioral, physiological, and VEP effects were comparable at immediate and delayed assessments, the cortical substrates evolved markedly over time, transferring from high-order cortices [inferotemporal/fusiform cortex and orbitofrontal cortex (OFC)] immediately to the primary and secondary visual cortex after the delay. In sum, the contrast between P- and M-biased conditioning confirms privileged conditioning acquisition via the subcortical pathway while the immediate cortical plasticity lends credence to the triadic amygdala-OFC-fusiform network thought to underlie threat processing. Importantly, long-term retention of conditioning in the basic sensory cortices supports the conserved role of the human sensory cortex in the long-term storage of aversive conditioning.SIGNIFICANCE STATEMENT A growing network of neural substrates has been identified in threat learning and memory. The sensory cortex plays a key role in long-term threat memory in animals, but such a function in humans remains unclear. To explore this problem, we conducted multidimensional assessments of immediate and delayed (15 d) effects of human aversive conditioning. Behavioral, physiological, and scalp electrophysiological data demonstrated conditioning effects and long-term retention. High-density EEG intracranial source analysis further revealed the cortical underpinnings, implicating high-order cortices immediately and primary and secondary visual cortices after the long delay. Therefore, while high-order cortices support aversive conditioning acquisition (i.e., threat learning), the human sensory cortex (akin to the animal homolog) underpins long-term storage of conditioning (i.e., long-term threat memory).

Decoding Neural Representations of Affective Scenes in Retinotopic Visual Cortex

The perception of opportunities and threats in complex visual scenes represents one of the main functions of the human visual system. The underlying neurophysiology is often studied by having observers view pictures varying in affective content. It has been shown that viewing emotionally engaging, compared with neutral, pictures (1) heightens blood flow in limbic, frontoparietal, and anterior visual structures and (2) enhances the late positive event-related potential (LPP). The role of retinotopic visual cortex in this process has, however, been contentious, with competing theories predicting the presence versus absence of emotion-specific signals in retinotopic visual areas. Recording simultaneous electroencephalography-functional magnetic resonance imaging while observers viewed pleasant, unpleasant, and neutral affective pictures, and applying multivariate pattern analysis, we found that (1) unpleasant versus neutral and pleasant versus neutral decoding accuracy were well above chance level in retinotopic visual areas, (2) decoding accuracy in ventral visual cortex (VVC), but not in early or dorsal visual cortex, was correlated with LPP, and (3) effective connectivity from amygdala to VVC predicted unpleasant versus neutral decoding accuracy, whereas effective connectivity from ventral frontal cortex to VVC predicted pleasant versus neutral decoding accuracy. These results suggest that affective scenes evoke valence-specific neural representations in retinotopic visual cortex and that these representations are influenced by reentry signals from anterior brain regions.

Image Valence Modulates the Processing of Low-Resolution Affective Natural Scenes

In natural vision, noisy and distorted visual inputs often change our perceptual strategy in scene perception. However, it is unclear the extent to which the affective meaning embedded in the degraded natural scenes modulates our scene understanding and associated eye movements. In this eye-tracking experiment by presenting natural scene images with different categories and levels of emotional valence (high-positive, medium-positive, neutral/low-positive, medium-negative, and high-negative), we systematically investigated human participants' perceptual sensitivity (image valence categorization and arousal rating) and image-viewing gaze behaviour to the changes of image resolution. Our analysis revealed that reducing image resolution led to decreased valence recognition and arousal rating, decreased number of fixations in image-viewing but increased individual fixation duration, and stronger central fixation bias. Furthermore, these distortion effects were modulated by the scene valence with less deterioration impact on the valence categorization of negatively valenced scenes and on the gaze behaviour in viewing of high emotionally charged (high-positive and high-negative) scenes. It seems that our visual system shows a valence-modulated susceptibility to the image distortions in scene perception.

Valence-dependent Disruption in Processing of Facial Expressions of Emotion in Early Visual Cortex—A Transcranial Magnetic Stimulation Study

Our visual inputs are often entangled with affective meanings in natural vision, implying the existence of extensive interaction between visual and emotional processing. However, little is known about the neural mechanism underlying such interaction. This exploratory transcranial magnetic stimulation (TMS) study examined the possible involvement of the early visual cortex (EVC, Area V1/V2/V3) in perceiving facial expressions of different emotional valences. Across three experiments, single-pulse TMS was delivered at different time windows (50–150 msec) after a brief 10-msec onset of face images, and participants reported the visibility and perceived emotional valence of faces. Interestingly, earlier TMS at ∼90 msec only reduced the face visibility irrespective of displayed expressions, but later TMS at ∼120 msec selectively disrupted the recognition of negative facial expressions, indicating the involvement of EVC in the processing of negative expressions at a later time window, possibly beyond the initial processing of fed-forward facial structure information. The observed TMS effect was further modulated by individuals' anxiety level. TMS at ∼110–120 msec disrupted the recognition of anger significantly more for those scoring relatively low in trait anxiety than the high scorers, suggesting that cognitive bias influences the processing of facial expressions in EVC. Taken together, it seems that EVC is involved in structural encoding of (at least) negative facial emotional valence, such as fear and anger, possibly under modulation from higher cortical areas.

Effects of Experience on Spatial Frequency Tuning in the Visual System: Behavioral, Visuocortical, and Alpha-band Responses

Using electrophysiology and a classic fear conditioning paradigm, this work examined adaptive visuocortical changes in spatial frequency tuning in a sample of 50 undergraduate students. High-density EEG was recorded while participants viewed 400 total trials of individually presented Gabor patches of 10 different spatial frequencies. Patches were flickered to produce sweep steady-state visual evoked potentials (ssVEPs) at a temporal frequency of 13.33 Hz, with stimulus contrast ramping up from 0% to 41% Michelson over the course of each 2800-msec trial. During the final 200 trials, a selected range of Gabor stimuli (either the lowest or highest spatial frequencies, manipulated between participants) were paired with an aversive 90-dB white noise auditory stimulus. Changes in spatial frequency tuning from before to after conditioning for paired and unpaired gratings were evaluated at the behavioral and electrophysiological level. Specifically, ssVEP amplitude changes were evaluated for lateral inhibition and generalization trends, whereas change in alpha band (8-12 Hz) activity was tested for a generalization trend across spatial frequencies, using permutation-controlled F contrasts. Overall time courses of the sweep ssVEP amplitude envelope and alpha-band power were orthogonal, and ssVEPs proved insensitive to spatial frequency conditioning. Alpha reduction (blocking) was most pronounced when viewing fear-conditioned spatial frequencies, with blocking decreasing along the gradient of spatial frequencies preceding conditioned frequencies, indicating generalization across spatial frequencies. Results suggest that alpha power reduction-conceptually linked to engagement of attention and alertness/arousal mechanisms-to fear-conditioned stimuli operates independently of low-level spatial frequency processing (indexed by ssVEPs) in primary visual cortex.

Fear: It's All in Your Line of Sight

The brain circuits that create our sense of fear rely on ancient 'hard-wired' components of the limbic system, but also use sensory processing to determine what we become afraid of. A new study shows that, when viewing of simple oriented line stimuli is coupled with aversive experiences, neurons in primary visual cortex rapidly alter their responses in a manner that indicates the line stimuli become a source of fear.