Differential temporal dynamics during visual imagery and perception

Dynamics of mental imagery

Phenomenology of mental imagery can reveal the structure of underlying mental representations, yet progress has been limited because of its private nature. Through a phenomenology-recreation task we elucidate the dynamics of mental imagery. Specifically, the temporal grain, speed of object manipulation, smoothness of contents unfolding, and temporal extent of stability of imagined contents. To gauge these properties, we asked a large cohort of participants (N = 827) to recreate these aspects of their imagination in six tasks. Results showed that temporal features of imagination unfold at distinct timescales, though a factor analysis showed that variance in these tasks could be accounted for via two factors; temporal ability and stability of mental imagery. Additionally, we contrast these regularities with those documented for visual perception, showing that imagined contents are sluggish but more stable than perception. However, both imagination and perception share a common constraint; maintaining identically sized temporal windows of conscious experience.

Evolving Engrams Demand Changes in Effective Cues

A longstanding principle in episodic memory research, known as the encoding specificity hypothesis, holds that an effective retrieval cue should closely match the original encoding conditions. This principle assumes that a successful retrieval cue remains static over time. Despite the broad acceptance of this idea, it conflicts with one of the most well-established findings in memory research: The dynamic and ever-changing nature of episodic memories. In this article, we propose that the most effective retrieval cue should engage with the current state of the memory, which may have shifted significantly since encoding. By redefining the criteria for successful recall, we challenge a core principle of the field and open new avenues for exploring memory accessibility, offering fresh insights into both theoretical, and applied domains.

Covariance-based decoding reveals a category-specific functional connectivity network for imagined visual objects

The coordination of different brain regions is required for the visual imagery of complex objects (e.g., faces and places). Short-range connectivity within sensory areas is necessary to construct the mental image. Long-range connectivity between control and sensory areas is necessary to re-instantiate and maintain the mental image. While dynamic changes in functional connectivity are expected during visual imagery, it is unclear whether a category-specific network exists in which the strength and the spatial destination of the connections vary depending on the imagery target. In this magnetoencephalography study, we used a minimally constrained experimental paradigm wherein imagery categories were prompted using visual word cues only, and we decoded face versus place imagery based on their underlying functional connectivity patterns as estimated from the spatial covariance across brain regions. A subnetwork analysis further disentangled the contribution of different connections. The results show that face and place imagery can be decoded from both short-range and long-range connections. Overall, the results show that imagined object categories can be distinguished based on functional connectivity patterns observed in a category-specific network. Notably, functional connectivity estimates rely on purely endogenous brain signals suggesting that an external reference is not necessary to elicit such category-specific network dynamics.

Neurocognitive mechanisms of social scenario imagery generation in individuals with social anxiety

Cognitive behavioral theory emphasizes the significant role of mental imagery in the onset and development of Social Anxiety Disorder (SAD). However, the neural mechanisms underlying the generation of social scenario imagery in individuals with social anxiety remain unclear. In this study, 28 individuals with social anxiety and 31 healthy controls performed a retrospective cue imagery generation task to examine their neural responses. Behavioral results showed that, compared to negative social scenarios, the vividness of positive social scenario imagery was significantly lower in the social anxiety group, while the control group showed no significant difference between the two conditions. Event-related potential (ERP) results revealed that, for the social anxiety group, N170 and LPP amplitudes were significantly larger under the neutral condition compared to the negative condition, whereas the control group exhibited no significant difference between these conditions. Furthermore, the social anxiety group showed significantly larger LPP amplitudes than the control group in both the positive and neutral conditions. These findings provide the first neurophysiological evidence that individuals with social anxiety exhibit processing biases when generating imagery of positive and neutral social scenarios, suggesting heightened neural engagement in these conditions.

Testing the role of spontaneous activity in visuospatial perception with patterned optogenetics

A major debate in the field of consciousness pertains to whether neuronal activity or rather the causal structure of neural circuits underlie the generation of conscious experience. The former position is held by theoretical accounts of consciousness based on the predictive processing framework (such as neurorepresentationalism and active inference), while the latter is posited by the integrated information theory. This protocol describes an experiment, part of a larger adversarial collaboration, that was designed to address this question through a combination of behavioral tests in mice, functional imaging, patterned optogenetics and electrophysiology. The experiment will directly test if optogenetic inactivation of a portion of the visual cortex not responding to behaviorally relevant stimuli will affect the perception of the spatial distribution of these stimuli, even when the neurons being inactivated display no or very low spiking activity, so low that it does not induce a significant effect on other cortical areas. The results of the experiment will be compared against theoretical predictions, and will provide a major contribution towards understanding what the neuronal substrate of consciousness is.

The design and implementation of multi-character classification scheme based on EEG signals of visual imagery

In visual-imagery-based brain-computer interface (VI-BCI), there are problems of singleness of imagination task and insufficient description of feature information, which seriously hinder the development and application of VI-BCI technology in the field of restoring communication. In this paper, we design and optimize a multi-character classification scheme based on electroencephalogram (EEG) signals of visual imagery (VI), which is used to classify 29 characters including 26 lowercase English letters and three punctuation marks. Firstly, a new paradigm of randomly presenting characters and including preparation stage is designed to acquire EEG signals and construct a multi-character dataset, which can eliminate the influence between VI tasks. Secondly, tensor data is obtained by the Morlet wavelet transform, and a feature extraction algorithm based on tensor-uncorrelated multilinear principal component analysis is used to extract high-quality features. Finally, three classifiers, namely support vector machine, K-nearest neighbor, and extreme learning machine, are employed for classifying multi-character, and the results are compared. The experimental results demonstrate that, the proposed scheme effectively extracts character features with minimal redundancy, weak correlation, and strong representation capability, and successfully achieves an average classification accuracy 97.59% for 29 characters, surpassing existing research in terms of both accuracy and quantity of classification. The present study designs a new paradigm for acquiring EEG signals of VI, and combines the Morlet wavelet transform and UMPCA algorithm to extract the character features, enabling multi-character classification in various classifiers. This research paves a novel pathway for establishing direct brain-to-world communication.

Phantasia, aphantasia, and hyperphantasia: Empirical data and conceptual considerations

Within the past decade, the term "phantasia" has been increasingly used to describe the human capacity, faculty, or power of visual mental imagery, with extremes of imagery vividness characterised as "aphantasia" and "hyperphantasia". A substantial volume of empirical research addressing these constructs has now been published, including attempts to find inductive correlates of behaviourally defined aphantasia, for example using research questionnaires and functional magnetic resonance imaging. Mental imagery has long been noted as a source of conceptual confusions but no specific conceptual analysis of the new formulation of phantasia, aphantasia, and hyperphantasia has been undertaken hitherto. We offer some conceptual considerations on phantasia, noting the ongoing confusion of perceptual with mental images, and the ubiquitous use of unvalidated subjective assessment instruments such as the Vividness of Visual Imagery Questionnaire (VVIQ) in diagnosis and assessment, development of which was predicated on these conceptual confusions. We offer some suggestions for a conceptual framework for future empirical studies in this field, circumventing these conceptual confusions.

Representations of imaginary scenes and their properties in cortical alpha activity

Imagining natural scenes enables us to engage with a myriad of simulated environments. How do our brains generate such complex mental images? Recent research suggests that cortical alpha activity carries information about individual objects during visual imagery. However, it remains unclear if more complex imagined contents such as natural scenes are similarly represented in alpha activity. Here, we answer this question by decoding the contents of imagined scenes from rhythmic cortical activity patterns. In an EEG experiment, participants imagined natural scenes based on detailed written descriptions, which conveyed four complementary scene properties: openness, naturalness, clutter level and brightness. By conducting classification analyses on EEG power patterns across neural frequencies, we were able to decode both individual imagined scenes as well as their properties from the alpha band, showing that also the contents of complex visual images are represented in alpha rhythms. A cross-classification analysis between alpha power patterns during the imagery task and during a perception task, in which participants were presented images of the described scenes, showed that scene representations in the alpha band are partly shared between imagery and late stages of perception. This suggests that alpha activity mediates the top-down re-activation of scene-related visual contents during imagery.

Visual mental imagery: Evidence for a heterarchical neural architecture

Theories of Visual Mental Imagery (VMI) emphasize the processes of retrieval, modification, and recombination of sensory information from long-term memory. Yet, only few studies have focused on the behavioral mechanisms and neural correlates supporting VMI of stimuli from different semantic domains. Therefore, we currently have a limited understanding of how the brain generates and maintains mental representations of colors, faces, shapes - to name a few. Such an undetermined scenario renders unclear the organizational structure of neural circuits supporting VMI, including the role of the early visual cortex. We aimed to fill this gap by reviewing the scientific literature of five semantic domains: visuospatial, face, colors, shapes, and letters imagery. Linking theory to evidence from over 60 different experimental designs, this review highlights three main points. First, there is no consistent activity in the early visual cortex across all VMI domains, contrary to the prediction of the dominant model. Second, there is consistent activity of the frontoparietal networks and the left hemisphere's fusiform gyrus during voluntary VMI irrespective of the semantic domain investigated. We propose that these structures are part of a domain-general VMI sub-network. Third, domain-specific information engages specific regions of the ventral and dorsal cortical visual pathways. These regions partly overlap with those found in visual perception studies (e.g., fusiform face area for faces imagery; lingual gyrus for color imagery). Altogether, the reviewed evidence suggests the existence of domain-general and domain-specific mechanisms of VMI selectively engaged by stimulus-specific properties (e.g., colors or faces). These mechanisms would be supported by an organizational structure mixing vertical and horizontal connections (heterarchy) between sub-networks for specific stimulus domains. Such a heterarchical organization of VMI makes different predictions from current models of VMI as reversed perception. Our conclusions set the stage for future research, which should aim to characterize the spatiotemporal dynamics and interactions among key regions of this architecture giving rise to visual mental images.

Distinct neural signatures underlying information maintenance and manipulation in working memory

Previous working memory research has demonstrated robust stimulus representations during memory maintenance in both voltage and alpha-band activity in electroencephalography. However, the exact functions of these 2 neural signatures have remained controversial. Here we systematically investigated their respective contributions to memory manipulation. Human participants either maintained a previously seen spatial location, or manipulated the location following a mental rotation cue over a delay. Using multivariate decoding, we observed robust location representations in low-frequency voltage and alpha-band oscillatory activity with distinct spatiotemporal dynamics: location representations were most evident in posterior channels in alpha-band activity, but were most prominent in the more anterior, central channels in voltage signals. Moreover, the temporal emergence of manipulated representation in central voltage preceded that in posterior alpha-band activity, suggesting that voltage might carry stimulus-specific source signals originated internally from anterior cortex, whereas alpha-band activity might reflect feedback signals in posterior cortex received from higher-order cortex. Lastly, while location representations in both signals were coded in a low-dimensional neural subspace, location representation in central voltage was higher-dimensional and underwent a representational transformation that exclusively predicted memory behavior. Together, these results highlight the crucial role of central voltage in working memory, and support functional distinctions between voltage and alpha-band activity.

Working memory signals in early visual cortex are present in weak and strong imagers

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

Identifying content-invariant neural signatures of perceptual vividness

Some conscious experiences are more vivid than others. Although perceptual vividness is a key component of human consciousness, how variation in this magnitude property is registered by the human brain is unknown. A striking feature of neural codes for magnitude in other psychological domains, such as number or reward, is that the magnitude property is represented independently of its sensory features. To test whether perceptual vividness also covaries with neural codes that are invariant to sensory content, we reanalyzed existing magnetoencephalography and functional MRI data from two distinct studies which quantified perceptual vividness via subjective ratings of awareness and visibility. Using representational similarity and decoding analyses, we find evidence for content-invariant neural signatures of perceptual vividness distributed across visual, parietal, and frontal cortices. Our findings indicate that the neural correlates of subjective vividness may share similar properties to magnitude codes in other cognitive domains.

Spatiotemporal dynamics of self-generated imagery reveal a reverse cortical hierarchy from cue-induced imagery

Visual imagery allows for the construction of rich internal experience in our mental world. However, it has remained poorly understood how imagery experience derives volitionally as opposed to being cue driven. Here, using electroencephalography and functional magnetic resonance imaging, we systematically investigate the spatiotemporal dynamics of self-generated imagery by having participants volitionally imagining one of the orientations from a learned pool. We contrast self-generated imagery with cue-induced imagery, where participants imagined line orientations based on associative cues acquired previously. Our results reveal overlapping neural signatures of cue-induced and self-generated imagery. Yet, these neural signatures display substantially differential sensitivities to the two types of imagery: self-generated imagery is supported by an enhanced involvement of the anterior cortex in representing imagery contents. By contrast, cue-induced imagery is supported by enhanced imagery representations in the posterior visual cortex. These results jointly support a reverse cortical hierarchy in generating and maintaining imagery contents in self-generated versus externally cued imagery.

Neural retrieval processes occur more rapidly for visual mental images that were previously encoded with high-vividness

Visual mental imagery refers to our ability to experience visual images in the absence of sensory stimulation. Studies have shown that visual mental imagery can improve episodic memory. However, we have limited understanding of the neural mechanisms underlying this improvement. Using electroencephalography, we examined the neural processes associated with the retrieval of previously generated visual mental images, focusing on how the vividness at generation can modulate retrieval processes. Participants viewed word stimuli referring to common objects, forming a visual mental image of each word and rating the vividness of the mental image. This was followed by a surprise old/new recognition task. We compared retrieval performance for items rated as high- versus low-vividness at encoding. High-vividness items were retrieved with faster reaction times and higher confidence ratings in the memory judgment. While controlling for confidence, neural measures indicated that high-vividness items produced an earlier decrease in alpha-band activity at retrieval compared with low-vividness items, suggesting an earlier memory reinstatement. Even when low-vividness items were remembered with high confidence, they were not retrieved as quickly as high-vividness items. These results indicate that when highly vivid mental images are encoded, the speed of their retrieval occurs more rapidly, relative to low-vivid items.

Neural Representations in Visual and Parietal Cortex Differentiate between Imagined, Perceived, and Illusory Experiences

Humans constantly receive massive amounts of information, both perceived from the external environment and imagined from the internal world. To function properly, the brain needs to correctly identify the origin of information being processed. Recent work has suggested common neural substrates for perception and imagery. However, it has remained unclear how the brain differentiates between external and internal experiences with shared neural codes. Here we tested this question in human participants (male and female) by systematically investigating the neural processes underlying the generation and maintenance of visual information from voluntary imagery, veridical perception, and illusion. The inclusion of illusion allowed us to differentiate between objective and subjective internality: while illusion has an objectively internal origin and can be viewed as involuntary imagery, it is also subjectively perceived as having an external origin like perception. Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, we observed superior orientation representations in parietal cortex during imagery compared with perception, and conversely in early visual cortex. This imagery dominance gradually developed along a posterior-to-anterior cortical hierarchy from early visual to parietal cortex, emerged in the early epoch of imagery and sustained into the delay epoch, and persisted across varied imagined contents. Moreover, representational strength of illusion was more comparable to imagery in early visual cortex, but more comparable to perception in parietal cortex, suggesting content-specific representations in parietal cortex differentiate between subjectively internal and external experiences, as opposed to early visual cortex. These findings together support a domain-general engagement of parietal cortex in internally generated experience.SIGNIFICANCE STATEMENT How does the brain differentiate between imagined and perceived experiences? Combining fMRI, eye-tracking, multivariate decoding, and encoding approaches, the current study revealed enhanced stimulus-specific representations in visual imagery originating from parietal cortex, supporting the subjective experience of imagery. This neural principle was further validated by evidence from visual illusion, wherein illusion resembled perception and imagery at different levels of cortical hierarchy. Our findings provide direct evidence for the critical role of parietal cortex as a domain-general region for content-specific imagery, and offer new insights into the neural mechanisms underlying the differentiation between subjectively internal and external experiences.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

A comparison of reward processing during Becker-DeGroot-Marschak and Vickrey auctions: An ERP study

Vickrey auctions (VA) and Becker-DeGroot-Marschak auctions (BDM) are strategically equivalent demand-revealing mechanisms, differentiated only by a human opponent in the VA, and a random-number-generator opponent in the BDM. Game parameters are such that players are incentivized to reveal their private subjective values (SV) and behavior should be identical in both tasks. However, this has been repeatedly shown not to be the case. In this study, the neural correlates of outcome feedback processing during VA and BDM were directly compared using electroencephalography. Twenty-eight healthy participants bid for household products which were then divided into high- and low-SV categories. The VA included a human opponent deception to induce a social environment, while in reality a random-number-generator was used in both tasks. A P3 component peaking at 336 ms over midline parietal sites showed more positive amplitudes for high bid values, and for win outcomes in the VA but not the BDM. Both auctions also elicited a Reward Positivity potential, maximal at 275 ms along the central midline electrodes, that was not modulated by auction task or SV. Further, an exploratory N170 potential in the right occipitotemporal electrodes and a vertex positive potential component were stronger in the VA relative to the BDM. Results point to an enhanced cortical response to bid outcomes during VA task in a potential component associated with emotional control, and to the occurrence of face-sensitive potentials in VA but not in BDM auction. These findings suggest modulation of bid outcome processing by the social-competitive aspect of auction tasks. Directly comparing two prominent auction paradigms affords the opportunity to isolate the impact of social environment on competitive, risky decision-making. Findings suggest that feedback processing as early as 176 ms is facilitated by the presence of a human competitor, and later processing is modulated by social context and subjective value.

Modelling perception as a hierarchical competition differentiates imagined, veridical, and hallucinated percepts

Mental imagery is a process by which thoughts become experienced with sensory characteristics. Yet, it is not clear why mental images appear diminished compared to veridical images, nor how mental images are phenomenologically distinct from hallucinations, another type of non-veridical sensory experience. Current evidence suggests that imagination and veridical perception share neural resources. If so, we argue that considering how neural representations of externally generated stimuli (i.e. sensory input) and internally generated stimuli (i.e. thoughts) might interfere with one another can sufficiently differentiate between veridical, imaginary, and hallucinatory perception. We here use a simple computational model of a serially connected, hierarchical network with bidirectional information flow to emulate the primate visual system. We show that modelling even first approximations of neural competition can more coherently explain imagery phenomenology than non-competitive models. Our simulations predict that, without competing sensory input, imagined stimuli should ubiquitously dominate hierarchical representations. However, with competition, imagination should dominate high-level representations but largely fail to outcompete sensory inputs at lower processing levels. To interpret our findings, we assume that low-level stimulus information (e.g. in early visual cortices) contributes most to the sensory aspects of perceptual experience, while high-level stimulus information (e.g. towards temporal regions) contributes most to its abstract aspects. Our findings therefore suggest that ongoing bottom-up inputs during waking life may prevent imagination from overriding veridical sensory experience. In contrast, internally generated stimuli may be hallucinated when sensory input is dampened or eradicated. Our approach can explain individual differences in imagery, along with aspects of daydreaming, hallucinations, and non-visual mental imagery.

Spatiotemporal phase slip patterns for visual evoked potentials, covert object naming tasks, and insight moments extracted from 256 channel EEG recordings

Phase slips arise from state transitions of the coordinated activity of cortical neurons which can be extracted from the EEG data. The phase slip rates (PSRs) were studied from the high-density (256 channel) EEG data, sampled at 16.384 kHz, of five adult subjects during covert visual object naming tasks. Artifact-free data from 29 trials were averaged for each subject. The analysis was performed to look for phase slips in the theta (4-7 Hz), alpha (7-12 Hz), beta (12-30 Hz), and low gamma (30-49 Hz) bands. The phase was calculated with the Hilbert transform, then unwrapped and detrended to look for phase slip rates in a 1.0 ms wide stepping window with a step size of 0.06 ms. The spatiotemporal plots of the PSRs were made by using a montage layout of 256 equidistant electrode positions. The spatiotemporal profiles of EEG and PSRs during the stimulus and the first second of the post-stimulus period were examined in detail to study the visual evoked potentials and different stages of visual object recognition in the visual, language, and memory areas. It was found that the activity areas of PSRs were different as compared with EEG activity areas during the stimulus and post-stimulus periods. Different stages of the insight moments during the covert object naming tasks were examined from PSRs and it was found to be about 512 ± 21 ms for the 'Eureka' moment. Overall, these results indicate that information about the cortical phase transitions can be derived from the measured EEG data and can be used in a complementary fashion to study the cognitive behavior of the brain.

What do you have in mind? ERP markers of visual and auditory imagery

This study aimed to investigate the psychophysiological markers of imagery processes through EEG/ERP recordings. Visual and auditory stimuli representing 10 different semantic categories were shown to 30 healthy participants. After a given interval and prompted by a light signal, participants were asked to activate a mental image corresponding to the semantic category for recording synchronized electrical potentials. Unprecedented electrophysiological markers of imagination were recorded in the absence of sensory stimulation. The following peaks were identified at specific scalp sites and latencies, during imagination of infants (centroparietal positivity, CPP, and late CPP), human faces (anterior negativity, AN), animals (anterior positivity, AP), music (P300-like), speech (N400-like), affective vocalizations (P2-like) and sensory (visual vs auditory) modality (PN300). Overall, perception and imagery conditions shared some common electro/cortical markers, but during imagery the category-dependent modulation of ERPs was long latency and more anterior, with respect to the perceptual condition. These ERP markers might be precious tools for BCI systems (pattern recognition, classification, or A.I. algorithms) applied to patients affected by consciousness disorders (e.g., in a vegetative or comatose state) or locked-in-patients (e.g., spinal or SLA patients).

Functional connectivity fingerprints of the frontal eye field and inferior frontal junction suggest spatial versus nonspatial processing in the prefrontal cortex

Neuroimaging evidence suggests that the frontal eye field (FEF) and inferior frontal junction (IFJ) govern the encoding of spatial and nonspatial (such as feature- or object-based) representations, respectively, both during visual attention and working memory tasks. However, it is still unclear whether such contrasting functional segregation is also reflected in their underlying functional connectivity patterns. Here, we hypothesized that FEF has predominant functional coupling with spatiotopically organized regions in the dorsal ('where') visual stream whereas IFJ has predominant functional connectivity with the ventral ('what') visual stream. We applied seed-based functional connectivity analyses to temporally high-resolving resting-state magnetoencephalography (MEG) recordings. We parcellated the brain according to the multimodal Glasser atlas and tested, for various frequency bands, whether the spontaneous activity of each parcel in the ventral and dorsal visual pathway has predominant functional connectivity with FEF or IFJ. The results show that FEF has a robust power correlation with the dorsal visual pathway in beta and gamma bands. In contrast, anterior IFJ (IFJa) has a strong power coupling with the ventral visual stream in delta, beta and gamma oscillations. Moreover, while FEF is phase-coupled with the superior parietal lobe in the beta band, IFJa is phase-coupled with the middle and inferior temporal cortex in delta and gamma oscillations. We argue that these intrinsic connectivity fingerprints are congruent with each brain region's function. Therefore, we conclude that FEF and IFJ have dissociable connectivity patterns that fit their respective functional roles in spatial versus nonspatial top-down attention and working memory control.

Sustained neural representations of personally familiar people and places during cued recall

The recall and visualization of people and places from memory is an everyday occurrence, yet the neural mechanisms underpinning this phenomenon are not well understood. In particular, the temporal characteristics of the internal representations generated by active recall are unclear. Here, we used magnetoencephalography (MEG) and multivariate pattern analysis to measure the evolving neural representation of familiar places and people across the whole brain when human participants engage in active recall. To isolate self-generated imagined representations, we used a retro-cue paradigm in which participants were first presented with two possible labels before being cued to recall either the first or second item. We collected personalized labels for specific locations and people familiar to each participant. Importantly, no visual stimuli were presented during the recall period, and the retro-cue paradigm allowed the dissociation of responses associated with the labels from those corresponding to the self-generated representations. First, we found that following the retro-cue it took on average ∼1000 ms for distinct neural representations of freely recalled people or places to develop. Second, we found distinct representations of personally familiar concepts throughout the 4 s recall period. Finally, we found that these representations were highly stable and generalizable across time. These results suggest that self-generated visualizations and recall of familiar places and people are subserved by a stable neural mechanism that operates relatively slowly when under conscious control.

Attentional capture mediates the emergence and suppression of intrusive memories

Intrusive memories hijack consciousness and their control may lead to forgetting. However, the contribution of reflexive attention to qualifying a memory signal as interfering is unknown. We used machine learning to decode the brain's electrical activity and pinpoint the otherwise hidden emergence of intrusive memories reported during a memory suppression task. Importantly, the algorithm was trained on an independent attentional model of visual activity, mimicking either the abrupt and interfering appearance of visual scenes into conscious awareness or their deliberate exploration. Intrusion of memories into conscious awareness were decoded above chance. The decoding accuracy increased when the algorithm was trained using a model of reflexive attention. Conscious detection of intrusive activity decoded from the brain signal was central to the future silencing of suppressed memories and later forgetting. Unwanted memories require the reflexive orienting of attention and access to consciousness to be suppressed effectively by inhibitory control.

A large and rich EEG dataset for modeling human visual object recognition

The human brain achieves visual object recognition through multiple stages of linear and nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models' prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.

Decoding the temporal dynamics of affective scene processing

Natural images containing affective scenes are used extensively to investigate the neural mechanisms of visual emotion processing. Functional fMRI studies have shown that these images activate a large-scale distributed brain network that encompasses areas in visual, temporal, and frontal cortices. The underlying spatial and temporal dynamics, however, remain to be better characterized. We recorded simultaneous EEG-fMRI data while participants passively viewed affective images from the International Affective Picture System (IAPS). Applying multivariate pattern analysis to decode EEG data, and representational similarity analysis to fuse EEG data with simultaneously recorded fMRI data, we found that: (1) ∼80 ms after picture onset, perceptual processing of complex visual scenes began in early visual cortex, proceeding to ventral visual cortex at ∼100 ms, (2) between ∼200 and ∼300 ms (pleasant pictures: ∼200 ms; unpleasant pictures: ∼260 ms), affect-specific neural representations began to form, supported mainly by areas in occipital and temporal cortices, and (3) affect-specific neural representations were stable, lasting up to ∼2 s, and exhibited temporally generalizable activity patterns. These results suggest that affective scene representations in the brain are formed temporally in a valence-dependent manner and may be sustained by recurrent neural interactions among distributed brain areas.

Memories with a blind mind: Remembering the past and imagining the future with aphantasia

Our capacity to re-experience the past and simulate the future is thought to depend heavily on visual imagery, which allows us to construct complex sensory representations in the absence of sensory stimulation. There are large individual differences in visual imagery ability, but their impact on autobiographical memory and future prospection remains poorly understood. Research in this field assumes the normative use of visual imagery as a cognitive tool to simulate the past and future, however some individuals lack the ability to visualise altogether (a condition termed "aphantasia"). Aphantasia represents a rare and naturally occurring knock-out model for examining the role of visual imagery in episodic memory recall. Here, we assessed individuals with aphantasia on an adapted form of the Autobiographical Interview, a behavioural measure of the specificity and richness of episodic details underpinning the memory of events. Aphantasic participants generated significantly fewer episodic details than controls for both past and future events. This effect was most pronounced for novel future events, driven by selective reductions in visual detail retrieval, accompanied by comparatively reduced ratings of the phenomenological richness of simulated events, and paralleled by quantitative linguistic markers of reduced perceptual language use in aphantasic participants compared to those with visual imagery. Our findings represent the first systematic evidence (using combined objective and subjective data streams) that aphantasia is associated with a diminished ability to re-experience the past and simulate the future, indicating that visual imagery is an important cognitive tool for the dynamic retrieval and recombination of episodic details during mental simulation.

Exploring the Representations of Individual Entities in the Brain Combining EEG and Distributional Semantics

Semantic knowledge about individual entities (i.e., the referents of proper names such as Jacinta Ardern) is fine-grained, episodic, and strongly social in nature, when compared with knowledge about generic entities (the referents of common nouns such as politician). We investigate the semantic representations of individual entities in the brain; and for the first time we approach this question using both neural data, in the form of newly-acquired EEG data, and distributional models of word meaning, employing them to isolate semantic information regarding individual entities in the brain. We ran two sets of analyses. The first set of analyses is only concerned with the evoked responses to individual entities and their categories. We find that it is possible to classify them according to both their coarse and their fine-grained category at appropriate timepoints, but that it is hard to map representational information learned from individuals to their categories. In the second set of analyses, we learn to decode from evoked responses to distributional word vectors. These results indicate that such a mapping can be learnt successfully: this counts not only as a demonstration that representations of individuals can be discriminated in EEG responses, but also as a first brain-based validation of distributional semantic models as representations of individual entities. Finally, in-depth analyses of the decoder performance provide additional evidence that the referents of proper names and categories have little in common when it comes to their representation in the brain.

Perceptual reality monitoring: Neural mechanisms dissociating imagination from reality

There is increasing evidence that imagination relies on similar neural mechanisms as externally triggered perception. This overlap presents a challenge for perceptual reality monitoring: deciding what is real and what is imagined. Here, we explore how perceptual reality monitoring might be implemented in the brain. We first describe sensory and cognitive factors that could dissociate imagery and perception and conclude that no single factor unambiguously signals whether an experience is internally or externally generated. We suggest that reality monitoring is implemented by higher-level cortical circuits that evaluate first-order sensory and cognitive factors to determine the source of sensory signals. According to this interpretation, perceptual reality monitoring shares core computations with metacognition. This multi-level architecture might explain several types of source confusion as well as dissociations between simply knowing whether something is real and actually experiencing it as real. We discuss avenues for future research to further our understanding of perceptual reality monitoring, an endeavour that has important implications for our understanding of clinical symptoms as well as general cognitive function.

Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition

To a crucial extent, the efficiency of reading results from the fact that visual word recognition is faster in predictive contexts. Predictive coding models suggest that this facilitation results from pre-activation of predictable stimulus features across multiple representational levels before stimulus onset. Still, it is not sufficiently understood which aspects of the rich set of linguistic representations that are activated during reading-visual, orthographic, phonological, and/or lexical-semantic-contribute to context-dependent facilitation. To investigate in detail which linguistic representations are pre-activated in a predictive context and how they affect subsequent stimulus processing, we combined a well-controlled repetition priming paradigm, including words and pseudowords (i.e., pronounceable nonwords), with behavioral and magnetoencephalography measurements. For statistical analysis, we used linear mixed modeling, which we found had a higher statistical power compared to conventional multivariate pattern decoding analysis. Behavioral data from 49 participants indicate that word predictability (i.e., context present vs. absent) facilitated orthographic and lexical-semantic, but not visual or phonological processes. Magnetoencephalography data from 38 participants show sustained activation of orthographic and lexical-semantic representations in the interval before processing the predicted stimulus, suggesting selective pre-activation at multiple levels of linguistic representation as proposed by predictive coding. However, we found more robust lexical-semantic representations when processing predictable in contrast to unpredictable letter strings, and pre-activation effects mainly resembled brain responses elicited when processing the expected letter string. This finding suggests that pre-activation did not result in "explaining away" predictable stimulus features, but rather in a "sharpening" of brain responses involved in word processing.

Comparing stimulus-evoked and spontaneous response of the face-selective multi-units in the human posterior fusiform gyrus

The stimulus-evoked neural response is a widely explored phenomenon. Conscious awareness is associated in many cases with the corresponding selective stimulus-evoked response. For example, conscious awareness of a face stimulus is associated with or accompanied by stimulus-evoked activity in the fusiform face area (FFA). In addition to the stimulus-evoked response, spontaneous (i.e. task-unrelated) activity in the brain is also abundant. Notably, spontaneous activity is considered unconscious. For example, spontaneous activity in the FFA is not associated with conscious awareness of a face. The question is: what is the difference at the neural level between stimulus-evoked activity in a case that this activity is associated with conscious awareness of some content (e.g. activity in the FFA in response to fully visible face stimuli) and spontaneous activity in that same region of the brain? To answer this question, in the present study, we had a rare opportunity to record two face-selective multi-units in the vicinity of the FFA in a human patient. We compared multi-unit face-selective task-evoked activity with spontaneous prestimulus and a resting-state activity. We found that when activity was examined over relatively long temporal windows (e.g. 100–200 ms), face-selective stimulus-evoked firing in the recorded multi-units was much higher than the spontaneous activity. In contrast, when activity was examined over relatively short windows, we found many cases of high firing rates within the spontaneous activity that were comparable to stimulus-evoked activity. Our results thus indicate that the sustained activity is what might differentiate between stimulus-evoked activity that is associated with conscious awareness and spontaneous activity.

Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations

How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.

No Evidence for Neural Overlap between Unconsciously Processed and Imagined Stimuli

Visual representations can be generated via feedforward or feedback processes. The extent to which these processes result in overlapping representations remains unclear. Previous work has shown that imagined stimuli elicit similar representations as perceived stimuli throughout the visual cortex. However, while representations during imagery are indeed only caused by feedback processing, neural processing during perception is an interplay of both feedforward and feedback processing. This means that any representational overlap could be because of overlap in feedback processes. In the current study, we aimed to investigate this issue by characterizing the overlap between feedforward- and feedback-initiated category representations during imagined stimuli, conscious perception, and unconscious processing using fMRI in humans of either sex. While all three conditions elicited stimulus representations in left lateral occipital cortex (LOC), significant similarities were observed only between imagery and conscious perception in this area. Furthermore, connectivity analyses revealed stronger connectivity between frontal areas and left LOC during conscious perception and in imagery compared with unconscious processing. Together, these findings can be explained by the idea that long-range feedback modifies visual representations, thereby reducing representational overlap between purely feedforward- and feedback-initiated stimulus representations measured by fMRI. Neural representations influenced by feedback, either stimulus driven (perception) or purely internally driven (imagery), are, however, relatively similar.

Causal neural mechanisms of context-based object recognition

Objects can be recognized based on their intrinsic features, including shape, color, and texture. In daily life, however, such features are often not clearly visible, for example when objects appear in the periphery, in clutter, or at a distance. Interestingly, object recognition can still be highly accurate under these conditions when objects are seen within their typical scene context. What are the neural mechanisms of context-based object recognition? According to parallel processing accounts, context-based object recognition is supported by the parallel processing of object and scene information in separate pathways. Output of these pathways is then combined in downstream regions, leading to contextual benefits in object recognition. Alternatively, according to feedback accounts, context-based object recognition is supported by (direct or indirect) feedback from scene-selective to object-selective regions. Here, in three pre-registered transcranial magnetic stimulation (TMS) experiments, we tested a key prediction of the feedback hypothesis: that scene-selective cortex causally and selectively supports context-based object recognition before object-selective cortex does. Early visual cortex (EVC), object-selective lateral occipital cortex (LOC), and scene-selective occipital place area (OPA) were stimulated at three time points relative to stimulus onset while participants categorized degraded objects in scenes and intact objects in isolation, in different trials. Results confirmed our predictions: relative to isolated object recognition, context-based object recognition was selectively and causally supported by OPA at 160–200 ms after onset, followed by LOC at 260–300 ms after onset. These results indicate that context-based expectations facilitate object recognition by disambiguating object representations in the visual cortex.

Beyond imagination: Hypnotic visual hallucination induces greater lateralised brain activity than visual mental imagery

Hypnotic suggestions can produce a broad range of perceptual experiences, including hallucinations. Visual hypnotic hallucinations differ in many ways from regular mental images. For example, they are usually experienced as automatic, vivid, and real images, typically compromising the sense of reality. While both hypnotic hallucination and mental imagery are believed to mainly rely on the activation of the visual cortex via top-down mechanisms, it is unknown how they differ in the neural processes they engage. Here we used an adaptation paradigm to test and compare top-down processing between hypnotic hallucination, mental imagery, and visual perception in very highly hypnotisable individuals whose ability to hallucinate was assessed. By measuring the N170/VPP event-related complex and using multivariate decoding analysis, we found that hypnotic hallucination of faces involves greater top-down activation of sensory processing through lateralised neural mechanisms in the right hemisphere compared to mental imagery. Our findings suggest that the neural signatures that distinguish hypnotically hallucinated faces from imagined faces lie in the right brain hemisphere.

The Neural Codes Underlying Internally Generated Representations in Visual Working Memory

Humans can construct rich subjective experience even when no information is available in the external world. Here, we investigated the neural representation of purely internally generated stimulus-like information during visual working memory. Participants performed delayed recall of oriented gratings embedded in noise with varying contrast during fMRI scanning. Their trialwise behavioral responses provided an estimate of their mental representation of the to-be-reported orientation. We used multivariate inverted encoding models to reconstruct the neural representations of orientation in reference to the response. We found that response orientation could be successfully reconstructed from activity in early visual cortex, even on 0% contrast trials when no orientation information was actually presented, suggesting the existence of a purely internally generated neural code in early visual cortex. In addition, cross-generalization and multidimensional scaling analyses demonstrated that information derived from internal sources was represented differently from typical working memory representations, which receive influences from both external and internal sources. Similar results were also observed in intraparietal sulcus, with slightly different cross-generalization patterns. These results suggest a potential mechanism for how externally driven and internally generated information is maintained in working memory.

Why do imagery and perception look and feel so different?

Despite the past few decades of research providing convincing evidence of the similarities in function and neural mechanisms between imagery and perception, for most of us, the experience of the two are undeniably different, why? Here, we review and discuss the differences between imagery and perception and the possible underlying causes of these differences, from function to neural mechanisms. Specifically, we discuss the directional flow of information (top-down versus bottom-up), the differences in targeted cortical layers in primary visual cortex and possible different neural mechanisms of modulation versus excitation. For the first time in history, neuroscience is beginning to shed light on this long-held mystery of why imagery and perception look and feel so different. This article is part of the theme issue 'Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation'.

Eye movements during visual imagery and perception show spatial correspondence but have unique temporal signatures

Eye fixation patterns during mental imagery are similar to those during perception of the same picture, suggesting that oculomotor mechanisms play a role in mental imagery (i.e., the "looking at nothing" effect). Previous research has focused on the spatial similarities of eye movements during perception and mental imagery. The primary aim of this study was to assess whether the spatial similarity translates to the temporal domain. We used recurrence quantification analysis (RQA) to assess the temporal structure of eye fixations in visual perception and mental imagery and we compared the temporal as well as the spatial characteristics in mental imagery with perception by means of Bayesian hierarchical regression models. We further investigated how person and picture-specific characteristics contribute to eye movement behavior in mental imagery. Working memory capacity and mental imagery abilities were assessed to either predict gaze dynamics in visual imagery or to moderate a possible correspondence between spatial or temporal gaze dynamics in perception and mental imagery. We were able to show the spatial similarity of fixations between visual perception and imagery and we provide first evidence for its moderation by working memory capacity. Interestingly, the temporal gaze dynamics in mental imagery were unrelated to those in perception and their variance between participants was not explained by variance in visuo-spatial working memory capacity or vividness of mental images. The semantic content of the imagined pictures was the only meaningful predictor of temporal gaze dynamics. The spatial correspondence reflects shared spatial structure of mental images and perceived pictures, while the unique temporal gaze behavior could be driven by generation, maintenance and protection processes specific to visual imagery. The unique temporal gaze dynamics offer a window to new insights into the genuine process of mental imagery independent of its similarity to perception.

In your phase: neural phase synchronisation underlies visual imagery of faces

Mental imagery is the process through which we retrieve and recombine information from our memory to elicit the subjective impression of “seeing with the mind’s eye”. In the social domain, we imagine other individuals while recalling our encounters with them or modelling alternative social interactions in future. Many studies using imaging and neurophysiological techniques have shown several similarities in brain activity between visual imagery and visual perception, and have identified frontoparietal, occipital and temporal neural components of visual imagery. However, the neural connectivity between these regions during visual imagery of socially relevant stimuli has not been studied. Here we used electroencephalography to investigate neural connectivity and its dynamics between frontal, parietal, occipital and temporal electrodes during visual imagery of faces. We found that voluntary visual imagery of faces is associated with long-range phase synchronisation in the gamma frequency range between frontoparietal electrode pairs and between occipitoparietal electrode pairs. In contrast, no effect of imagery was observed in the connectivity between occipitotemporal electrode pairs. Gamma range synchronisation between occipitoparietal electrode pairs predicted subjective ratings of the contour definition of imagined faces. Furthermore, we found that visual imagery of faces is associated with an increase of short-range frontal synchronisation in the theta frequency range, which temporally preceded the long-range increase in the gamma synchronisation. We speculate that the local frontal synchrony in the theta frequency range might be associated with an effortful top-down mnemonic reactivation of faces. In contrast, the long-range connectivity in the gamma frequency range along the fronto-parieto-occipital axis might be related to the endogenous binding and subjective clarity of facial visual features.

Visual mental imagery engages the left fusiform gyrus, but not the early visual cortex: A meta-analysis of neuroimaging evidence

The dominant neural model of visual mental imagery (VMI) stipulates that memories from the medial temporal lobe acquire sensory features in early visual areas. However, neurological patients with damage restricted to the occipital cortex typically show perfectly vivid VMI, while more anterior damages extending into the temporal lobe, especially in the left hemisphere, often cause VMI impairments. Here we present two major results reconciling neuroimaging findings in neurotypical subjects with the performance of brain-damaged patients: (1) A large-scale meta-analysis of 46 fMRI studies, of which 27 investigated specifically visual mental imagery, revealed that VMI engages fronto-parietal networks and a well-delimited region in the left fusiform gyrus. (2) A Bayesian analysis showed no evidence for imagery-related activity in early visual cortices. We propose a revised neural model of VMI that draws inspiration from recent cytoarchitectonic and lesion studies, whereby fronto-parietal networks initiate, modulate, and maintain activity in a core temporal network centered on the fusiform imagery node, a high-level visual region in the left fusiform gyrus.

Narrowband multivariate source separation for semi-blind discovery of experiment contrasts

BACKGROUND

Electrophysiological recordings contain mixtures of signals from distinct neural sources, impeding a straightforward interpretation of the sensor-level data. This mixing is particularly detrimental when distinct sources resonate in overlapping frequencies. Fortunately, the mixing is linear and instantaneous. Multivariate source separation methods may therefore successfully separate statistical sources, even with overlapping spatial distributions.

NEW METHOD

We demonstrate a feature-guided multivariate source separation method that is tuned to narrowband frequency content as well as binary condition differences. This method - comparison scanning generalized eigendecomposition, csGED - harnesses the covariance structure of multichannel data to find directions (i.e., eigenvectors) that maximally separate two subsets of data. To drive condition specificity and frequency specificity, our data subsets were taken from different task conditions and narrowband-filtered prior to applying GED.

RESULTS

To validate the method, we simulated MEG data in two conditions with shared noise characteristics and unique signal. csGED outperformed the best sensor at reconstructing the ground truth signals, even in the presence of large amounts of noise. We next applied csGED to a published empirical MEG dataset on visual perception vs. imagery. csGED identified sources in alpha, beta, and gamma bands, and successfully separated distinct networks in the same frequency band.

COMPARISON WITH EXISTING METHOD(S)

GED is a flexible feature-guided decomposition method that has previously successfully been applied. Our combined frequency- and condition-tuning is a novel adaptation that extends the power of GED in cognitive electrophysiology.

CONCLUSIONS

We demonstrate successful condition-specific source separation by applying csGED to simulated and empirical data.

Predictive activation of sensory representations as a source of evidence in perceptual decision-making

Our brains can represent expected future states of our sensory environment. Recent work has shown that, when we expect a specific stimulus to appear at a specific time, we can predictively generate neural representations of that stimulus even before it is physically presented. These observations raise two exciting questions: Are pre-activated sensory representations used for perceptual decision-making? And, do we transiently perceive an expected stimulus that does not actually appear? To address these questions, we propose that pre-activated neural representations provide sensory evidence that is used for perceptual decision-making. This can be understood within the framework of the Diffusion Decision Model as an early accumulation of decision evidence in favour of the expected percept. Our proposal makes novel predictions relating to expectation effects on neural markers of decision evidence accumulation, and also provides an explanation for why we sometimes perceive stimuli that are expected, but do not appear.

Neural dynamics of perceptual inference and its reversal during imagery

After the presentation of a visual stimulus, neural processing cascades from low-level sensory areas to increasingly abstract representations in higher-level areas. It is often hypothesised that a reversal in neural processing underlies the generation of mental images as abstract representations are used to construct sensory representations in the absence of sensory input. According to predictive processing theories, such reversed processing also plays a central role in later stages of perception. Direct experimental evidence of reversals in neural information flow has been missing. Here, we used a combination of machine learning and magnetoencephalography to characterise neural dynamics in humans. We provide direct evidence for a reversal of the perceptual feed-forward cascade during imagery and show that, during perception, such reversals alternate with feed-forward processing in an 11 Hz oscillatory pattern. Together, these results show how common feedback processes support both veridical perception and mental imagery.

Visual Imagery and Perception Share Neural Representations in the Alpha Frequency Band

To behave adaptively with sufficient flexibility, biological organisms must cognize beyond immediate reaction to a physically present stimulus. For this, humans use visual mental imagery [1, 2], the ability to conjure up a vivid internal experience from memory that stands in for the percept of the stimulus. Visually imagined contents subjectively mimic perceived contents, suggesting that imagery and perception share common neural mechanisms. Using multivariate pattern analysis on human electroencephalography (EEG) data, we compared the oscillatory time courses of mental imagery and perception of objects. We found that representations shared between imagery and perception emerged specifically in the alpha frequency band. These representations were present in posterior, but not anterior, electrodes, suggesting an origin in parieto-occipital cortex. Comparison of the shared representations to computational models using representational similarity analysis revealed a relationship to later layers of deep neural networks trained on object representations, but not auditory or semantic models, suggesting representations of complex visual features as the basis of commonality. Together, our results identify and characterize alpha oscillations as a cortical signature of representations shared between visual mental imagery and perception.

•

Perception and imagery share neural representations in the alpha frequency band

•

Shared representations stem from parieto-occipital sources

•

Modeling suggests contents of shared representations are complex visual features

Feature-specific neural reactivation during episodic memory

We present a multi-voxel analytical approach, feature-specific informational connectivity (FSIC), that leverages hierarchical representations from a neural network to decode neural reactivation in fMRI data collected while participants performed an episodic visual recall task. We show that neural reactivation associated with low-level (e.g. edges), high-level (e.g. facial features), and semantic (e.g. “terrier”) features occur throughout the dorsal and ventral visual streams and extend into the frontal cortex. Moreover, we show that reactivation of both low- and high-level features correlate with the vividness of the memory, whereas only reactivation of low-level features correlates with recognition accuracy when the lure and target images are semantically similar. In addition to demonstrating the utility of FSIC for mapping feature-specific reactivation, these findings resolve the contributions of low- and high-level features to the vividness of visual memories and challenge a strict interpretation the posterior-to-anterior visual hierarchy.

Memory recollection involves reactivation of neural activity that occurred during the recalled experience. Here, the authors show that neural reactivation can be decomposed into visual-semantic features, is widely synchronized throughout the brain, and predicts memory vividness and accuracy.

The bottom-up and top-down processing of faces in the human occipitotemporal cortex

Although face processing has been studied extensively, the dynamics of how face-selective cortical areas are engaged remains unclear. Here, we uncovered the timing of activation in core face-selective regions using functional Magnetic Resonance Imaging and Magnetoencephalography in humans. Processing of normal faces started in the posterior occipital areas and then proceeded to anterior regions. This bottom-up processing sequence was also observed even when internal facial features were misarranged. However, processing of two-tone Mooney faces lacking explicit prototypical facial features engaged top-down projection from the right posterior fusiform face area to right occipital face area. Further, face-specific responses elicited by contextual cues alone emerged simultaneously in the right ventral face-selective regions, suggesting parallel contextual facilitation. Together, our findings chronicle the precise timing of bottom-up, top-down, as well as context-facilitated processing sequences in the occipital-temporal face network, highlighting the importance of the top-down operations especially when faced with incomplete or ambiguous input.