Learning and Recognition of a Non-conscious Sequence of Events in Human Primary Visual Cortex

Finding Pattern in the Noise: Persistent Implicit Statistical Knowledge Impacts the Processing of Unpredictable Stimuli

Humans can extract statistical regularities of the environment to predict upcoming events. Previous research recognized that implicitly acquired statistical knowledge remained persistent and continued to influence behavior even when the regularities were no longer present in the environment. Here, in an fMRI experiment, we investigated how the persistence of statistical knowledge is represented in the brain. Participants (n = 32) completed a visual, four-choice, RT task consisting of statistical regularities. Two types of blocks constantly alternated with one another throughout the task: predictable statistical regularities in one block type and unpredictable ones in the other. Participants were unaware of the statistical regularities and their changing distribution across the blocks. Yet, they acquired the statistical regularities and showed significant statistical knowledge at the behavioral level not only in the predictable blocks but also in the unpredictable ones, albeit to a smaller extent. Brain activity in a range of cortical and subcortical areas, including early visual cortex, the insula, the right inferior frontal gyrus, and the right globus pallidus/putamen contributed to the acquisition of statistical regularities. The right insula, inferior frontal gyrus, and hippocampus as well as the bilateral angular gyrus seemed to play a role in maintaining this statistical knowledge. The results altogether suggest that statistical knowledge could be exploited in a relevant, predictable context as well as transmitted to and retrieved in an irrelevant context without a predictable structure.

The dead salmon strikes again: Reports of unconscious processing in the hippocampus may reflect Type-I error

Steinkrauss and Slotnick (2024) reviewed neuroimaging studies linking the hippocampus with implicit memory. They conclude that there is no convincing evidence that the hippocampus is associated with implicit memory because prior studies are confounded by explicit memory (among other factors). Here, we ask a different yet equally important question: do reports of unconscious hippocampal activity reflect a Type-I error (i.e. a false positive)? We find that 39% of studies linking the hippocampus with implicit memory (7 of 18) do not report correcting for multiple comparisons. These results indicate that many unconscious hippocampal effects may reflect a Type-I error.

When perception fades, the hippocampus may support implicit memory

Steinkrauss and Slotnick (2024) conclude that current evidence is insufficient to sustain a link between implicit memory and the hippocampus. However, behavioral protocols designed to minimize visual awareness, so that memoranda are objectively invisible both at study and at test, can yield brain-based signals of implicit memory, which circumvent several of the identified constraints. Furthermore, while differences in novelty and attention complicate the interpretation of hippocampal involvement in implicit memory tasks, these processes can occur with and without conscious awareness, suggesting a more complex interplay between the hippocampus and memory-related processes than an exclusive association with consciousness would indicate.

Differential effects of bilateral hippocampal CA3 damage on the implicit learning and recognition of complex event sequences

Learning regularities in the environment is a fundament of human cognition, which is supported by a network of brain regions that include the hippocampus. In two experiments, we assessed the effects of selective bilateral damage to human hippocampal subregion CA3, which was associated with autobiographical episodic amnesia extending ~50 years prior to the damage, on the ability to recognize complex, deterministic event sequences presented either in a spatial or a non-spatial configuration. In contrast to findings from related paradigms, modalities, and homologue species, hippocampal damage did not preclude recognition memory for an event sequence studied and tested at four spatial locations, whereas recognition memory for an event sequence presented at a single location was at chance. In two additional experiments, recognition memory for novel single-items was intact, whereas the ability to recognize novel single-items in a different location from that presented at study was at chance. The results are at variance with a general role of the hippocampus in the learning and recognition of complex event sequences based on non-adjacent spatial and temporal dependencies. We discuss the impact of the results on established theoretical accounts of the hippocampal contributions to implicit sequence learning and episodic memory.

Stimulus awareness is necessary for both instrumental learning and instrumental responding to previously learned stimuli

Instrumental conditioning is a crucial part of adaptive behaviour, allowing agents to selectively interact with stimuli in their environment. Recent evidence suggests that instrumental conditioning cannot proceed without stimulus awareness. However, whether accurate unconscious instrumental responding can emerge from consciously acquired knowledge of the stimulus-action-outcome contingencies is unknown. We studied this question using instrumental trace conditioning, where participants learned to make approach/avoid decisions in two within-subject modes: conscious (stimuli in plain view) and unconscious (visually masked). Both tasks were followed by an unconscious-only instrumental performance task. We show that even when the contingencies are reliably learned in the conscious mode, participants fail to act upon them in the unconscious responding task. We also replicate the previous results that no instrumental learning occurs in the unconscious mode. Consequently, the absence of stimulus awareness not only precludes instrumental conditioning, but also precludes any kind of instrumental responding to already known stimuli. This suggests that instrumental behaviour is entirely supported by conscious awareness of the world, and corroborates the proposals that consciousness may be necessary for adaptive behaviours requiring selective action.

Location-specific deviant responses to object sequences in macaque inferior temporal cortex

Many species learn temporal regularities in their visual environment, demonstrating visual statistical learning. In this study, we explored the sensitivity of macaque inferior temporal (IT) cortical neurons to transition probabilities of sequentially presented visual images, presented at different locations in the visual field. We exposed monkeys to sequences of two images, where the first image was presented either foveally or peripherally, and the second image was consistently presented foveally. Following several weeks of exposure, we recorded IT responses to assess differences between the exposed (Fixed) and new, Deviant sequences, where the identity of the first image in a sequence differed from the exposure phase. While enhanced responses to Deviant sequences were observed when both images of a pair were foveally presented during exposure, no such deviant responses were present when the first image was presented peripherally. This finding challenges the notion that mere exposure to image sequences always leads to deviant responses in macaque IT. The results highlight the complexity of the mechanisms underlying statistical learning in primates, particularly in the context of peripheral image presentations, emphasizing the need for further investigation into the origins of these responses in the IT cortex.

Perception and Memory in the Ventral Visual Stream and Medial Temporal Lobe

Perception and memory are traditionally thought of as separate cognitive functions, supported by distinct brain regions. The canonical perspective is that perceptual processing of visual information is supported by the ventral visual stream, whereas long-term declarative memory is supported by the medial temporal lobe. However, this modular framework cannot account for the increasingly large body of evidence that reveals a role for early visual areas in long-term recognition memory and a role for medial temporal lobe structures in high-level perceptual processing. In this article, we review relevant research conducted in humans, nonhuman primates, and rodents. We conclude that the evidence is largely inconsistent with theoretical proposals that draw sharp functional boundaries between perceptual and memory systems in the brain. Instead, the weight of the empirical findings is best captured by a representational-hierarchical model that emphasizes differences in content, rather than in cognitive processes within the ventral visual stream and medial temporal lobe.

Covid-19 related cognitive, structural and functional brain changes among Italian adolescents and young adults: a multimodal longitudinal case-control study

Coronavirus disease 2019 (COVID-19) has been associated with brain functional, structural, and cognitive changes that persist months after infection. Most studies of the neurologic outcomes related to COVID-19 focus on severe infection and aging populations. Here, we investigated the neural activities underlying COVID-19 related outcomes in a case-control study of mildly infected youth enrolled in a longitudinal study in Lombardy, Italy, a global hotspot of COVID-19. All participants (13 cases, 27 controls, mean age 24 years) completed resting state functional (fMRI), structural MRI, cognitive assessments (CANTAB spatial working memory) at baseline (pre-COVID) and follow-up (post-COVID). Using graph theory eigenvector centrality (EC) and data-driven statistical methods, we examined differences in ECdelta (i.e., the difference in EC values pre- and post-COVID-19) and volumetricdelta (i.e., the difference in cortical volume of cortical and subcortical areas pre- and post-COVID) between COVID-19 cases and controls. We found that ECdeltasignificantly between COVID-19 and healthy participants in five brain regions; right intracalcarine cortex, right lingual gyrus, left hippocampus, left amygdala, left frontal orbital cortex. The left hippocampus showed a significant decrease in volumetricdelta between groups (p=0.041). The reduced ECdelta in the right amygdala associated with COVID-19 status mediated the association between COVID-19 and disrupted spatial working memory. Our results show persistent structural, functional and cognitive brain changes in key brain areas associated with olfaction and cognition. These results may guide treatment efforts to assess the longevity, reversibility and impact of the observed brain and cognitive changes following COVID-19.

Differential networks for processing structural dependencies in human language: linguistic capacity vs. memory-based ordering

Surface linear (left-to-right) arrangements of human languages are actually an amalgam of the core language system and systems that are not inherently related to language. It has been widely recognized that an unbounded array of hierarchically structured linguistic expressions is generated by the simplest combinatorial operation "Merge," and the notion of Merge-generability has been proposed as a key feature that characterizes structural dependencies among linguistic elements. Here we tested Merge-generable dependencies by using a Subject-Predicate matching task, which required both linguistic capacity and short-term memory. We used three types of dependency: Nesting, Crossing, and Grouping as the control. The Nesting dependency is totally Merge-generable, while the Crossing dependency requires some additional processes for memory-based ordering. In order to identify the regions employed for these two dependencies, we directly compared cortical responses to the sentence stimuli (with noun phrases and an adverb as the first half of stimuli, and with verbs as the latter) using functional magnetic resonance imaging (fMRI), and the following results were obtained. First, for the Nesting - Crossing contrast, significant activations were observed in the bilateral lateral premotor cortices (LPMCs) and inferior frontal gyri, left middle temporal gyrus, and bilateral angular/supramarginal gyri, indicating engagement of the syntax-related networks. In contrast, the Crossing - Nesting contrast showed focal activations in the left fusiform gyrus, lingual gyrus, and middle occipital gyrus (L. FG/LG/MOG). Secondly, for the first half of the Nesting stimuli, signal changes in the bilateral LPMCs were well fitted with the estimates of computational costs to search the workspace and to select items (Σ operations). Moreover, for the latter half of the Crossing stimuli, the signal changes in the L. FG/LG/MOG were differentially fitted with the estimates of loads related to the ordering of elements/words (numbers of Ordering). Thirdly, these fitting models were by far more likely than the exchanged estimates between bilateral LPMCs and L. FG/LG/MOG, confirming a double dissociation for primary processes with Σ and Ordering. In conclusion, these results indicate that separate cortical networks are differentially employed, and their careful elucidation will provide further insights and challenges.

Anxiety attenuates learning advantages conferred by statistical stability and induces loss of volatility-attuning in brain activity

Anxiety can alter an individual's perception of their external sensory environment. Previous studies suggest that anxiety can increase the magnitude of neural responses to unexpected (or surprising) stimuli. Additionally, surprise responses are reported to be boosted during stable compared to volatile environments. Few studies, however, have examined how learning is impacted by both threat and volatility. To investigate these effects, we used threat-of-shock to transiently increase subjective anxiety in healthy adults while they performed an auditory oddball task under stable and volatile environments and while undergoing functional Magnetic Resonance Imaging (fMRI) scanning. We then used Bayesian Model Selection (BMS) mapping to identify the brain areas where different models of anxiety displayed the highest evidence. Behaviourally, we found that threat-of-shock eliminated the accuracy advantage conferred by environmental stability over volatility. Neurally, we found that threat-of-shock led to attenuation and loss of volatility-attuning of brain activity evoked by surprising sounds across most subcortical and limbic regions including the thalamus, basal ganglia, claustrum, insula, anterior cingulate, hippocampal gyrus and the superior temporal gyrus. Taken together, our findings suggest that threat eliminates learning advantages conferred by statistical stability compared to volatility. Thus, we propose that anxiety disrupts behavioural adaptation to environmental statistics, and that multiple subcortical and limbic regions are implicated in this process.

Inferior temporal cortex leads prefrontal cortex in response to a violation of a learned sequence

Primates learn statistical regularities that are embedded in visual sequences, a form of statistical learning. Single-unit recordings in macaques showed that inferior temporal (IT) neurons are sensitive to statistical regularities in visual sequences. Here, we asked whether ventrolateral prefrontal cortex (VLPFC), which is connected to IT, is also sensitive to the transition probabilities in visual sequences and whether the statistical learning signal in IT originates in VLPFC. We recorded simultaneously multiunit activity (MUA) and local field potentials (LFPs) in IT and VLPFC after monkeys were exposed to triplets of images with a fixed presentation order. In both areas, the MUA was stronger to images that violated the learned sequence (deviants) compared to the same images presented in the learned triplets. The high-gamma and beta LFP power showed an enhanced and suppressed response, respectively, to the deviants in both areas. The enhanced response was present also for the image following the deviant, suggesting a sensitivity for temporal adjacent dependencies in IT and VLPFC. The increased response to the deviant occurred later in VLPFC than in IT, suggesting that the deviant response in IT was not inherited from VLPFC. These data support predictive coding theories that propose a feedforward flow of prediction errors.

Consensus Goals in the Field of Visual Metacognition

Despite the tangible progress in psychological and cognitive sciences over the last several years, these disciplines still trail other more mature sciences in identifying the most important questions that need to be solved. Reaching such consensus could lead to greater synergy across different laboratories, faster progress, and increased focus on solving important problems rather than pursuing isolated, niche efforts. Here, 26 researchers from the field of visual metacognition reached consensus on four long-term and two medium-term common goals. We describe the process that we followed, the goals themselves, and our plans for accomplishing these goals. If this effort proves successful within the next few years, such consensus building around common goals could be adopted more widely in psychological science.

Reduced hippocampal-cortical connectivity during memory suppression predicts the ability to forget unwanted memories

The ability to suppress unwelcome memories is important for productivity and well-being. Successful memory suppression is associated with hippocampal deactivations and a concomitant disruption of this region's functionality. Much of the previous neuroimaging literature exploring such suppression-related hippocampal modulations has focused on the region's negative coupling with the prefrontal cortex. Task-based changes in functional connectivity between the hippocampus and other brain regions still need further exploration. In the present study, we utilize psychophysiological interactions and seed connectome-based predictive modeling to investigate the relationship between the hippocampus and the rest of the brain as 134 participants attempted to suppress unwanted memories during the Think/No-Think task. The results show that during retrieval suppression, the right hippocampus exhibited decreased functional connectivity with visual cortical areas (lingual and cuneus gyrus), left nucleus accumbens and the brain-stem that predicted superior forgetting of unwanted memories on later memory tests. Validation tests verified that prediction performance was not an artifact of head motion or prediction method and that the negative features remained consistent across different brain parcellations. These findings suggest that systemic memory suppression involves more than the modulation of hippocampal activity-it alters functional connectivity patterns between the hippocampus and visual cortex, leading to successful forgetting.

Neuroscientific Evidence for Processing Without Awareness

The extent to which we are affected by perceptual input of which we are unaware is widely debated. By measuring neural responses to sensory stimulation, neuroscientific data could complement behavioral results with valuable evidence. Here we review neuroscientific findings of processing of high-level information, as well as interactions with attention and memory. Although the results are mixed, we find initial support for processing object categories and words, possibly to the semantic level, as well as emotional expressions. Robust neural evidence for face individuation and integration of sentences or scenes is lacking. Attention affects the processing of stimuli that are not consciously perceived, and such stimuli may exogenously but not endogenously capture attention when relevant, and be maintained in memory over time. Sources of inconsistency in the literature include variability in control for awareness as well as individual differences, calling for future studies that adopt stricter measures of awareness and probe multiple processes within subjects.

Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks

A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains.

Dynamic development of intuitions and explicit knowledge during implicit learning

Implicit learning refers to learning without conscious awareness of the content acquired. Theoretical frameworks of human cognition suggest that intuitions develop based on incomplete perceptions of regularity during implicit learning and, in turn, lead to the development of more explicit, consciously-accessible knowledge. Surprisingly, however, this putative information processing pathway (i.e., implicit learning ➔ intuition ➔ explicit knowledge) has yet to be empirically demonstrated. The present study investigated the relationship between implicit learning, intuitions, and explicit knowledge using a modified Serial Reaction Time Task. Results indicate that intuitions of implicitly-learned patterns emerge prior to the development of explicit knowledge. Moreover, intuition timing and accuracy were significantly associated with accuracy of explicit reports. We did not, however, find that stronger implicit learners developed more accurate intuitions. Our findings suggest a crucial role of intuition in the formation of explicit knowledge from implicit learning.

Occipital sleep spindles predict sequence learning in a visuo-motor task

STUDY OBJECTIVES

The brain appears to use internal models to successfully interact with its environment via active predictions of future events. Both internal models and the predictions derived from them are based on previous experience. However, it remains unclear how previously encoded information is maintained to support this function, especially in the visual domain. In the present study, we hypothesized that sleep consolidates newly encoded spatio-temporal regularities to improve predictions afterwards.

METHODS

We tested this hypothesis using a novel sequence-learning paradigm that aimed to dissociate perceptual from motor learning. We recorded behavioral performance and high-density electroencephalography (EEG) in male human participants during initial training and during testing two days later, following an experimental night of sleep (n = 16, including high-density EEG recordings) or wakefulness (n = 17).

RESULTS

Our results show sleep-dependent behavioral improvements correlated with sleep-spindle activity specifically over occipital cortices. Moreover, event-related potential (ERP) responses indicate a shift of attention away from predictable to unpredictable sequences after sleep, consistent with enhanced automaticity in the processing of predictable sequences.

CONCLUSIONS

These findings suggest a sleep-dependent improvement in the prediction of visual sequences, likely related to visual cortex reactivation during sleep spindles. Considering that controls in our experiments did not fully exclude oculomotor contributions, future studies will need to address the extent to which these effects depend on purely perceptual versus oculomotor sequence learning.

Larger capacity for unconscious versus conscious episodic memory

Episodic memory is the memory for experienced events. A peak competence of episodic memory is the mental combination of events to infer commonalities. Inferring commonalities may proceed with and without consciousness of events. Yet what distinguishes conscious from unconscious inference? This question inspired nine experiments that featured strongly and weakly masked cartoon clips presented for unconscious and conscious inference. Each clip featured a scene with a visually impenetrable hiding place. Five animals crossed the scene one-by-one consecutively. One animal trajectory represented one event. The animals moved through the hiding place, where they might linger or not. The participants' task was to observe the animals' entrances and exits to maintain a mental record of which animals hid simultaneously. We manipulated information load to explore capacity limits. Memory of inferences was tested immediately, 3.5 or 6 min following encoding. The participants retrieved inferences well when encoding was conscious. When encoding was unconscious, the participants needed to respond intuitively. Only habitually intuitive decision makers exhibited a significant delayed retrieval of inferences drawn unconsciously. Their unconscious retrieval performance did not drop significantly with increasing information load, while conscious retrieval performance dropped significantly. A working memory network, including hippocampus, was activated during both conscious and unconscious inference and correlated with retrieval success. An episodic retrieval network, including hippocampus, was activated during both conscious and unconscious retrieval of inferences and correlated with retrieval success. Only conscious encoding/retrieval recruited additional brain regions outside these networks. Hence, levels of consciousness influenced the memories' behavioral impact, memory capacity, and the neural representational code.

When the brain, but not the person, remembers: Cortical reinstatement is modulated by retrieval goal in developmental amnesia

Developmental amnesia (DA) is associated with early hippocampal damage and subsequent episodic amnesia emerging in childhood alongside age-appropriate development of semantic knowledge. We employed fMRI to assess whether patients with DA show evidence of 'cortical reinstatement', a neural correlate of episodic memory, despite their amnesia. At study, 23 participants (5 patients) were presented with words overlaid on a scene or a scrambled image for later recognition. Scene reinstatement was indexed by scene memory effects (greater activity for previously presented words paired with a scene rather than scrambled images) that overlapped with scene perception effects. Patients with DA demonstrated scene reinstatement effects in the parahippocampal and retrosplenial cortex that were equivalent to those shown by healthy controls. Behaviourally, however, patients with DA showed markedly impaired scene memory. The data indicate that reinstatement can occur despite hippocampal damage, but that cortical reinstatement is insufficient to support accurate memory performance. Furthermore, scene reinstatement effects were diminished during a retrieval task in which scene information was not relevant for accurate responding, indicating that strategic mnemonic processes operate normally in DA. The data suggest that cortical reinstatement of trial-specific contextual information is decoupled from the experience of recollection in the presence of severe hippocampal atrophy.

Neural Correlates of Auditory Pattern Learning in the Auditory Cortex

Learning of new auditory stimuli often requires repetitive exposure to the stimulus. Fast and implicit learning of sounds presented at random times enables efficient auditory perception. However, it is unclear how such sensory encoding is processed on a neural level. We investigated neural responses that are developed from a passive, repetitive exposure to a specific sound in the auditory cortex of anesthetized rats, using electrocorticography. We presented a series of random sequences that are generated afresh each time, except for a specific reference sequence that remains constant and re-appears at random times across trials. We compared induced activity amplitudes between reference and fresh sequences. Neural responses from both primary and non-primary auditory cortical regions showed significantly decreased induced activity amplitudes for reference sequences compared to fresh sequences, especially in the beta band. This is the first study showing that neural correlates of auditory pattern learning can be evoked even in anesthetized, passive listening animal models.

Brain network analysis reveals that amyloidopathy affects comorbid cognitive dysfunction in older adults with depression

Late-life depression (LLD) may increase the risk of Alzheimer's dementia (AD). While amyloidopathy accelerates AD progression, its role in such patients has not yet been elucidated. We hypothesized that cerebral amyloidopathy distinctly affects the alteration of brain network topology and may be associated with distinct cognitive symptoms. We recruited 26 and 27 depressed mild cognitive impairment (MCI) patients with (LLD-MCI-A(+)) and without amyloid accumulation (LLD-MCI-A(-)), respectively, and 21 normal controls. We extracted structural brain networks using their diffusion-weighted images. We aimed to compare the distinct network deterioration in LLD-MCI with and without amyloid accumulation and the relationship with their distinct cognitive decline. Thus, we performed a group comparison of the network topological measures and investigated any correlations with neurocognitive testing scores. Topological features of brain networks were different according to the presence of amyloid accumulation. Disrupted network connectivity was highly associated with impaired recall and recognition in LLD-MCI-A(+) patients. Inattention and dysexecutive function were more influenced by the altered networks involved in fronto-limbic circuitry dysfunction in LLD-MCI-A(-) patients. Our results show that alterations in brain network topology may reflect different cognitive dysfunction depending on amyloid accumulation in depressed older adults with MCI.

Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex

Animals of several species, including primates, learn the statistical regularities of their environment. In particular, they learn the temporal regularities that occur in streams of visual images. Previous human neuroimaging studies reported discrepant effects of such statistical learning, ranging from stronger occipito-temporal activations for sequences in which image order was fixed, compared with sequences of randomly ordered images, to weaker activations for fixed-order sequences compared with sequences that violated the learned order. Several single-unit studies in macaque monkeys reported that after statistical learning of temporal regularities, inferior temporal (IT) neurons show reduced responses to learned fixed-order sequences of visual images compared with random or mispredicted sequences. However, it is unknown how other macaque brain areas respond to such temporal statistical regularities. To address this gap, we exposed rhesus monkeys (Macaca mulatta) to two types of sequences of complex images. The “regular” sequences consisted of a continuous stream of quartets, and within each quartet, the image order was fixed. The quartets themselves were displayed, uninterrupted, in a random order. The same monkeys were exposed to sequences of other images having a pseudorandomized order (“random” sequence). After exposure, both monkeys were scanned with functional MRI (fMRI) using a block design with three conditions: regular sequence, random sequence, and fixation-only blocks. A whole-brain analysis showed a reduced activation in mainly the occipito-temporal cortex for the regular compared to the random sequences. Marked response reductions for the regular sequence were observed in early extrastriate visual cortical areas, including area V2, despite the use of rather complex images of animals. These data suggest that statistical learning signals are already present in early visual areas of monkeys, even for complex visual images. These monkey fMRI data are in line with recent human fMRI studies that showed a reduced activation in early visual areas for predicted compared with mispredicted complex images.

Memory Engrams in the Neocortex

While in the past much of our knowledge about memory representations in the brain has relied on loss-of-function studies in which whole brain regions were temporarily inactivated or permanently lesioned, the recent development of new methods has ushered in a new era of downright "engram excitement." Animal research is now able to specifically label, track, and manipulate engram cells in the brain. While early studies have mostly focused on single brain regions like the hippocampus, recently more and more evidence for brain-wide distributed engram networks is emerging. Memory research in humans has also picked up pace, fueled by promising magnetic resonance imaging (MRI)-based methods like diffusion-weighted MRI (DW-MRI) and brain decoding. In this review, we will outline recent advancements in engram research, with a focus on human data and neocortical representations. We will illustrate the available noninvasive methods for the detection of engrams in different neocortical regions like the medial prefrontal cortex and the posterior parietal cortex and discuss evidence for systems consolidation and parallel memory encoding. Finally, we will explore how reactivation and prior knowledge can lead to and enhance engram formation in the neocortex.

Similar history biases for distinct prospective decisions of self-performance

Metacognition can be deployed retrospectively -to reflect on the correctness of our behavior- or prospectively -to make predictions of success in one’s future behavior or make decisions about strategies to solve future problems. We investigated the factors that determine prospective decision making. Human participants performed a visual discrimination task followed by ratings of visibility and response confidence. Prior to each trial, participants made prospective judgments. In Experiment 1, they rated their belief of future success. In Experiment 2, they rated their decision to adopt a focused attention state. Prospective beliefs of success were associated with no performance changes while prospective decisions to engage attention were followed by better self-evaluation of the correctness of behavioral responses. Using standard machine learning classifiers we found that the current prospective decision could be predicted from information concerning task-correctness, stimulus visibility and response confidence from previous trials. In both Experiments, awareness and confidence were more diagnostic of the prospective decision than task correctness. Notably, classifiers trained with prospective beliefs of success in Experiment 1 predicted decisions to engage in Experiment 2 and vice-versa. These results indicate that the formation of these seemingly different prospective decisions share a common, dynamic representational structure.

The Confidence Database

Understanding how people rate their confidence is critical for the characterization of a wide range of perceptual, memory, motor and cognitive processes. To enable the continued exploration of these processes, we created a large database of confidence studies spanning a broad set of paradigms, participant populations and fields of study. The data from each study are structured in a common, easy-to-use format that can be easily imported and analysed using multiple software packages. Each dataset is accompanied by an explanation regarding the nature of the collected data. At the time of publication, the Confidence Database (which is available at https://osf.io/s46pr/) contained 145 datasets with data from more than 8,700 participants and almost 4 million trials. The database will remain open for new submissions indefinitely and is expected to continue to grow. Here we show the usefulness of this large collection of datasets in four different analyses that provide precise estimations of several foundational confidence-related effects.

Neural bases of learning and recognition of statistical regularities

Statistical learning is a set of cognitive mechanisms allowing for extracting regularities from the environment and segmenting continuous sensory input into discrete units. The current study used functional magnetic resonance imaging (fMRI) (N = 25) in conjunction with an artificial language learning paradigm to provide new insight into the neural mechanisms of statistical learning, considering both the online process of extracting statistical regularities and the subsequent offline recognition of learned patterns. Notably, prior fMRI studies on statistical learning have not contrasted neural activation during the learning and recognition experimental phases. Here, we found that learning is supported by the superior temporal gyrus and the anterior cingulate gyrus, while subsequent recognition relied on the left inferior frontal gyrus. Besides, prior studies only assessed the brain response during the recognition of trained words relative to novel nonwords. Hence, a further key goal of this study was to understand how the brain supports recognition of discrete constituents from the continuous input versus recognition of mere statistical structure that is used to build new constituents that are statistically congruent with the ones from the input. Behaviorally, recognition performance indicated that statistically congruent novel tokens were less likely to be endorsed as parts of the familiar environment than discrete constituents. fMRI data showed that the left intraparietal sulcus and angular gyrus support the recognition of old discrete constituents relative to novel statistically congruent items, likely reflecting an additional contribution from memory representations for trained items.

Working memory can compare two visual items without accessing visual consciousness

Recent studies argued that unconscious visual information could access the working memory, however, it is still unclear whether the central executive could be activated unconsciously. We investigated, using a delayed match-to-sample task, whether the central executive is an unconscious process. In the experiment of the present study, participants were asked to compare the locations of two given visual targets. Both targets (or one of the two targets, depending on the experimental condition) were masked by a visual masking paradigm. The results showed an above-chance-level performance even in the condition that participants compared two unconscious targets. However, when the trials with the non-visual conscious experience of the target were removed from the analysis, the performance was no longer significantly different from chance level. Our results suggest that the central executive could be activated unconsciously by some level of stimulus signal, that is still below the threshold for a subjective report.

Learning predictive structure without a teacher: decision strategies and brain routes

Extracting the structure of complex environments is at the core of our ability to interpret the present and predict the future. This skill is important for a range of behaviours from navigating a new city to learning music and language. Classical approaches that investigate our ability to extract the principles of organisation that govern complex environments focus on reward-based learning. Yet, the human brain is shown to be expert at learning generative structure based on mere exposure and without explicit reward. Individuals are shown to adapt to-unbeknownst to them-changes in the environment's temporal statistics and predict future events. Further, we present evidence for a common brain architecture for unsupervised structure learning and reward-based learning, suggesting that the brain is built on the premise that 'learning is its own reward' to support adaptive behaviour.

Post-training TMS abolishes performance improvement and releases future learning from interference

The period immediately after the offset of visual training is thought to be critical for memory consolidation. Nevertheless, we still lack direct evidence for the causal role of this period to perceptual learning of either previously or subsequently trained material. To address these issues, we had human subjects complete two consecutive trainings with different tasks (detecting different Gabor orientations). We applied continuous theta burst stimulation (cTBS) to either the visual cortex or a control site (vertex) immediately after the offset of the first training. In the vertex cTBS condition, subjects showed improvement on the first task but not on the second task, suggesting the presence of anterograde interference. Critically, cTBS to the visual cortex abolished the performance improvement on the first task and released the second training from the anterograde interference. These results provide causal evidence for a role of the immediate post-training period in the consolidation of perceptual learning.

A Roadmap for Understanding Memory: Decomposing Cognitive Processes into Operations and Representations

Thanks to patients Phineas Gage and Henry Molaison, we have long known that behavioral control depends on the frontal lobes, whereas declarative memory depends on the medial temporal lobes (MTL). For decades, cognitive functions-behavioral control, declarative memory-have served as labels for characterizing the division of labor in cortex. This approach has made enormous contributions to understanding how the brain enables the mind, providing a systems-level explanation of brain function that constrains lower-level investigations of neural mechanism. Today, the approach has evolved such that functional labels are often applied to brain networks rather than focal brain regions. Furthermore, the labels have diversified to include both broadly-defined cognitive functions (declarative memory, visual perception) and more circumscribed mental processes (recollection, familiarity, priming). We ask whether a process-a high-level mental phenomenon corresponding to an introspectively-identifiable cognitive event-is the most productive label for dissecting memory. For example, recollection conflates a neurocomputational operation (pattern completion-based retrieval) with a class of representational content (associative, high-dimensional memories). Because a full theory of memory must identify operations and representations separately, and specify how they interact, we argue that processes like recollection constitute inadequate labels for characterizing neural mechanisms. Instead, we advocate considering the component operations and representations of processes like recollection in isolation. For the organization of memory, the evidence suggests that pattern completion is recapitulated widely across the ventral visual stream and MTL, but the division of labor between sites within this pathway can be explained by representational content.

Feature-Specific Awake Reactivation in Human V1 after Visual Training

Brain activity patterns exhibited during task performance have been shown to spontaneously reemerge in the following restful awake state. Such "awake reactivation" has been observed across higher-order cortex for complex images or associations. However, it is still unclear whether the reactivation extends to primary sensory areas that encode simple stimulus features. To address this question, we trained human subjects from both sexes on a particular visual feature (Gabor orientation) and tested whether this feature will be reactivated immediately after training. We found robust reactivation in human V1 that lasted for at least 8 min after training offset. This effect was not present in higher retinotopic areas, such as V2, V3, V3A, or V4v. Further analyses suggested that the amount of awake reactivation was related to the amount of performance improvement on the visual task. These results demonstrate that awake reactivation extends beyond higher-order areas and also occurs in early sensory cortex.SIGNIFICANCE STATEMENT How do we acquire new memories and skills? New information is known to be consolidated during offline periods of rest. Recent studies suggest that a critical process during this period of consolidation is the spontaneous reactivation of previously experienced patterns of neural activity. However, research in humans has mostly examined such reactivation processes in higher-order cortex. Here we show that awake reactivation occurs even in the primary visual cortex V1 and that this reactivation is related to the amount of behavioral learning. These results pinpoint awake reactivation as a process that likely occurs across the entire human brain and could play an integral role in memory consolidation.

Sleep Strengthens Predictive Sequence Coding

Predictive-coding theories assume that perception and action are based on internal models derived from previous experience. Such internal models require selection and consolidation to be stored over time. Sleep is known to support memory consolidation. We hypothesized that sleep supports both consolidation and abstraction of an internal task model that is subsequently used to predict upcoming stimuli. Human subjects (of either sex) were trained on deterministic visual sequences and tested with interleaved deviant stimuli after retention intervals of sleep or wakefulness. Adopting a predictive-coding approach, we found increased prediction strength after sleep, as expressed by increased error rates to deviant stimuli, but fewer errors for the immediately following standard stimuli. Sleep likewise enhanced the formation of an abstract sequence model, independent of the temporal context during training. Moreover, sleep increased confidence for sequence knowledge, reflecting enhanced metacognitive access to the model. Our results suggest that sleep supports the formation of internal models which can be used to predict upcoming events in different contexts.SIGNIFICANCE STATEMENT To efficiently interact with the ever-changing world, we predict upcoming events based on similar previous experiences. Sleep is known to benefit memory consolidation. However, it is not clear whether sleep specifically supports the transformation of past experience into predictions of future events. Here, we find that, when human subjects sleep after learning a sequence of predictable visual events, they make better predictions about upcoming events compared with subjects who stayed awake for an equivalent period of time. In addition, sleep supports the transfer of such knowledge between different temporal contexts (i.e., when sequences unfold at different speeds). Thus, sleep supports perception and action by enhancing the predictive utility of previous experiences.

What's in a context? Cautions, limitations, and potential paths forward

The purpose of memory is to guide current and future behavior based on previous experiences. Part of this process involves either discriminating between or generalizing across similar experiences that contain overlapping conditions (such as space, time, or internal state), which we often conceptualize as "contexts". In this review, we highlight major challenges facing the field as we attempt a neuroscience-based approach to the study of context and its impact on learning and memory. Here, we review some of the methodologies and approaches used to investigate context in both animals and humans, including the neurobiological mechanisms involved. Finally, we propose three tenets for operationalizing context in the experimental setting: 1) contexts must be stable over time along an experiential dimension; 2) contexts must be at least moderately complex in nature and their representations must be modifiable or adaptable, and 3) contexts must have some behavioral relevance (be it overt or incidental) so that its role can be measured.

Learning of goal-relevant and -irrelevant complex visual sequences in human V1

Learning and memory are supported by a network involving the medial temporal lobe and linked neocortical regions. Emerging evidence indicates that primary visual cortex (i.e., V1) may contribute to recognition memory, but this has been tested only with a single visuospatial sequence as the target memorandum. The present study used functional magnetic resonance imaging to investigate whether human V1 can support the learning of multiple, concurrent complex visual sequences involving discontinous (second-order) associations. Two peripheral, goal-irrelevant but structured sequences of orientated gratings appeared simultaneously in fixed locations of the right and left visual fields alongside a central, goal-relevant sequence that was in the focus of spatial attention. Pseudorandom sequences were introduced at multiple intervals during the presentation of the three structured visual sequences to provide an online measure of sequence-specific knowledge at each retinotopic location. We found that a network involving the precuneus and V1 was involved in learning the structured sequence presented at central fixation, whereas right V1 was modulated by repeated exposure to the concurrent structured sequence presented in the left visual field. The same result was not found in left V1. These results indicate for the first time that human V1 can support the learning of multiple concurrent sequences involving complex discontinuous inter-item associations, even peripheral sequences that are goal-irrelevant.

White-Matter Pathways for Statistical Learning of Temporal Structures

Extracting the statistics of event streams in natural environments is critical for interpreting current events and predicting future ones. The brain is known to rapidly find structure and meaning in unfamiliar streams of sensory experience, often by mere exposure to the environment (i.e., without explicit feedback). Yet, we know little about the brain pathways that support this type of statistical learning. Here, we test whether changes in white-matter (WM) connectivity due to training relate to our ability to extract temporal regularities. By combining behavioral training and diffusion tensor imaging (DTI), we demonstrate that humans adapt to the environment's statistics as they change over time from simple repetition to probabilistic combinations. In particular, we show that learning relates to the decision strategy that individuals adopt when extracting temporal statistics. We next test for learning-dependent changes in WM connectivity and ask whether they relate to individual variability in decision strategy. Our DTI results provide evidence for dissociable WM pathways that relate to individual strategy: extracting the exact sequence statistics (i.e., matching) relates to connectivity changes between caudate and hippocampus, while selecting the most probable outcomes in a given context (i.e., maximizing) relates to connectivity changes between prefrontal, cingulate and basal ganglia (caudate, putamen) regions. Thus, our findings provide evidence for distinct cortico-striatal circuits that show learning-dependent changes of WM connectivity and support individual ability to learn behaviorally-relevant statistics.

Prevailing theories of consciousness are challenged by novel cross-modal associations acquired between subliminal stimuli

While theories of consciousness differ substantially, the 'conscious access hypothesis', which aligns consciousness with the global accessibility of information across cortical regions, is present in many of the prevailing frameworks. This account holds that consciousness is necessary to integrate information arising from independent functions such as the specialist processing required by different senses. We directly tested this account by evaluating the potential for associative learning between novel pairs of subliminal stimuli presented in different sensory modalities. First, pairs of subliminal stimuli were presented and then their association assessed by examining the ability of the first stimulus to prime classification of the second. In Experiments 1-4 the stimuli were word-pairs consisting of a male name preceding either a creative or uncreative profession. Participants were subliminally exposed to two name-profession pairs where one name was paired with a creative profession and the other an uncreative profession. A supraliminal task followed requiring the timed classification of one of those two professions. The target profession was preceded by either the name with which it had been subliminally paired (concordant) or the alternate name (discordant). Experiment 1 presented stimuli auditorily, Experiment 2 visually, and Experiment 3 presented names auditorily and professions visually. All three experiments revealed the same inverse priming effect with concordant test pairs associated with significantly slower classification judgements. Experiment 4 sought to establish if learning would be more efficient with supraliminal stimuli and found evidence that a different strategy is adopted when stimuli are consciously perceived. Finally, Experiment 5 replicated the unconscious cross-modal association achieved in Experiment 3 utilising non-linguistic stimuli. The results demonstrate the acquisition of novel cross-modal associations between stimuli which are not consciously perceived and thus challenge the global access hypothesis and those theories embracing it.

Working Memory and Consciousness: The Current State of Play

Working memory (WM), an important posit in cognitive science, allows one to temporarily store and manipulate information in the service of ongoing tasks. WM has been traditionally classified as an explicit memory system-that is, as operating on and maintaining only consciously perceived information. Recently, however, several studies have questioned this assumption, purporting to provide evidence for unconscious WM. In this article, we focus on visual working memory (VWM) and critically examine these studies as well as studies of unconscious perception that seem to provide indirect evidence for unconscious WM. Our analysis indicates that current evidence does not support an unconscious WM store, though we offer independent reasons to think that WM may operate on unconsciously perceived information.

Subliminally and Supraliminally Acquired Long-Term Memories Jointly Bias Delayed Decisions

Common wisdom and scientific evidence suggest that good decisions require conscious deliberation. But growing evidence demonstrates that not only conscious but also unconscious thoughts influence decision-making. Here, we hypothesize that both consciously and unconsciously acquired memories guide decisions. Our experiment measured the influence of subliminally and supraliminally presented information on delayed (30-40 min) decision-making. Participants were presented with subliminal pairs of faces and written occupations for unconscious encoding. Following a delay of 20 min, participants consciously (re-)encoded the same faces now presented supraliminally along with either the same written occupations, occupations congruous to the subliminally presented occupations (same wage-category), or incongruous occupations (opposite wage-category). To measure decision-making, participants viewed the same faces again (with occupations absent) and decided on the putative income of each person: low, low-average, high-average, or high. Participants were encouraged to decide spontaneously and intuitively. Hence, the decision task was an implicit or indirect test of relational memory. If conscious thought alone guided decisions (= H₀), supraliminal information should determine decision outcomes independently of the encoded subliminal information. This was, however, not the case. Instead, both unconsciously and consciously encoded memories influenced decisions: identical unconscious and conscious memories exerted the strongest bias on income decisions, while both incongruous and congruous (i.e., non-identical) subliminally and supraliminally formed memories canceled each other out leaving no bias on decisions. Importantly, the increased decision bias following the formation of identical unconscious and conscious memories and the reduced decision bias following to the formation of non-identical memories were determined relative to a control condition, where conscious memory formation alone could influence decisions. In view of the much weaker representational strength of subliminally vs. supraliminally formed memories, their long-lasting impact on decision-making is noteworthy.

Learning Predictive Statistics: Strategies and Brain Mechanisms

When immersed in a new environment, we are challenged to decipher initially incomprehensible streams of sensory information. However, quite rapidly, the brain finds structure and meaning in these incoming signals, helping us to predict and prepare ourselves for future actions. This skill relies on extracting the statistics of event streams in the environment that contain regularities of variable complexity from simple repetitive patterns to complex probabilistic combinations. Here, we test the brain mechanisms that mediate our ability to adapt to the environment's statistics and predict upcoming events. By combining behavioral training and multisession fMRI in human participants (male and female), we track the corticostriatal mechanisms that mediate learning of temporal sequences as they change in structure complexity. We show that learning of predictive structures relates to individual decision strategy; that is, selecting the most probable outcome in a given context (maximizing) versus matching the exact sequence statistics. These strategies engage distinct human brain regions: maximizing engages dorsolateral prefrontal, cingulate, sensory-motor regions, and basal ganglia (dorsal caudate, putamen), whereas matching engages occipitotemporal regions (including the hippocampus) and basal ganglia (ventral caudate). Our findings provide evidence for distinct corticostriatal mechanisms that facilitate our ability to extract behaviorally relevant statistics to make predictions.SIGNIFICANCE STATEMENT Making predictions about future events relies on interpreting streams of information that may initially appear incomprehensible. Past work has studied how humans identify repetitive patterns and associative pairings. However, the natural environment contains regularities that vary in complexity from simple repetition to complex probabilistic combinations. Here, we combine behavior and multisession fMRI to track the brain mechanisms that mediate our ability to adapt to changes in the environment's statistics. We provide evidence for an alternate route for learning complex temporal statistics: extracting the most probable outcome in a given context is implemented by interactions between executive and motor corticostriatal mechanisms compared with visual corticostriatal circuits (including hippocampal cortex) that support learning of the exact temporal statistics.

A theory of working memory without consciousness or sustained activity

Working memory and conscious perception are thought to share similar brain mechanisms, yet recent reports of non-conscious working memory challenge this view. Combining visual masking with magnetoencephalography, we investigate the reality of non-conscious working memory and dissect its neural mechanisms. In a spatial delayed-response task, participants reported the location of a subjectively unseen target above chance-level after several seconds. Conscious perception and conscious working memory were characterized by similar signatures: a sustained desynchronization in the alpha/beta band over frontal cortex, and a decodable representation of target location in posterior sensors. During non-conscious working memory, such activity vanished. Our findings contradict models that identify working memory with sustained neural firing, but are compatible with recent proposals of 'activity-silent' working memory. We present a theoretical framework and simulations showing how slowly decaying synaptic changes allow cell assemblies to go dormant during the delay, yet be retrieved above chance-level after several seconds.

The Anatomy of Non-conscious Recognition Memory

Cortical regions as early as primary visual cortex have been implicated in recognition memory. Here, we outline the challenges that this presents for neurobiological accounts of recognition memory. We conclude that understanding the role of early visual cortex (EVC) in this process will require the use of protocols that mask stimuli from visual awareness.

Is conscious awareness needed for all working memory processes?

Stein and colleagues argue there is no yet conclusive evidence for nonconscious working memory (WM) and that is critical to probe WM while ensuring null sensitivity to memory cues. While this stringent approach reduces the likelihood of nonconscious signaling for WM, we discuss existing work meeting this null sensitivity criteria, and, related work on nonconscious cognition in keeping with WM/awareness dissociations on the basis of a functional operational definition of WM. Further, because it is likely that WM is a nonunitary functional construct and visual awareness a gradual phenomenon, we propose that delineating the neural mechanisms for distinct WM types across different levels of awareness may prove the most fruitful approach for understanding the interplay between WM and consciousness.

Implicit memory for words heard during sleep

When we fall asleep, our awareness of the surrounding world fades. Yet, the sleeping brain is far from being dormant and recent research unraveled the preservation of complex sensory processing during sleep. In wakefulness, such processes usually lead to the formation of long-term memory traces, being it implicit or explicit. We examined here the consequences upon awakening of the processing of sensory information at a high level of representation during sleep. Participants were instructed to classify auditory stimuli as words or pseudo-words, through left and right hand responses, while transitioning toward sleep. An analysis of the electroencephalographic (EEG) signal revealed the preservation of lateralized motor activations in response to sounds, suggesting that stimuli were correctly categorized during sleep. Upon awakening, participants did not explicitly remember words processed during sleep and failed to distinguish them from new words (old/new recognition test). However, both behavioral and EEG data indicate the presence of an implicit memory trace for words presented during sleep. In addition, the underlying neural signature of such implicit memories markedly differed from the explicit memories formed during wakefulness, in line with dual-process accounts arguing for two independent systems for explicit and implicit memory. Thus, our results reveal that implicit learning mechanisms can be triggered during sleep and provide a novel approach to explore the neural implementation of memory without awareness.