Overlapping patterns of neural activity for different forms of novelty in fMRI

Simple synaptic modulations implement diverse novelty computations

Detecting novelty is ethologically useful for an organism's survival. Recent experiments characterize how different types of novelty over timescales from seconds to weeks are reflected in the activity of excitatory and inhibitory neuron types. Here, we introduce a learning mechanism, familiarity-modulated synapses (FMSs), consisting of multiplicative modulations dependent on presynaptic or pre/postsynaptic neuron activity. With FMSs, network responses that encode novelty emerge under unsupervised continual learning and minimal connectivity constraints. Implementing FMSs within an experimentally constrained model of a visual cortical circuit, we demonstrate the generalizability of FMSs by simultaneously fitting absolute, contextual, and omission novelty effects. Our model also reproduces functional diversity within cell subpopulations, leading to experimentally testable predictions about connectivity and synaptic dynamics that can produce both population-level novelty responses and heterogeneous individual neuron signals. Altogether, our findings demonstrate how simple plasticity mechanisms within a cortical circuit structure can produce qualitatively distinct and complex novelty responses.

Experiencing Surprise: The Temporal Dynamics of Its Impact on Memory

To efficiently process information, the brain shifts between encoding and retrieval states, prioritizing bottom-up or top-down processing accordingly. Expectation violation before or during learning has been shown to trigger an adaptive encoding mechanism, resulting in better memory for unexpected events. Using fMRI, we explored (1) whether this encoding mechanism is also triggered during retrieval, and if so, (2) what the temporal dynamics of its mnemonic consequences are. Male and female participants studied object images, then, with new objects, they learned a contingency between a cue and a semantic category. Rule-abiding (expected) and violating (unexpected) targets and similar foils were used at test. We found interactions between previous and current similar events' expectation, such that when an expected event followed a similar but unexpected event, its performance was boosted, underpinned by activation in the hippocampus, midbrain, and occipital cortex. In contrast, a sequence of two unexpected similar events also triggered occipital engagement; however, this did not enhance memory performance. Taken together, our findings suggest that when the goal is to retrieve, encountering surprising events engages an encoding mechanism, supported by bottom-up processing, that may enhance memory for future related events.SIGNIFICANCE STATEMENT Optimizing the balance between new learning and the retrieval of existing knowledge is an ongoing process, at the core of human cognition. Previous research into memory encoding suggests experiencing surprise leads to the prioritization of the learning of new memories, forming an adaptive encoding mechanism. We examined whether this mechanism is also engaged when the current goal is to retrieve information. Our results demonstrate that an expectation-driven shift toward an encoding state, supported by enhanced perceptual processing, is beneficial for the correct identification of subsequent expected similar events. These findings have important implications for our understanding of the temporal dynamics of the adaptive encoding of information into memory.

Examining the transition of novel information toward familiarity

Throughout their lives, humans encounter multiple instances of new information that can be inconsistent with prior knowledge (novel). Over time, the once-novel information becomes integrated into their established knowledge base, shifting from novelty to familiarity. In this study, we investigated the processes by which the first steps of this transition take place. We hypothesized that the neural representations of initially novel items gradually change over the course of repeated presentations, expressing a shift toward familiarity. We further assumed that this shift could be traced by examining neural patterns using fMRI. In two experiments, while being scanned, participants read noun-adjective word pairs that were either consistent or inconsistent with their prior knowledge. Stimuli were repeated 3-6 times within the scans. Employing mass univariate and multivariate similarity analyses, we showed that the neural representations associated with the initial presentation of familiar versus novel objects differed in lateral frontal and temporal regions, the medial prefrontal cortex, and the medial temporal lobe. Importantly, the neural representations of novel stimuli gradually changed throughout repetitions until they became indistinguishable from their respective familiar items. We interpret these findings as indicating that an early phase of familiarization can be completed within a few repetitions. This initial familiarization can then serve as the prerequisite to the integration of novel items into existing knowledge. Future empirical and theoretical works can build on the current findings to develop a comprehensive model of the transition from novelty to familiarity.

VTA and Anterior Hippocampus Target Dissociable Neocortical Networks for Post-Novelty Enhancements

The detection of novelty indicates changes in the environment and the need to update existing representations. In response to novelty, interactions across the VTA-hippocampal circuit support experience-dependent plasticity in the hippocampus. While theories have broadly suggested plasticity-related changes are also instantiated in the cortex, research has also shown evidence for functional heterogeneity in cortical networks. It therefore remains unclear how the hippocampal-VTA circuit engages cortical networks, and whether novelty targets specific cortical regions or diffuse, large-scale cortical networks. To adjudicate the role of the VTA and hippocampus in cortical network plasticity, we used fMRI to compare resting-state functional coupling before and following exposure to novel scene images in human subjects of both sexes. Functional coupling between right anterior hippocampus and VTA was enhanced following novelty exposure. However, we also found evidence for a double dissociation, with anterior hippocampus and VTA showing distinct patterns of post-novelty functional coupling enhancements, targeting task-relevant regions versus large-scale networks, respectively. Further, significant correlations between these networks and the novelty-related plasticity in the anterior hippocampal-VTA functional network suggest that the central hippocampal-VTA network may facilitate the interactions with the cortex. These findings support an extended model of novelty-induced plasticity, in which novelty elicits plasticity-related changes in both local and global cortical networks.SIGNIFICANCE STATEMENT Novelty detection is critical for adaptive behavior, signaling the need to update existing representations. By engaging the bidirectional hippocampal-VTA circuit, novelty has been shown to induce plasticity-related changes in the hippocampus. However, it remains an open question how novelty targets such plasticity-related changes in cortical networks. We show that anterior hippocampus and VTA target cortical networks at different spatial scales, with respective enhancements in post-novelty functional coupling with a task-relevant cortical region and a large-scale memory network. The results presented here support an extended model of novelty-related plasticity, in which engaging the anterior hippocampal-VTA circuit through novelty exposure propagates cortical plasticity through hippocampal and VTA functional pathways at distinct scales, targeting specific or diffuse cortical networks.

Training-Induced Changes in Radial-Tangential Anisotropy of Visual Crowding

Purpose

One of the diagnostic features of visual crowding, radial–tangential anisotropy, has been observed both in behavioral experiments as well as in responses of the blood-oxygenation-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) signal. As has been shown previously, crowding is stronger for radially arranged flankers, and this tendency is reflected in BOLD signal suppression. In the current study, we examined the effect of practice on the neural correlates of crowding. We expected that training on a crowding task would cause shrinkage of the crowding zone that would be mirrored in corresponding BOLD signal responses.

Methods

Pre- and post-training fMRI images were acquired in 17 healthy volunteers using a 3-tesla MRI scanner. Participants were trained over 4 consecutive days on a crowding task.

Results

Comparison of the pre- and post-training behavioral data indicates a significant shrinkage of the crowding zone as a result of training. Moreover, we observed a pronounced radial–tangential anisotropy in the BOLD signal prior to training; that is, radial flankers induced a larger reduction in the BOLD signal compared to equally spaced tangential flankers. After training, this radial–tangential anisotropy in the BOLD signal was significantly reduced. Specifically, we found significant changes in BOLD responses for the radial flanker configuration.

Conclusions

Our results demonstrate that training-induced changes in the anisotropic shape of the crowding zone are reflected in the BOLD signal in the early visual cortex.

Translational Relevance

Perceptual learning tasks may have the potential to improve visual performance by promoting neural plasticity. Our results could motivate the development of suitable rehabilitation protocols for patients with central vision loss.

Unravelling the effects of methylphenidate on the dopaminergic and noradrenergic functional circuits

Functional magnetic resonance imaging (fMRI) can be combined with drugs to investigate the system-level functional responses in the brain to such challenges. However, most psychoactive agents act on multiple neurotransmitters, limiting the ability of fMRI to identify functional effects related to actions on discrete pharmacological targets. We recently introduced a multimodal approach, REACT (Receptor-Enriched Analysis of functional Connectivity by Targets), which offers the opportunity to disentangle effects of drugs on different neurotransmitters and clarify the biological mechanisms driving clinical efficacy and side effects of a compound. Here, we focus on methylphenidate (MPH), which binds to the dopamine transporter (DAT) and the norepinephrine transporter (NET), to unravel its effects on dopaminergic and noradrenergic functional circuits in the healthy brain at rest. We then explored the relationship between these target-enriched resting state functional connectivity (FC) maps and inter-individual variability in behavioural responses to a reinforcement-learning task encompassing a novelty manipulation to disentangle the molecular systems underlying specific cognitive/behavioural effects. Our main analysis showed a significant MPH-induced FC increase in sensorimotor areas in the functional circuit associated with DAT. In our exploratory analysis, we found that MPH-induced regional variations in the DAT and NET-enriched FC maps were significantly correlated with some of the inter-individual differences on key behavioural responses associated with the reinforcement-learning task. Our findings show that main MPH-related FC changes at rest can be understood through the distribution of DAT in the brain. Furthermore, they suggest that when compounds have mixed pharmacological profiles, REACT may be able to capture regional functional effects that are underpinned by the same cognitive mechanism but are related to engagement of distinct molecular targets.

Imaging object-scene relations processing in visible and invisible natural scenes

Integrating objects with their context is a key step in interpreting complex visual scenes. Here, we used functional Magnetic Resonance Imaging (fMRI) while participants viewed visual scenes depicting a person performing an action with an object that was either congruent or incongruent with the scene. Univariate and multivariate analyses revealed different activity for congruent vs. incongruent scenes in the lateral occipital complex, inferior temporal cortex, parahippocampal cortex, and prefrontal cortex. Importantly, and in contrast to previous studies, these activations could not be explained by task-induced conflict. A secondary goal of this study was to examine whether processing of object-context relations could occur in the absence of awareness. We found no evidence for brain activity differentiating between congruent and incongruent invisible masked scenes, which might reflect a genuine lack of activation, or stem from the limitations of our study. Overall, our results provide novel support for the roles of parahippocampal cortex and frontal areas in conscious processing of object-context relations, which cannot be explained by either low-level differences or task demands. Yet they further suggest that brain activity is decreased by visual masking to the point of becoming undetectable with our fMRI protocol.

How do memory systems detect and respond to novelty?

The efficiency of the memory system lies not only in its readiness to detect and retrieve old stimuli but also in its ability to detect and integrate novel information. In this review, we discuss recent evidence suggesting that the neural substrates sensitive to detecting familiarity and novelty are not entirely overlapping. Instead, these partially distinct familiarity and novelty signals are integrated to support recognition memory decisions. We propose here that the mediodorsal thalamus is critical for familiarity detection, and for combining novelty signals from the medial temporal lobe cortex with the relative familiarity outputs of computations performed in other cortical structures, especially the prefrontal cortex. Importantly, we argue that the anterior hippocampus has a prominent role in detecting novelty and in communicating this with midbrain and striatal structures. We argue that different types of novelty (absolute or contextual) engage different neurotransmitter systems that converge in the hippocampus. We suggest that contextual or unexpected novelty triggers dopaminergic hippocampal-midbrain coupling and noradrenergic-mediated pupil dilation. In contrast, absolute novelty triggers cholinergic-mediated hippocampal encoding accompanied by diminished pupil dilation. These two, distinct hippocampal encoding mechanisms both lead to later recollection but are sensitive to different types of novelty. We conclude that this neurotransmitter-mediated hippocampal encoding establishes the hippocampus in an encoding mode that briefly prevents the engagement of retrieval.

Retrieval Demands Adaptively Change Striatal Old/New Signals and Boost Subsequent Long-Term Memory

The striatum is a central part of the dopaminergic mesolimbic system and contributes both to the encoding and retrieval of long-term memories. In this regard, the co-occurrence of striatal novelty and retrieval success effects in independent studies underlines the structure's double duty and suggests dynamic contextual adaptation. To test this hypothesis and further investigate the underlying mechanisms of encoding and retrieval dynamics, human subjects viewed pre-familiarized scene images intermixed with new scenes and classified them as indoor versus outdoor (encoding task) or old versus new (retrieval task), while fMRI and eye tracking data were recorded. Subsequently, subjects performed a final recognition task. As hypothesized, striatal activity and pupil size reflected task-conditional salience of old and new stimuli, but, unexpectedly, this effect was not reflected in the substantia nigra and ventral tegmental area (SN/VTA), medial temporal lobe, or subsequent memory performance. Instead, subsequent memory generally benefitted from retrieval, an effect possibly driven by task difficulty and activity in a network including different parts of the striatum and SN/VTA. Our findings extend memory models of encoding and retrieval dynamics by pinpointing a specific contextual factor that differentially modulates the functional properties of the mesolimbic system.SIGNIFICANCE STATEMENT The mesolimbic system is involved in the encoding and retrieval of information but it is unclear how these two processes are achieved within the same network of brain regions. In particular, memory retrieval and novelty encoding were considered in independent studies, implying that novelty (new > old) and retrieval success (old > new) effects may co-occur in the striatum. Here, we used a common framework implicating the striatum, but not other parts of the mesolimbic system, in tracking context-dependent salience of old and new information. The current study, therefore, paves the way for a more comprehensive understanding of the functional properties of the mesolimbic system during memory encoding and retrieval.

Learning to name smells increases activity in heteromodal semantic areas

Semantic description of odors is a cognitively demanding task. Learning to name smells is, however, possible with training. This study set out to examine how improvement in olfactory semantic knowledge following training reorganizes the neural representation of smells. First, 19 nonexpert volunteers were trained for 3 days; they were exposed (i) to odorants presented without verbal labels (perceptual learning) and (ii) to other odorants paired with lexicosemantic labels (associative learning). Second, the same participants were tested in a brain imaging study (fMRI) measuring hemodynamic responses to learned odors presented in both the perceptual and associative learning conditions. The lexicosemantic training enhanced the ability to describe smells semantically. Neurally, this change was associated with enhanced activity in a set of heteromodal areas-including superior frontal gyrus-and parietal areas. These findings demonstrate that odor-name associative learning induces recruitment of brain areas involved in the integration and representation of semantic attributes of sensory events. They also offer new insights into the brain plasticity underlying the acquisition of olfactory expertise in lay people. Hum Brain Mapp 38:5958-5969, 2017. © 2017 Wiley Periodicals, Inc.

Differing Time of Onset of Concurrent TMS-fMRI during Associative Memory Encoding: A Measure of Dynamic Connectivity

There has been a distinct shift in neuroimaging from localization of function into a more network based approach focused on connectivity. While fMRI has proven very fruitful for this, the hemodynamic signal is inherently slow which limits the temporal resolution of fMRI-only connectivity measures. The brain, however, works on a time scale of milliseconds. This study utilized concurrent transcranial magnetic stimulation (TMS)-fMRI in a novel way to obtain measures of dynamic connectivity by measuring changes in fMRI signal amplitude in regions distal to the site of stimulation following differing TMS onset times. Seventeen healthy subjects completed an associative memory encoding task known to involve the DLPFC, viewing pairs of objects which could be semantically related or unrelated. Three pulses of 10 Hz repetitive TMS were applied over the left DLPFC starting either at 200, 600, or 1000 ms after stimulus onset. Associations for related pairs were better remembered than unrelated pairs in a post-scan cued recall test. Differences in neural activity were assessed across different TMS onsets, separately for related and unrelated pairs. Time specific TMS effects were observed in several regions, including those associated with higher-level processing (lateral frontal, anterior cingulate), visual areas (occipital), and regions involved in semantic processing (e.g., left mid-temporal and medial frontal). Activity in the frontal cortex was decreased at 200 ms post-stimulus for unrelated pairs, and 1000 ms post-stimulus for related pairs. This suggests differences in the timing across conditions in which the DLFPC interacts with other PFC regions, consistent with the notion that the DLPFC is facilitating extended semantic processing for related items. This study demonstrates that time-varying TMS onset inside the MRI can be used to reliably measure fast dynamic connectivity with a temporal resolution in the hundreds of milliseconds.

Distinctiveness Benefits Novelty (and Not Familiarity), but Only Up to a Limit: The Prior Knowledge Perspective

Novelty is a pivotal player in cognition, and its contribution to superior memory performance is a widely accepted convention. On the other hand, mnemonic advantages for familiar information are also well documented. Here, we examine the role of experimental distinctiveness as a potential explanation for these apparently conflicting findings. Across two experiments, we demonstrate that conceptual novelty, an unfamiliar combination of familiar constituents, is sensitive to its experimental proportions: Improved memory for novelty was observed when novel stimuli were relatively rare. Memory levels for familiar items, in contrast, were completely unaffected by experimental proportions, highlighting their insensitivity to list-based distinctiveness. Finally, no mnemonic advantage for conceptual novelty over familiarity was observed even when novel stimuli were extremely rare at study. Together, these results imply that novel and familiar items are processed via partially distinct mechanisms, with (at least some facets of) novelty not providing a mnemonic advantage over familiarity.

Distinct Neural Suppression and Encoding Effects for Conceptual Novelty and Familiarity

Like yin and yang, novelty and familiarity are typically described as separate-yet-complementary aspects of an experience, two ends of a single continuum. However, novelty and familiarity are also multifaceted. For instance, novelty can sometimes result in enhanced mnemonic performance, whereas at other times familiarity is better remembered. As previous investigations focused primarily on the experimental aspect of novelty, the mechanisms supporting conceptual novelty (the novel combination of two previously unrelated existing concepts) remain unclear. Importantly, conceptual novelty can be recognized as such only when compared with preexperimental familiar knowledge, regardless of experimental status. Here we applied a combined repetition suppression/subsequent memory fMRI paradigm, focusing on the conceptual aspect of novelty and familiarity as the subject matter. Conceptual novelty was characterized by sustained neural activity; familiarity, on the other hand, exhibited repetition effects in multiple cortical regions, a subset of which was modulated by successful encoding. Subsequent memory of novelty was associated only with activation differences in a distinct set of regions, including the hippocampus and medial cortical regions. These results suggest that conceptual novelty (a) does not (easily) trigger the repetition suppression phenomenon but requires sustained neural recruitment and (b) activates dedicated encoding mechanisms. Conceptual familiarity, in contrast, allows rapid neural processing that depends upon existing neural representations. Overall, these findings challenge the definition of novelty as a unitary concept. Furthermore, they bear important implications for research into the neural bases of knowledge representation and recognition memory.