Action-Based Learning of Multistate Objects in the Medial Temporal Lobe

The precision of hippocampal representations predicts incremental value-learning across the adult lifespan

Correctly assigning value to different options and leveraging this information to guide choice is a cornerstone of adaptive decision-making. Reinforcement learning (RL) has provided a computational framework to study this process, and neural signals linked to RL have been identified in the striatum and medial prefrontal cortex. More recently, hippocampal contributions to this kind of value-learning have been proposed, at least under some conditions. Here, we test whether the hippocampus provides a signal of the option's identity that aids in credit assignment when learning about several perceptually similar items, and evaluate how this process differs across the lifespan. A sample of 251 younger and older adults, including a subset (n = 76) with simultaneous fMRI, completed an RL task in which they learned the value of four houses through trial-and-error. Older adults showed decreased choice accuracy, accompanied by reduced neural signaling of value at choice but not feedback. Using representational similarity analysis, we found that the precision with which choice options were represented in the posterior hippocampus during choice predicted accurate decisions across age groups. Interestingly, despite previous evidence for neural de-differentiation in older adults, we found no support for a "blurring" of these stimulus representations in older adults. Rather, we observed reduced connectivity between the posterior hippocampus and the medial PFC in older adults, and this connectivity correlated with choice consistency. Taken together, these findings identify a hippocampal contribution to incremental value learning, and that reductions in incremental value learning in older adults are associated with the reduced transfer of information between the hippocampus and mPFC, rather than the precision of the information in the hippocampus itself.

Developmental Differences in a Hippocampal-Cingulate Pathway Involved in Learned Safety Following Interpersonal Trauma Exposure

OBJECTIVE

Nearly 65% of youth experience trauma, and up to one-third of youth with trauma exposure face profound mental health sequelae. There remains a need to elucidate factors that contribute to psychopathology following trauma exposure, and to optimize interventions for youth who do not benefit sufficiently from existing treatments. Here, we probe safety signal learning (SSL), which is a mechanism of fear reduction that leverages learned safety to inhibit fear in the presence of threat-associated stimuli and has been shown to attenuate fear via a hippocampal-cingulate--specifically, a dorsal anterior cingulate cortex (dACC)--pathway.

METHOD

The present study used behavioral and task-based functional magnetic resonance imaging data to examine age-related associations between interpersonal trauma exposure and the behavioral and neural correlates (ie, activation and functional connectivity) of SSL in a group of 102 youth (aged 9-19 years; 46 female, 56 male) with (n = 52) and without (n = 50) interpersonal trauma exposure. Primary analyses examined anterior hippocampal activation and anterior hippocampus-dACC functional connectivity. Exploratory analyses examined centromedial amygdala (CMA) and laterobasal amygdala (LBA) activation and anterior hippocampal, CMA, and LBA functional connectivity with additional anterior cingulate subregions (ie, subgenual anterior cingulate cortex [sgACC] and rostral anterior cingulate cortex [rosACC]).

RESULTS

Both youth with and without interpersonal trauma exposure successfully learned conditioned safety, which was determined by using self-report of contingency awareness. Youth with interpersonal trauma exposure (relative to youth in the comparison group) exhibited age-specific patterns of lower hippocampal activation (F2,96 = 3.75, pFDR = .049, ηp2 = 0.072), and, in exploratory analyses, showed heightened centromedial amygdala activation (F1,96 = 5.37, pFDR = .046, ηp2 = 0.053) and an age-related decrease in hippocampal-sgACC functional connectivity during SSL (F1,94 = 10.68, pFDR = .015, ηp2 = 0.102). We also show that hippocampal-sgACC functional connectivity mediated the association between interpersonal trauma exposure and post-traumatic stress disorder symptoms in an age-specific manner in the overall sample.

CONCLUSION

Together, these findings suggest that although age- and trauma-specific differences in the neural correlates of SSL may relate to the development of psychopathology, youth with interpersonal trauma exposure demonstrate successful learning of conditioned safety over time.

DIVERSITY & INCLUSION STATEMENT

We worked to ensure that the study questionnaires were prepared in an inclusive way. We worked to ensure sex and gender balance in the recruitment of human participants. We worked to ensure race, ethnic, and/or other types of diversity in the recruitment of human participants. While citing references scientifically relevant for this work, we also actively worked to promote sex and gender balance in our reference list. While citing references scientifically relevant for this work, we also actively worked to promote inclusion of historically underrepresented racial and/or ethnic groups in science in our reference list. We actively worked to promote sex and gender balance in our author group. We actively worked to promote inclusion of historically underrepresented racial and/or ethnic groups in science in our author group. One or more of the authors of this paper self-identifies as a member of one or more historically underrepresented racial and/or ethnic groups in science. One or more of the authors of this paper received support from a program designed to increase minority representation in science.

Hippocampal Involvement in Safety Signal Learning Varies With Anxiety Among Healthy Adults

Background

Safety signal learning (SSL), based on conditioned inhibition of fear in the presence of learned safety, can effectively attenuate threat responses in animal models and humans. Difficulty regulating threat responses is a core feature of anxiety disorders, suggesting that SSL may provide a novel mechanism for fear reduction. Cross-species evidence suggests that SSL involves functional connectivity between the anterior hippocampus and the dorsal anterior cingulate cortex. However, the neural mechanisms supporting SSL have not been examined in relation to trait anxiety or while controlling for the effect of novelty.

Methods

Here, we investigated the neural mechanisms involved in SSL and associations with trait anxiety in a sample of 64 healthy (non-clinically anxious) adults (ages 18-30 years; 43 female, 21 male) using physiological, behavioral, and neuroimaging (functional magnetic resonance imaging) data collected during an SSL task.

Results

During SSL, compared with individuals with lower trait anxiety, individuals with higher trait anxiety showed less fear reduction as well as altered hippocampal activation and hippocampal-dorsal anterior cingulate cortex functional connectivity, and lower inferior frontal gyrus and ventrolateral prefrontal cortex activation. Importantly, the findings show that SSL reduces threat responding, across learning and over and above the effect of novelty, and involves hippocampal activation.

Conclusions

These findings provide new insights into the nature of SSL and suggest that there may be meaningful variation in SSL and related neural correlates as a function of trait anxiety, with implications for better understanding fear reduction and optimizing interventions for individuals with anxiety disorders.

Neural circuitry involved in conditioned inhibition via safety signal learning is sensitive to trauma exposure

Exposure to trauma throughout the lifespan is prevalent and increases the likelihood for the development of mental health conditions such as anxiety and post-traumatic stress disorder (PTSD). Safety signal learning (SSL)--a form of conditioned inhibition that involves reducing fear via conditioned safety--has been shown to effectively attenuate fear responses among individuals with trauma exposure, but the association between trauma exposure and the neural mechanisms of SSL remains unknown. Adults with varied prior exposure to trauma completed a conditioned inhibition task during functional MRI scanning and collection of skin conductance response (SCR). Conditioned safety signals reduced psychophysiological reactivity (i.e., SCR) in the overall sample. Although exposure to a higher number of traumatic events was associated with elevated SCR across all task conditions, SCR did not differ between threat in the presence of conditioned safety (i.e., SSL) relative to threat alone in a trauma-related manner. At the neural level, however, higher levels of trauma exposure were associated with lower hippocampal, amygdala, and dorsolateral prefrontal cortical activation during SSL. These findings suggest that while conditioned safety signals can reduce fear in the presence of threat even among individuals exposed to higher degrees of trauma, the neural circuitry involved in SSL is in fact sensitive to trauma exposure. Future research investigating neural processes during SSL among individuals with PTSD or anxiety can further elucidate the ways in which SSL and its neural correlates may reduce fear and link trauma exposure with later mental health conditions.

Comparing neural correlates of conditioned inhibition between children with and without anxiety disorders - A preliminary study

Cognitive-behavioral therapy (CBT), a first-line treatment for pediatric anxiety disorders, is based on principles of threat learning and extinction. However, CBT does not work sufficiently for up to 40% of clinically anxious youth. The neural and behavioral correlates of conditioned inhibition might provide promising targets for attempts to improve CBT response. During conditioned inhibition, threat and safety cues appear together, forming a safety compound. Here, we test whether this safety compound elicits a reduced fear response compared to pairing the threat cue with a novel cue (novel compound). The current pilot study compares behavioral, physiological, and neural correlates of conditioned inhibition between children with (n = 17, Mage = 13.09, SDage = 3.05) and without (n = 18, Mage = 14.49, SDage = 2.38) anxiety disorders. Behavioral and physiological measures did not differ between children with and without anxiety disorders during fear acquisition. During testing, children with anxiety disorders showed overall higher skin conductance response and expected to hear the aversive sound following the novel compound more often than children without anxiety disorders. Children with anxiety disorders showed more activity in the right ventromedial prefrontal cortex (vmPFC) to the safety versus novel compound. Children without anxiety disorders showed the opposite pattern - more right vmPFC activity to the novel versus safety compound (F(1,31) = 5.40, p = 0.03). No group differences manifested within the amygdala, dorsal anterior cingulate cortex, or hippocampus. These pilot findings suggest a feasible approach for examining conditioned inhibition in pediatric anxiety disorders. If replicated in larger samples, findings may implicate perturbed conditioned inhibition in pediatric anxiety disorders and provide targets for CBT.

Early childhood stress is associated with blunted development of ventral tegmental area functional connectivity

Early life stress increases risk for later psychopathology, due in part to changes in dopaminergic brain systems that support reward processing and motivation. Work in animals has shown that early life stress has a profound impact on the ventral tegmental area (VTA), which provides dopamine to regions including nucleus accumbens (NAcc), anterior hippocampus, and medial prefrontal cortex (mPFC), with cascading effects over the course of development. However, little is known about how early stress exposure shifts the developmental trajectory of mesocorticolimbic circuitry in humans. In the current study, 88 four- to nine-year-old children participated in resting-state fMRI. Parents completed questionnaires on their children's chronic stress exposure, including socioeconomic status (SES) and adverse childhood experiences (ACEs). We found an age x SES interaction on VTA connectivity, such that children from higher SES backgrounds showed a positive relationship between age and VTA-mPFC connectivity. Similarly, we found an age x ACEs exposure interaction on VTA connectivity, such that children with no ACEs exposure showed a positive relationship between age and VTA-mPFC connectivity. Our findings suggest that early stress exposure relates to the blunted maturation of VTA connectivity in young children, which may lead to disrupted reward processing later in childhood and beyond.

Distinct Neural Circuits Underlie Prospective and Concurrent Memory-Guided Behavior

The past is the best predictor of the future. This simple postulate belies the complex neurobiological mechanisms that facilitate an individual's use of memory to guide decisions. Previous research has shown integration of memories bias decision-making. Alternatively, memories can prospectively guide our choices. Here, we elucidate the mechanisms and timing of hippocampal (HPC), medial prefrontal cortex (mPFC), and striatal contributions during prospective memory-guided decision-making. We develop an associative learning task in which the correct choice is conditional on the preceding stimulus. Two distinct networks emerge: (1) a prospective circuit consisting of the HPC, putamen, mPFC, and other cortical regions, which exhibit increased activation preceding successful conditional decisions and (2) a concurrent circuit comprising the caudate, dorsolateral prefrontal cortex (dlPFC), and additional cortical structures that engage during the execution of correct conditional choices. Our findings demonstrate distinct neurobiological circuits through which memory prospectively biases decisions and influences choice execution.

Ventral hippocampus interacts with prelimbic cortex during inhibition of threat response via learned safety in both mice and humans

Heightened fear and inefficient safety learning are key features of fear and anxiety disorders. Evidence-based interventions for anxiety disorders, such as cognitive behavioral therapy, primarily rely on mechanisms of fear extinction. However, up to 50% of clinically anxious individuals do not respond to current evidence-based treatment, suggesting a critical need for new interventions based on alternative neurobiological pathways. Using parallel human and rodent conditioned inhibition paradigms alongside brain imaging methodologies, we investigated neural activity patterns in the ventral hippocampus in response to stimuli predictive of threat or safety and compound cues to test inhibition via safety in the presence of threat. Distinct hippocampal responses to threat, safety, and compound cues suggest that the ventral hippocampus is involved in conditioned inhibition in both mice and humans. Moreover, unique response patterns within target-differentiated subpopulations of ventral hippocampal neurons identify a circuit by which fear may be inhibited via safety. Specifically, ventral hippocampal neurons projecting to the prelimbic cortex, but not to the infralimbic cortex or basolateral amygdala, were more active to safety and compound cues than threat cues, and activity correlated with freezing behavior in rodents. A corresponding distinction was observed in humans: hippocampal-dorsal anterior cingulate cortex functional connectivity-but not hippocampal-anterior ventromedial prefrontal cortex or hippocampal-basolateral amygdala connectivity-differentiated between threat, safety, and compound conditions. These findings highlight the potential to enhance treatment for anxiety disorders by targeting an alternative neural mechanism through safety signal learning.

Content-based Dissociation of Hippocampal Involvement in Prediction

Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top-down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli.

Hippocampal-neocortical interactions sharpen over time for predictive actions

When an action is familiar, we are able to anticipate how it will change the state of the world. These expectations can result from retrieval of action-outcome associations in the hippocampus and the reinstatement of anticipated outcomes in visual cortex. How does this role for the hippocampus in action-based prediction change over time? We use high-resolution fMRI and a dual-training behavioral paradigm to examine how the hippocampus interacts with visual cortex during predictive and nonpredictive actions learned either three days earlier or immediately before the scan. Just-learned associations led to comparable background connectivity between the hippocampus and V1/V2, regardless of whether actions predicted outcomes. However, three-day-old associations led to stronger background connectivity and greater differentiation between neural patterns for predictive vs. nonpredictive actions. Hippocampal prediction may initially reflect indiscriminate binding of co-occurring events, with action information pruning weaker associations and leading to more selective and accurate predictions over time.

In familiar environments, humans automatically anticipate the sensory consequences of their motor actions. Here, the authors show how action-based predictions arise from interactions between the hippocampus and visual cortex, and how these interactions strengthen and weaken over time.

Human hippocampal replay during rest prioritizes weakly learned information and predicts memory performance

The hippocampus replays experiences during quiet rest periods, and this replay benefits subsequent memory. A critical open question is how memories are prioritized for this replay. We used functional magnetic resonance imaging (fMRI) pattern analysis to track item-level replay in the hippocampus during an awake rest period after participants studied 15 objects and completed a memory test. Objects that were remembered less well were replayed more during the subsequent rest period, suggesting a prioritization process in which weaker memories—memories most vulnerable to forgetting—are selected for replay. In a second session 12 hours later, more replay of an object during a rest period predicted better subsequent memory for that object. Replay predicted memory improvement across sessions only for participants who slept during that interval. Our results provide evidence that replay in the human hippocampus prioritizes weakly learned information, predicts subsequent memory performance, and relates to memory improvement across a delay with sleep.

The hippocampus is known to 'replay' experiences and memories during rest periods, but it is unclear how particular memories are prioritized for replay. Here, the authors show that information that is remembered less well is replayed more often, suggesting that weaker memories are selected for replay.

General Transformations of Object Representations in Human Visual Cortex

The brain actively represents incoming information, but these representations are only useful to the extent that they flexibly reflect changes in the environment. How does the brain transform representations across changes, such as in size or viewing angle? We conducted a fMRI experiment and a magnetoencephalography experiment in humans (both sexes) in which participants viewed objects before and after affine viewpoint changes (rotation, translation, enlargement). We used a novel approach, representational transformation analysis, to derive transformation functions that linked the distributed patterns of brain activity evoked by an object before and after an affine change. Crucially, transformations derived from one object could predict a postchange representation for novel objects. These results provide evidence of general operations in the brain that are distinct from neural representations evoked by particular objects and scenes.SIGNIFICANCE STATEMENT The dominant focus in cognitive neuroscience has been on how the brain represents information, but these representations are only useful to the extent that they flexibly reflect changes in the environment. How does the brain transform representations, such as linking two states of an object, for example, before and after an object undergoes a physical change? We used a novel method to derive transformations between the brain activity evoked by an object before and after an affine viewpoint change. We show that transformations derived from one object undergoing a change generalized to a novel object undergoing the same change. This result shows that there are general perceptual operations that transform object representations from one state to another.

When past is present: Substitutions of long-term memory for sensory evidence in perceptual judgments

When perception is underdetermined by current sensory inputs, memories for related experiences in the past might fill in missing detail. To evaluate this possibility, we measured the likelihood of relying on long-term memory versus sensory evidence when judging the appearance of an object near the threshold of awareness. Specifically, we associated colors with shapes in long-term memory and then presented the shapes again later in unrelated colors and had observers judge the appearance of the new colors. We found that responses were well characterized as a bimodal mixture of original and current-color representations (vs. an integrated unimodal representation). That is, although irrelevant to judgments of the current color, observers occasionally anchored their responses on the original colors in memory. Moreover, the likelihood of such memory substitutions increased when sensory input was degraded. In fact, they occurred even in the absence of sensory input when observers falsely reported having seen something. Thus, although perceptual judgments intuitively seem to reflect the current state of the environment, they can also unknowingly be dictated by past experiences.

Neural pattern change during encoding of a narrative predicts retrospective duration estimates

What mechanisms support our ability to estimate durations on the order of minutes? Behavioral studies in humans have shown that changes in contextual features lead to overestimation of past durations. Based on evidence that the medial temporal lobes and prefrontal cortex represent contextual features, we related the degree of fMRI pattern change in these regions with people's subsequent duration estimates. After listening to a radio story in the scanner, participants were asked how much time had elapsed between pairs of clips from the story. Our ROI analyses found that duration estimates were correlated with the neural pattern distance between two clips at encoding in the right entorhinal cortex. Moreover, whole-brain searchlight analyses revealed a cluster spanning the right anterior temporal lobe. Our findings provide convergent support for the hypothesis that retrospective time judgments are driven by 'drift' in contextual representations supported by these regions.

Implicit short- and long-term memory direct our gaze in visual search

Visual attention is strongly affected by the past: both by recent experience and by long-term regularities in the environment that are encoded in and retrieved from memory. In visual search, intertrial repetition of targets causes speeded response times (short-term priming). Similarly, targets that are presented more often than others may facilitate search, even long after it is no longer present (long-term priming). In this study, we investigate whether such short-term priming and long-term priming depend on dissociable mechanisms. By recording eye movements while participants searched for one of two conjunction targets, we explored at what stages of visual search different forms of priming manifest. We found both long- and short- term priming effects. Long-term priming persisted long after the bias was present, and was again found even in participants who were unaware of a color bias. Short- and long-term priming affected the same stage of the task; both biased eye movements towards targets with the primed color, already starting with the first eye movement. Neither form of priming affected the response phase of a trial, but response repetition did. The results strongly suggest that both long- and short-term memory can implicitly modulate feedforward visual processing.

Statistical learning of temporal community structure in the hippocampus

The hippocampus is involved in the learning and representation of temporal statistics, but little is understood about the kinds of statistics it can uncover. Prior studies have tested various forms of structure that can be learned by tracking the strength of transition probabilities between adjacent items in a sequence. We test whether the hippocampus can learn higher-order structure using sequences that have no variance in transition probability and instead exhibit temporal community structure. We find that the hippocampus is indeed sensitive to this form of structure, as revealed by its representations, activity dynamics, and connectivity with other regions. These findings suggest that the hippocampus is a sophisticated learner of environmental regularities, able to uncover higher-order structure that requires sensitivity to overlapping associations.

Competition between Mutually Exclusive Object States in Event Comprehension

Successful language comprehension requires one to correctly match symbols in an utterance to referents in the world, but the rampant ambiguity present in that mapping poses a challenge. Sometimes the ambiguity lies in which of two (or more) types of things in the world are under discussion (i.e., lexical ambiguity); however, even a word with a single sense can have an ambiguous referent. This ambiguity occurs when an object can exist in multiple states. Here, we consider two cases in which the presence of multiple object states may render a single-sense word ambiguous. In the first case, one must disambiguate between two states of a single object token in a short discourse. In the second case, the discourse establishes two different tokens of the object category. Both cases involve multiple object states: These states are mutually exclusive in the first case, whereas in the second case, these states can logically exist at the same time. We use fMRI to contrast same-token and different-token discourses, using responses in left posterior ventrolateral prefrontal cortex (pVLPFC) as an indicator of conflict. Because the left pVLPFC is sensitive to competition between multiple, incompatible representations, we predicted that state ambiguity should engender conflict only when those states are mutually exclusive. Indeed, we find evidence of conflict in same-token, but not different-token, discourses. Our data support a theory of left pVLPFC function in which general conflict resolution mechanisms are engaged to select between multiple incompatible representations that arise in many kinds of ambiguity present in language.