Dissociating medial temporal and basal ganglia memory systems with a latent learning task

Humans learn generalizable representations through efficient coding

Reinforcement learning theory explains human behavior as driven by the goal of maximizing reward. Conventional approaches, however, offer limited insights into how people generalize from past experiences to new situations. Here, we propose refining the classical reinforcement learning framework by incorporating an efficient coding principle, which emphasizes maximizing reward using the simplest necessary representations. This refined framework predicts that intelligent agents, constrained by simpler representations, will inevitably: 1) distill environmental stimuli into fewer, abstract internal states, and 2) detect and utilize rewarding environmental features. Consequently, complex stimuli are mapped to compact representations, forming the foundation for generalization. We tested this idea in two experiments that examined human generalization. Our findings reveal that while conventional models fall short in generalization, models incorporating efficient coding achieve human-level performance. We argue that the classical RL objective, augmented with efficient coding, represents a more comprehensive computational framework for understanding human behavior in both learning and generalization.

Pay attention and you might miss it: Greater learning during attentional lapses

Attentional lapses have been found to impair everything from basic perception to learning and memory. Yet, despite the well-documented costs of lapses on cognition, recent work suggests that lapses might unexpectedly confer some benefits. One potential benefit is that lapses broaden our learning to integrate seemingly irrelevant content that could later prove useful-a benefit that prior research focusing only on goal-relevant memory would miss. Here, we measure how fluctuations in sustained attention influence the learning of seemingly goal-irrelevant content that competes for attention with target content. Participants completed a correlated flanker task in which they categorized central targets (letters or numbers) while ignoring peripheral flanking symbols that shared hidden probabilistic relationships with the targets. We found that across participants, higher rates of attentional lapses correlated with greater learning of the target-flanker relationships. Moreover, within participants, learning was more evident during attentional lapses. These findings address long-standing theoretical debates and reveal a benefit of attentional lapses: they expand the scope of learning and decisions beyond the strictly relevant.

Visually guided equivalence learning in borderline personality disorder

The hallmark symptoms of borderline personality disorder are maladaptive behavior and impulsive emotional reactions. However, the condition is occasionally associated with cognitive alterations. Recently, it has been found that the function of the basal ganglia and the hippocampi might also be affected. Hence, deterioration in learning and memory processes associated with these structures is expected. Thus, we sought to investigate visually guided associative learning, a type of conditioning associated with the basal ganglia and the hippocampi, in patients suffering from borderline personality disorder. In this study, the modified Rutgers Acquired Equivalence Test was used to assess associative learning in 23 patients and age-, sex-, and educational level-matched controls. The acquisition phase of the test, which is associated primarily with the frontostriatal loops, was altered in patients with borderline personality disorder: the patients exhibited poor performance in terms of building associations. However, the retrieval and generalization functions, which are primarily associated with the hippocampi and the medial temporal lobes, were not affected. These results corroborate that the basal ganglia are affected in borderline personality disorder. However, maintained retrieval and generalization do not support the assumption that the hippocampi are affected too.

Corticosterone differentially modulates time-dependent fear generalization following mild or moderate fear conditioning training in rats

Stressful and emotionally arousing experiences create strong memories that seem to lose specificity over time. It is uncertain, however, how the stress system contributes to the phenomenon of time-dependent fear generalization. Here, we investigated whether post-training corticosterone (CORT-HBC) injections, given after different training intensities, affect contextual fear memory specificity at several time points. We trained male Wistar rats on the contextual fear conditioning (CFC) task using two footshock intensities (mild CFC, 3 footshocks of 0.3 mA, or moderate CFC, 3x 0.6 mA) and immediately after the training session we administered CORT-HBC systemically. We first tested the animals in a novel context and then in the training context at different intervals following training (2, 14, 28 or 42 days). By measuring freezing in the novel context and then contrasting freezing times shown in both contexts, we inferred contextual fear generalization for each rat, classifying them into Generalizers or Discriminators. Following mild CFC training, the glucocorticoid injection promoted an accurate contextual memory at the recent time point (2 days), and increase the contextual memory accuracy 28 days after training. In contrast, after the moderate CFC training, CORT-HBC facilitated contextual generalization at 14 days, compared to the control group that maintained contextual discrimination at this timepoint. For this training intensity, however, CORT-HBC did not have any effect on recent memory specificity. These findings indicate that treatment with CORT-HBC immediately after the encoding of mild or moderately arousing experiences may differentially modulate memory consolidation and time-dependent fear generalization.

Learning functions in short-term cocaine users

Objective

This study examined learning functions in short-term cocaine users and control participants.

Method

Seventeen active cocaine users (reporting 3.5 mean years of cocaine use) and seventeen non-cocaine-using controls (with similar reported levels of alcohol and marijuana use) were compared on tasks measuring different aspects of learning.

Results

The cocaine users performed more poorly on the Weather Prediction and List-Learning tasks, as well as supplementary executive and psychomotor function tasks, than controls.

Conclusions

Individuals with a relatively short duration of cocaine use exhibited moderate weaknesses in probabilistic category learning, verbal learning and psychomotor functions, relative to controls. These weaknesses may underpin difficulty in learning from the probabilistic consequences of behavior and hinder the ability to respond to cognitive-behavioral treatments.

•

Learning functions in short-term cocaine users have been understudied.

•

Short-term cocaine users performed worse than controls on probabilistic category and verbal learning tasks.

•

Short-term cocaine users performed similarly to controls on an equivalence learning task.

•

Clinicians should be aware that even short-term cocaine use may be associated with cognitive weaknesses.

Dynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning

Complex learned behaviors must involve the integrated action of distributed brain circuits. Although the contributions of individual regions to learning have been extensively investigated, much less is known about how distributed brain networks orchestrate their activity over the course of learning. To address this gap, we used fMRI combined with tools from dynamic network neuroscience to obtain time-resolved descriptions of network coordination during reinforcement learning in humans. We found that learning to associate visual cues with reward involves dynamic changes in network coupling between the striatum and distributed brain regions, including visual, orbitofrontal, and ventromedial prefrontal cortex (n = 22; 13 females). Moreover, we found that this flexibility in striatal network coupling correlates with participants' learning rate and inverse temperature, two parameters derived from reinforcement learning models. Finally, we found that episodic learning, measured separately in the same participants at the same time, was related to dynamic connectivity in distinct brain networks. These results suggest that dynamic changes in striatal-centered networks provide a mechanism for information integration during reinforcement learning.SIGNIFICANCE STATEMENT Learning from the outcomes of actions, referred to as reinforcement learning, is an essential part of life. The roles of individual brain regions in reinforcement learning have been well characterized in terms of updating values for actions or cues. Missing from this account, however, is an understanding of how different brain areas interact during learning to integrate sensory and value information. Here we characterize flexible striatal-cortical network dynamics that relate to reinforcement learning behavior.

Behavioral facilitation after hippocampal lesion: A review

When parts of the brain suffer from damage, certain functional deficits or impairments are the expected and typical outcome. A myriad of examples show such negative consequences, which afford the daily tasks of neurologists, neuropsychologists, and also behavioral neuroscientists working with experimental brain lesions. Compared to lesion-induced deficits, examples for functional enhancements or facilitation after brain lesions are rather rare and usually not well studied. Here, the mammalian hippocampus seems to provide an exception, since substantial evidence shows that its damage can have facilitatory behavioral effects under certain conditions. This review will address these effects and their possible mechanisms. It will show that facilitatory effects of hippocampal lesions, although mostly studied in rats, can be found in many mammalian species, that is, they are apparently not species-specific. Furthermore, they can be found with various lesion techniques, from tissue ablation, to neurotoxic damage, and from damage of hippocampal structure itself to damage of fiber systems innervating it. The major emphasis of this review, however, lies on the behavioral effects and their interpretations. Thus, facilitatory effects can be found in several learning paradigms, especially active avoidance, and some forms of Pavlovian and instrumental conditioning. These will be discussed in light of pertinent theories of hippocampal function, such as inhibition, spatial cognition, and multiple memory systems theories, which state that facilitatory effects of hippocampal lesions may reflect the loss of interference between hippocampal spatial and striatal procedural cognition. Using the example of the rat sequential reaction time task, it will also be discussed how such lesions can have direct and indirect consequences on certain behavioral readouts. A final note will advocate considering possible functional facilitation also in neurologic patients, especially those with hippocampal damage, since such a strategy might provide new avenues for therapeutic treatments.

In vivo measurements of limbic glutamate and GABA concentrations in epileptic patients during affective and cognitive tasks: A microdialysis study

Limbic system structures such as the amygdala (AMG) and the hippocampus (HIPP) are involved in affective and cognitive processing. However, because of the limitations in noninvasive technology, absolute concentrations of the neurotransmitters underlying limbic system engagement are not known. Here, we report changes in the concentrations of the neurotransmitters glutamate (Glu) and gamma-aminobutyric acid (GABA) in the HIPP and the AMG of patients with nonlesional temporal lobe epilepsy undergoing surgery for intracranial subdural and depth electrode implantation. We utilized an in-vivo microdialysis technique while subjects were engaged in cognitive tasks with or without emotional content. The performance of an emotion learning task (EmoLearn) was associated with a significant increase in the concentration of glutamate in the HIPP when images with high valence content were processed, as compared to its concentration while processing images with low valence. In addition, significantly decreased levels of glutamate were found in the AMG when images with predominantly low valence content were processed, as compared to its concentration at baseline. The processing of face stimuli with anger/fear content (FaceMatch task) was accompanied with significantly decreased concentrations of GABA in the AMG and HIPP compared to its levels at the baseline. The processing of shapes on the other hand was accompanied with a significantly decreased concentration of the glutamate in the AMG as well as in the HIPP compared to the baseline. Finally, the performance of a nondeclarative memory task (weather prediction task-WPT) was associated with relatively large and opposite changes in the GABA levels compared to the baseline in the AMG (decrease) and the HIPP (increase). These data are relevant for showing an involvement of the amygdala and the hippocampus in emotional processing and provide additional neurochemical clues towards a more refined model of the functional circuitry of the human limbic system.

Latent learning in End Stage Renal Disease (ESRD)

Cognitive functions such as attention and memory are known to be impaired in End Stage Renal Disease (ESRD), but the sites of the neural changes underlying these impairments are uncertain. Patients and controls took part in a latent learning task, which had previously shown a dissociation between patients with Parkinson's disease and those with medial temporal damage. ESRD patients (n=24) and age and education-matched controls (n=24) were randomly assigned to either an exposed or unexposed condition. In Phase 1 of the task, participants learned that a cue (word) on the back of a schematic head predicted that the subsequently seen face would be smiling. For the exposed (but not unexposed) condition, an additional (irrelevant) colour cue was shown during presentation. In Phase 2, a different association, between colour and facial expression, was learned. Instructions were the same for each phase: participants had to predict whether the subsequently viewed face was going to be happy or sad. No difference in error rate between the groups was found in Phase 1, suggesting that patients and controls learned at a similar rate. However, in Phase 2, a significant interaction was found between group and condition, with exposed controls performing significantly worse than unexposed (therefore demonstrating learned irrelevance). In contrast, exposed patients made a similar number of errors to unexposed in Phase 2. The pattern of results in ESRD was different from that previously found in Parkinson's disease, suggesting a different neural origin.

Depression impairs learning, whereas the selective serotonin reuptake inhibitor, paroxetine, impairs generalization in patients with major depressive disorder

To better understand how medication status and task demands affect cognition in major depressive disorder (MDD), we evaluated medication-naïve patients with MDD, medicated patients with MDD receiving the selective serotonin reuptake inhibitors (SSRI) paroxetine, and healthy controls. All three groups were administered a computer-based cognitive task with two phases, an initial phase in which a sequence is learned through reward-based feedback (which our prior studies suggest is striatal-dependent), followed by a generalization phase that involves a change in the context where learned rules are to be applied (which our prior studies suggest is hippocampal-region dependent). Medication-naïve MDD patients were slow to learn the initial sequence but were normal on subsequent generalization of that learning. In contrast, medicated patients learned the initial sequence normally, but were impaired at the generalization phase. We argue that these data suggest (i) an MDD-related impairment in striatal-dependent sequence learning which can be remediated by SSRIs and (ii) an SSRI-induced impairment in hippocampal-dependent generalization of past learning to novel contexts, not otherwise seen in the medication-naïve MDD group. Thus, SSRIs might have a beneficial effect on striatal function required for sequence learning, but a detrimental effect on the hippocampus and other medial temporal lobe structures is critical for generalization.

Positive Clinical Neuroscience

Disorders of the brain and its sensory organs have traditionally been associated with deficits in movement, perception, cognition, emotion, and behavior. It is increasingly evident, however, that positive phenomena may also occur in such conditions, with implications for the individual, science, medicine, and for society. This article provides a selective review of such positive phenomena – enhanced function after brain lesions, better-than-normal performance in people with sensory loss, creativity associated with neurological disease, and enhanced performance associated with aging. We propose that, akin to the well-established field of positive psychology and the emerging field of positive clinical psychology, the nascent fields of positive neurology and positive neuropsychology offer new avenues to understand brain-behavior relationships, with both theoretical and therapeutic implications.

Computational cognitive models of prefrontal-striatal-hippocampal interactions in Parkinson's disease and schizophrenia

Disruption to different components of the prefrontal cortex, basal ganglia, and hippocampal circuits leads to various psychiatric and neurological disorders including Parkinson's disease (PD) and schizophrenia. Medications used to treat these disorders (such as levodopa, dopamine agonists, antipsychotics, among others) affect the prefrontal-striatal-hippocampal circuits in a complex fashion. We have built models of prefrontal-striatal and striatal-hippocampal interactions which simulate cognitive dysfunction in PD and schizophrenia. In these models, we argue that the basal ganglia is key for stimulus-response learning, the hippocampus for stimulus-stimulus representational learning, and the prefrontal cortex for stimulus selection during learning about multidimensional stimuli. In our models, PD is associated with reduced dopamine levels in the basal ganglia and prefrontal cortex. In contrast, the cognitive deficits in schizophrenia are associated primarily with hippocampal dysfunction, while the occurrence of negative symptoms is associated with frontostriatal deficits in a subset of patients. In this paper, we review our past models and provide new simulation results for both PD and schizophrenia. We also describe an extended model that includes simulation of the different functional role of D1 and D2 dopamine receptors in the basal ganglia and prefrontal cortex, a dissociation we argue is essential for understanding the non-uniform effects of levodopa, dopamine agonists, and antipsychotics on cognition. Motivated by clinical and physiological data, we discuss model limitations and challenges to be addressed in future models of these brain disorders.

A neurocomputational model of tonic and phasic dopamine in action selection: a comparison with cognitive deficits in Parkinson's disease

The striatal dopamine signal has multiple facets; tonic level, phasic rise and fall, and variation of the phasic rise/fall depending on the expectation of reward/punishment. We have developed a network model of the striatal direct pathway using an ionic current level model of the medium spiny neuron that incorporates currents sensitive to changes in the tonic level of dopamine. The model neurons in the network learn action selection based on a novel set of mathematical rules that incorporate the phasic change in the dopamine signal. This network model is capable of learning to perform a sequence learning task that in humans is thought to be dependent on the basal ganglia. When both tonic and phasic levels of dopamine are decreased, as would be expected in unmedicated Parkinson's disease (PD), the model reproduces the deficits seen in a human PD group off medication. When the tonic level is increased to normal, but with reduced phasic increases and decreases in response to reward and punishment, respectively, as would be expected in PD medicated with L-Dopa, the model again reproduces the human data. These findings support the view that the cognitive dysfunctions seen in Parkinson's disease are not solely either due to the decreased tonic level of dopamine or to the decreased responsiveness of the phasic dopamine signal to reward and punishment, but to a combination of the two factors that varies dependent on disease stage and medication status.

A neurocomputational model of classical conditioning phenomena: a putative role for the hippocampal region in associative learning

Some existing models of hippocampal function simulate performance in classical conditioning tasks using the error backpropagation algorithm to guide learning (Gluck, M.A., and Myers, C.E., (1993). Hippocampal mediation of stimulus representation: a computational theory. Hippocampus, 3(4), 491-516.). This algorithm is not biologically plausible because it requires information to be passed backward through layers of nodes and assumes that the environment provides information to the brain about what correct outputs should be. Here, we show that the same information-processing function proposed for the hippocampal region in the Gluck and Myers (1993) model can also be implemented in a network without using the backpropagation algorithm. Instead, our newer instantiation of the theory uses only (a) Hebbian learning methods which match more closely with synaptic and associative learning mechanisms ascribed to the hippocampal region and (b) a more plausible representation of input stimuli. We demonstrate here that this new more biologically plausible model is able to simulate various behavioral effects, including latent inhibition, acquired equivalence, sensory preconditioning, negative patterning, and context shift effects. In addition, the newer model is able to address some new phenomena including the effect of the number of training trials on blocking and overshadowing.

Is awareness necessary for true inference?

In transitive inference, participants learn a set of context-dependent discriminations that can be organized into a hierarchy that supports inference. Several studies show that inference occurs with or without task awareness. However, some studies assert that without awareness, performance is attributable to pseudoinference. By this account, inference-like performance is achieved by differential stimulus weighting according to the stimuli's proximity to the end items of the hierarchy. We implement an inference task that cannot be based on differential stimulus weighting. The design itself rules out pseudoinference strategies. Success on the task without evidence of deliberative strategies would therefore suggest that true inference can be achieved implicitly. We found that accurate performance on the inference task was not dependent on explicit awareness. The finding is consistent with a growing body of evidence that indicates that forms of learning and memory supporting inference and flexibility do not necessarily depend on task awareness.

Cognitive complexity effects in perceptual classification are dissociable

It has been proposed that a procedural-based classification system mediates the learning of information-integration categories, whereas a hypothesis-testing system mediates the learning of rule-based categories. Ashby, Ell and Waldron (2003) provided support for this claim by showing that a button switch introduced during classification transfer adversely affected information-integration but not rule-based performance. Nosofsky, Stanton and Zaki (2005) showed that increasing "cognitive complexity" can lead to button switch costs on rule-based performance. They argue that "cognitive complexity," and not the existence of separable classification systems, accounts for Ashby et al.'s empirical dissociation. The present study shows that experimental manipulations that increase "cognitive complexity" often have dissociable effects on information-integration and rule-based classification that are predicted a priori from the processing characteristics associated with the procedural-based and hypothesis-testing systems. These results suggest that manipulations of "cognitive complexity" can be dissociated, suggesting that "cognitive complexity" in not a unitary construct that affects a single psychological process.

Discontinuous categories affect information-integration but not rule-based category learning

Three experiments were conducted that provide a direct examination of within-category discontinuity manipulations on the implicit, procedural-based learning and the explicit, hypothesis-testing systems proposed in F. G. Ashby, L. A. Alfonso-Reese, A. U. Turken, and E. M. Waldron's (1998) competition between verbal and implicit systems model. Discontinuous categories adversely affected information-integration but not rule-based category learning. Increasing the magnitude of the discontinuity did not lead to a significant decline in performance. The distance to the bound provides a reasonable description of the generalization profile associated with the hypothesis-testing system, whereas the distance to the bound plus the distance to the trained response region provides a reasonable description of the generalization profile associated with the procedural-based learning system. These results suggest that within-category discontinuity differentially impacts information-integration but not rule-based category learning and provides information regarding the detailed processing characteristics of each category learning system.

Cortico-hippocampal interaction and adaptive stimulus representation: a neurocomputational theory of associative learning and memory

Computational models of the hippocampal region link psychological theories of associative learning with their underlying physiological and anatomical substrates. Our approach to theory development began with a broad description of the computations that depend on the hippocampal region in classical conditioning (Gluck and Myers, 1993 and Gluck and Myers, 2001). In this initial model, the hippocampal region was treated as an Information-processing system that transformed stimulus representations, compressing (making more similar) representations of inputs that co-occur or are otherwise redundant, while differentiating (or making less similar) representations of inputs that predict different future events. This model led to novel predictions for the behavioral consequences of hippocampal-region lesions in rodents and of brain damage in humans who have amnesia or are in the earliest stages of Alzheimer's disease. Many of these predictions have, since been confirmed by our lab and others. Functional brain imaging studies have provided further supporting evidence. In more recent computational modeling, we have shown how some aspects of this proposed information-processing function could emerge from known anatomical and physiological characteristics of the hippocampal region, including the entorhinal cortex and the septo-hippocampal cholinergic system. The modeling to date lays the groundwork for future directions that increase the depth of detail of the biological modeling, as well as the breadth of behavioral phenomena addressed. In particular, we are working now to reconcile these kinds of incremental associative learning models with other models of the hippocampal region that account for the rapid formation of declarative memories.

Contributions of the hippocampus and the striatum to simple association and frequency-based learning

Using fMRI and a learning paradigm, this study examined the independent contributions of the hippocampus and striatum to simple association and frequency-based learning. We scanned 10 right-handed young adult subjects using a spiral in/out sequence on a GE 3.0 T scanner during performance of the learning paradigm. The paradigm consisted of 2 cues that predicted each of 3 targets with varying probabilities. Simultaneously, we varied the frequency with which each target was presented throughout the task, independent of cue associations. Subjects had shorter response latencies to frequently occurring and highly associated target stimuli and longer response latencies to infrequent target stimuli, indicating learning. Imaging results showed increased caudate activity to infrequent relative to frequent targets and increased hippocampal activity to infrequent relative to frequent cue-target associations. This work provides evidence of different neural mechanisms underlying learning based on simple frequencies versus associations within a single paradigm.

The relevance of irrelevance to schizophrenia

Jeffrey Gray's neuropsychological theory of the positive symptoms of schizophrenia has been highly influential by enabling a strong link between animal and human research. Central to the development and testing of this theory has been the phenomenon and paradigm of latent inhibition (LI-the retardation of learning that one stimulus predicts the occurrence of another due to pre-exposure of the first stimulus). We review findings relating to its alteration in patients with schizophrenia (acute and chronic), people high on dimensions of schizotypy and the effects of amphetamine and anti-psychotic medication in humans. We suggest that many human-LI paradigms still suffer from theoretical and practical limitations, but that recent developments are beginning to address these. Finally we explore the idea that the paradigm of Learned Irrelevance (LIRR-the retardation of learning that one stimulus predicts the occurrence of another due to pre-exposure of both stimuli but in an unrelated manner) might be used to complement studies on LI in exploring the cognitive distortions suffered by patients with schizophrenia.