Learning by neurones: role of attention, reinforcement and behaviour

Lower novelty-related locus coeruleus function is associated with Aβ-related cognitive decline in clinically healthy individuals

Animal and human imaging research reported that the presence of cortical Alzheimer’s Disease’s (AD) neuropathology, beta-amyloid and neurofibrillary tau, is associated with altered neuronal activity and circuitry failure, together facilitating clinical progression. The locus coeruleus (LC), one of the initial subcortical regions harboring pretangle hyperphosphorylated tau, has widespread connections to the cortex modulating cognition. Here we investigate whether LC’s in-vivo neuronal activity and functional connectivity (FC) are associated with cognitive decline in conjunction with beta-amyloid. We combined functional MRI of a novel versus repeated face-name paradigm, beta-amyloid-PET and longitudinal cognitive data of 128 cognitively unimpaired older individuals. We show that LC activity and LC-FC with amygdala and hippocampus was higher during novelty. We also demonstrated that lower novelty-related LC activity and LC-FC with hippocampus and parahippocampus were associated with steeper beta-amyloid-related cognitive decline. Our results demonstrate the potential of LC’s functional properties as a gauge to identify individuals at-risk for AD-related cognitive decline.

Older individuals exhibiting diminished function of the locus coeruleus while learning new information show faster cognitive decline that is typical for Alzheimer’s disease.

Dentate granule and mossy cells exhibit distinct spatiotemporal responses to local change in a one-dimensional landscape of visual-tactile cues

The dentate gyrus (DG) is critical for detecting changes in environments; however, how granule cells (GCs) and mossy cells (MCs), the two excitatory cell types of the DG, respond to small changes in the object layout is unclear. Here, we recorded GCs and MCs, identified by spike feature and optogenetic tagging, as mice ran on a treadmill belt enriched with visual-tactile cues. We observed that fixing a new cue on the belt induced a reconfiguration of GC and MC spatial representations via the emergence, extinction and rate alteration of firing fields. For both GCs and MCs, the response was maximal near the cue and spread over the entire belt. However, compared to the GC response, the MC response was stronger and more immediate, peaked at a slightly earlier belt position, and exhibited a transient component reminiscent of neuromodulatory activity. A competitive neural network model reproduced the GC response contingent on both the introduction of new object-vector inputs and the reconfiguration of MC activity, the former being critical for spreading the GC response in locations distant from the cue. These findings suggest that GCs operate as a competitive network and that MCs precede GCs in detecting changes and help expand the range of GC pattern separation.

Orexin modulates behavioral fear expression through the locus coeruleus

Emotionally salient information activates orexin neurons in the lateral hypothalamus, leading to increase in sympathetic outflow and vigilance level. How this circuit alters animals’ behavior remains unknown. Here we report that noradrenergic neurons in the locus coeruleus (NA^LC neurons) projecting to the lateral amygdala (LA) receive synaptic input from orexin neurons. Pharmacogenetic/optogenetic silencing of this circuit as well as acute blockade of the orexin receptor-1 (OX1R) decreases conditioned fear responses. In contrast, optogenetic stimulation of this circuit potentiates freezing behavior against a similar but distinct context or cue. Increase of orexinergic tone by fasting also potentiates freezing behavior and LA activity, which are blocked by pharmacological blockade of OX1R in the LC. These findings demonstrate the circuit involving orexin, NA^LC and LA neurons mediates fear-related behavior and suggests inappropriate excitation of this pathway may cause fear generalization sometimes seen in psychiatric disorders, such as PTSD.

Vigilance involves the activation of orexinergic neurons in the lateral hypothalamus (LH-ox). Here the authors report the functional role of a monosynaptically connected circuit with orexinergic neurons connected to noradrenergic neurons in the locus coeruleus which target lateral amygdala neurons and enhance fear expression and generalization.

Balancing New against Old Information: The Role of Puzzlement Surprise in Learning

Surprise describes a range of phenomena from unexpected events to behavioral responses. We propose a novel measure of surprise and use it for surprise-driven learning. Our surprise measure takes into account data likelihood as well as the degree of commitment to a belief via the entropy of the belief distribution. We find that surprise-minimizing learning dynamically adjusts the balance between new and old information without the need of knowledge about the temporal statistics of the environment. We apply our framework to a dynamic decision-making task and a maze exploration task. Our surprise-minimizing framework is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes, and it could eventually provide a framework to study the behavior of humans and animals as they encounter surprising events.

Clinical utility of guanfacine extended release in the treatment of ADHD in children and adolescents

Attention deficit hyperactivity disorder (ADHD) is the most common psychiatric illness in children and adolescents. Several stimulant medications, such as methylphenidate and amphetamine derivatives, are available to treat ADHD in pediatric patients. Nonstimulant medications are more preferred by some parents, other caregivers, and patients because they lack the abuse potential of stimulant medications. In the US, one available nonstimulant option is guanfacine extended release (XR). As a selective α2A adrenergic receptor, guanfacine acts on the central noradrenergic pathways and cortical noradrenergic targets to improve working memory and attention. The XR formulation of guanfacine, compared with the immediate-release formulation, is more effective for the long-term management of ADHD and is associated with fewer adverse effects. Available data also indicate that guanfacine XR is superior to atomoxetine and is as effective as the nonselective α2 adrenergic receptor agonist, clonidine XR. The most common adverse effects associated with guanfacine XR are somnolence, fatigue, bradycardia, and hypotension. Somnolence is the most often cited reason for discontinuation. Guanfacine XR is also labeled for use as an adjuvant to stimulant treatment for ADHD. A similar profile of adverse effects as reported with monotherapy is reported when guanfacine XR is "added on" to stimulant therapy with somnolence as the most commonly reported adverse event. This review discusses the clinical efficacy and patient preference of guanfacine XR based on available published data on the safety, relative effectiveness, and tolerance of this medication to treat ADHD.

Picture novelty attenuates semantic interference and modulates concomitant neural activity in the anterior cingulate cortex and the locus coeruleus

Goal-directed behavior requires the ability to focus on information that is relevant to a given task and to ignore information that might interfere with it. In the Stroop task, for example, the influence of an irrelevant word needs to be overcome, which is believed to be difficult because it arises in a fast and automatic fashion, which effectively renders it very salient. Here we address the question of whether this can be counteracted by increasing the saliency of the task-relevant input, for example by modulating its relative novelty, which increases saliency in a fairly implicit and controlled fashion. To test the influence of novelty on interference processing, we employed a picture-word interference task in the fMRI scanner, in which we manipulated the novelty of the task-relevant picture. We found that picture novelty indeed reduced typical behavioral interference from incongruent words. Moreover, familiar incongruent trials were associated with activity increases in the anterior cingulate cortex (ACC), a prime conflict-processing region, as well as in the noradrenergic locus coeruleus (LC), which entertains connections both to and from the ACC. The lack of analogous activations in novel incongruent trials suggests that the reduction of behavioral interference was not related to enhanced conflict-resolution processes, but rather to the automatic prioritization of novel pictures which appears to avert the influence of irrelevant words at the front end. Interestingly, activity in the ACC and LC was slightly stronger in novel congruent trials compared to incongruent ones, which may reflect increased relevance of novel stimuli when encoded in a congruent context. In summary, the present data demonstrate that stimulus novelty clearly reduces semantic interference, and highlights a complex interaction of interference and novelty processing on the neural level, including an involvement of the noradrenergic system in the processing of cognitively and perceptually salient events.

Attention, uncertainty, and free-energy

We suggested recently that attention can be understood as inferring the level of uncertainty or precision during hierarchical perception. In this paper, we try to substantiate this claim using neuronal simulations of directed spatial attention and biased competition. These simulations assume that neuronal activity encodes a probabilistic representation of the world that optimizes free-energy in a Bayesian fashion. Because free-energy bounds surprise or the (negative) log-evidence for internal models of the world, this optimization can be regarded as evidence accumulation or (generalized) predictive coding. Crucially, both predictions about the state of the world generating sensory data and the precision of those data have to be optimized. Here, we show that if the precision depends on the states, one can explain many aspects of attention. We illustrate this in the context of the Posner paradigm, using the simulations to generate both psychophysical and electrophysiological responses. These simulated responses are consistent with attentional bias or gating, competition for attentional resources, attentional capture and associated speed-accuracy trade-offs. Furthermore, if we present both attended and non-attended stimuli simultaneously, biased competition for neuronal representation emerges as a principled and straightforward property of Bayes-optimal perception.

Galanin and consummatory behavior: special relationship with dietary fat, alcohol and circulating lipids

Galanin (GAL) plays an integral role in consummatory behavior. In particular, hypothalamic GAL has a positive, reciprocal relationship with dietary fat and alcohol. In this relationship, GAL increases the consumption of fat or alcohol which, in turn, stimulates the expression of GAL, ultimately leading to overconsumption. Through actions in the amygdala, this relationship may become especially important in stress-induced food or drug intake. These effects of GAL in promoting overconsumption may involve various neurotransmitters, with GAL facilitating intake by stimulating norepinephrine and dopamine and reducing satiety by decreasing serotonin and acetylcholine. In addition, GAL in the hypothalamus stimulates the opioid, enkephalin, throughout the brain, which also promotes overconsumption. The relationship between GAL, fat, and alcohol may involve triglycerides, circulating lipids that are released by fat or alcohol and that correlate positively with hypothalamic GAL expression. In females, levels of endogenous GAL also fluctuate across the reproductive cycle, driven by a rise in the ovarian steroids, estrogen, and progesterone. They peak during the proestrous phase and also at puberty, simultaneous to a sharp increase in preference for fat to meet energy demands. Prenatal exposure to a high-fat diet also enhances hypothalamic expression of GAL into adulthood because of an increase in neurogenesis and proliferation of GAL-expressing neurons in this region. This organizational change may reflect the role of GAL in neuronal development, including neurite growth in adulthood, cell survival in aging, and cell stability in the disease state. By responding positively to fat and alcohol and guiding further neuronal development, GAL potentiates a long-term propensity to overconsume fat and alcohol.

Adrenergic drugs modify the level of noradrenaline in the insular cortex and alter extinction of conditioned taste aversion in rats

We compared the effect of conditioned taste aversion in rats by measuring the amount of sucrose that they drunk after conditioning, which differed according to whether rats had drunk the sucrose freely (SD: self drinking) during the conditioning session, or had been forced to drink it (IO: intra-oral administration through a chronically implanted cannula). The SD procedure delayed the extinction of conditioned taste aversion. Enhanced arousal, alertness, awareness or attention in the SD condition may have strengthened the memory of the taste. Brain noradrenergic networks are involved in such processes. We administered two noradrenergic drugs that produce opposite effects on noradrenaline release in the brain, methoxy-idazoxan, RX821002 (1mg/kg, i.p.), and guanfacine (0.12mg/kg, i.p.). We evaluated their effect (i) on the level of noradrenaline in the gustatory cortex using microdialysis, (ii) on glycaemia that is an essential factor of taste learning and (iii) on the comparative SD versus IO conditioned taste aversion protocol mentioned above. Injecting RX821001 increased the level of noradrenaline in the gustatory cortex up to two-fold of the baseline. This effect lasted 1h. The same dose of RX821002 did not elicit any alteration of glycaemia. It enhanced extinction of conditioned taste aversion in the SD group of rats. Injecting 0.12mg/kg of guanfacine produced the opposite effect. The noradrenaline level of the gustatory cortex decreased, but only down to 20% of the baseline. This decrease lasted 2h. Guanfacine increased glycaemia. Extinction of conditioned taste aversion was only marginally decreased by guanfacine in the SD group of rats. These results fit with Aston-Jones' point of view that the role of the noradrenergic coeruleo-cortical system may be to enhance arousal, alertness, awareness or attention to an event by a transient increase of cortical noradrenaline.

Expecting the unexpected: modeling of neuromodulation

An important aspect of decision-making is the need to make interpretations and predictions in the face of uncertain information. In this issue of Neuron, Yu and Dayan describe a model of the role of acetylcholine and norepinephrine in balancing top-down expectation and bottom-up sensory input in guiding behavior. The model builds from data on physiological effects of modulators regulating the balance of cortical feedback and thalamic input.

Uncertainty, neuromodulation, and attention

Uncertainty in various forms plagues our interactions with the environment. In a Bayesian statistical framework, optimal inference and prediction, based on unreliable observations in changing contexts, require the representation and manipulation of different forms of uncertainty. We propose that the neuromodulators acetylcholine and norepinephrine play a major role in the brain's implementation of these uncertainty computations. Acetylcholine signals expected uncertainty, coming from known unreliability of predictive cues within a context. Norepinephrine signals unexpected uncertainty, as when unsignaled context switches produce strongly unexpected observations. These uncertainty signals interact to enable optimal inference and learning in noisy and changeable environments. This formulation is consistent with a wealth of physiological, pharmacological, and behavioral data implicating acetylcholine and norepinephrine in specific aspects of a range of cognitive processes. Moreover, the model suggests a class of attentional cueing tasks that involve both neuromodulators and shows how their interactions may be part-antagonistic, part-synergistic.

Competitive interactions between endogenous LTD and LTP in the hippocampus underlie the storage of emotional memories and stress-induced amnesia

This speculative review serves two purposes. First, it as an extension of the ideas we developed in a previous review (Diamond et al., Hippocampus, 2004;14:281-291), and second, it is a rebuttal to Abraham's (Hippocampus, 2004;14:675-676) critique of that review. We had speculated on the functional significance of the finding that post-training LTP induction produces retrograde amnesia. We noted the similarities between the findings that strong tetanizing stimulation can produce LTP and retrograde amnesia, and that a strong emotional experience can produce a long-lasting memory and retrograde amnesia, as well. The commonalities between LTP induction and emotional learning provided the basis of our hypothesis that an emotional experience generates endogenous LTD/depotentiation, which reverses synaptic plasticity formed during previous learning experiences, and endogenous LTP, which underlies the storage of new information. Abraham raised several concerns with our review, including the criticism that our speculation "falters because there is no evidence that stress causes LTD or depotentiation," and that research on stress and hippocampus has "failed to report any LTP-like changes." Abraham's points are well-taken because stress, in isolation, does not appear to generate long-lasting changes in baseline measures of hippocampal excitability. Here, within the context of a reply to Abraham's critique, we have provided a review of the literature on the influence of stress, novelty, fear conditioning, and the retrieval of emotional memories on cognitive and physiological measures of hippocampal functioning. An emphasis of this review is our hypothesis that endogenous forms of depotentiation, LTD and LTP are generated only when arousing experiences occur in conjunction with memory-related activation of the hippocampus and amygdala. We conclude with speculation that interactions among the different forms of endogenous plasticity underlie a form of competition by synapses and memories for access to retrieval resources.

Alpha 2-adrenoceptors in the basal ganglia have a role in memory consolidation and reinforcement

This study demonstrates a role for alpha(2)-adrenoceptors in the basal ganglia in the consolidation of memory using weakly and strongly reinforced models of discriminated avoidance learning in the chick. The memory enhancing action of noradrenaline injected into the basal ganglia (lobus parolfactorius-LPO) was reduced in the presence of the alpha(2)-adrenoceptor antagonist yohimbine, but when noradrenaline was injected into the multi-modal association area (intermediate medial hyperstriatum ventrale-IMHV), yohimbine failed to prevent memory enhancement. Yohimbine injected into the LPO prevented, whereas the alpha(2)-adrenoceptor agonists oxymetazoline and clonidine enhanced, consolidation of memory. The timing of the inhibitory effect of yohimbine in the LPO suggested that alpha(2)-adrenoceptor involvement occurs 10-15 min after training, and that stimulation of alpha(2)-ARs in LPO is necessary for subsequent consolidation of memory. Oxymetazoline, being hydrophilic, was ineffective injected into IMHV, whereas the action of the lipophilic alpha(2)-adrenoceptor agonist clonidine in the IMHV was interpreted as an action at a site more distal in the brain, probably the LPO. The results suggest that noradrenaline release in the basal ganglia in the chick stimulates alpha(2)-adrenoceptors, which modulate and consolidate memory formation mediated by beta(2)- or beta(3)-ARs in the association area. The LPO may be responsible for the reinforcement of memory in the IMHV.

Pontine sources of norepinephrine in the cat cochlear nucleus

In the current study, the distribution of noradrenergic neurons in the pontine tegmentum that project to the cochlear nucleus was determined with retrograde tract tracing combined with neurotransmitter immunohistochemistry in the cat. Double-labeled neurons were observed in all noradrenergic cell groups, in both the dorsolateral and the ventrolateral tegmentum. Half of the double-labeled cells were located in the locus coeruleus complex. Most of these were situated in its ventral division. Most other double-labeled cells were located in peribrachial regions, especially lateral to the brachium conjunctivum. Relatively few double-labeled cells were observed in both the A4 and the A5 cell groups, 2% and 0.4%, respectively, of the total. Except for neurons in A5, which projected only contralaterally, the projections were bilateral, with an ipsilateral preponderance. The results indicate that neurons located in the ipsilateral dorsolateral tegmentum, namely, in the locus coeruleus complex and the peribrachial region, are the primary source of pontine noradrenergic afferents to the cochlear nucleus of the cat.

Effect of an alpha(1)-adrenergic blocker on plasticity elicited by motor training

Recovery of motor function elicited by motor training after cortical lesions in rats is enhanced by norepinephrine (neurotransmitter mediating alpha(1)-adrenergic function) and downregulated by alpha(1)-adrenergic antagonists. In spite of this, alpha(1)-adrenergic antagonists are used to treat elderly patients with hypertension and prostate hyperplasia in stroke settings. The purpose of this study was to determine the effects of a single oral dose of the alpha(1)-adrenergic antagonist prazosin on training-dependent plasticity in intact humans, a function thought to contribute to recovery of motor function after cortical lesions. We report that prazosin decreased the ability of motor training to elicit training-dependent plasticity relative to a drug-free condition. These data suggest caution when using alpha(1)-adrenergic blockers in rehabilitative clinical settings following brain lesions.

Stimulation of alpha1-adrenoceptors inhibits memory consolidation in the chick

Investigation of the effects of the different adrenoceptor (AR) subtypes in memory formation may reveal discrete actions of noradrenaline in memory modulation and storage mediated through particular AR subtypes. Noradrenaline injected intracerebrally in the chick produced biphasic effects on memory consolidation with enhancement at low doses and inhibition at high doses. We have previously shown that the enhancement by the lower doses of noradrenaline is attributable to actions at beta2- and beta3-adrenoceptors, whereas the inhibitory effect of higher doses is attributable to alpha1-adrenoceptors. The present studies show that the inhibition of memory by high doses of noradrenaline is mimicked by the alpha1-AR agonist methoxamine, and the dose-response curve is shifted to the right by pretreatment with the alpha1-AR antagonist prazosin. alpha1-ARs may play a critical role in memory formation in highly stressful situations, when noradrenaline levels are high in particular brain regions. It is not known where the alpha1-ARs responsible for the effect on memory are localized. alpha1-ARs are found on neurons and astrocytes and in the cerebral vasculature and therefore the action of high doses of noradrenaline via alpha1-AR agonists could be via an action at any of these sites. Activation of alpha1-adrenoceptors in the intermediate hyperstriatum ventrale in the chick forebrain by the alpha1 adrenoceptor agonist methoxamine inhibits the consolidation of memory. Because the same effect is produced by high levels of noradrenaline, it is likely that stimulation of alpha1-ARs is the mechanism underlying this effect.

Consolidation of Memory for Odor–Reward Association: β-Adrenergic Receptor Involvement in the Late Phase

Experimentally naive rats can learn rapidly to discriminate among three odors to obtain food reinforcement. After three massed trials, they show almost errorless performance. This task has proved to be useful in studying time-dependent postacquisition intracellular processes necessary for long-term memory. The present experiments evaluated the temporal dynamics of the role of β-noradrenergic receptors in long-term consolidation. Rats were implanted with intracerebroventricular cannulae and trained in a single session to find reinforcement in a hole in a sponge impregnated with a particular odor. Injections of the β-receptor antagonist timolol were made at 5 min, 1, 2, or 5 hr after training. Memory and relearning ability were evaluated 48 hr later. Rats treated with timolol 2 hr after training showed a memory deficit at the retention test, but were able to relearn the task normally. Injections at the earlier or later time points were ineffective. The results reinforce previous observations with systemic injections that β-noradrenergic receptors are involved in the late phase of memory consolidation and suggest a critical time window during which they are necessary. The time window is compatible with the current view that long-term memory depends on late involvement of the cAMP cascade leading to new protein synthesis necessary for synaptic reorganization.