A frontal dopamine system for reflective exploratory behavior

Seeking motivation and reward: roles of dopamine, hippocampus and supramammillo-septal pathway

Reinforcement learning and goal-seeking behavior are thought to be mediated by midbrain dopamine neurons. However, little is known about neural substrates of curiosity and exploratory behavior, which occur in the absence of clear goal or reward. This is despite behavioral scientists having long suggested that curiosity and exploratory behaviors are regulated by an innate drive. We refer to such behavior as information-seeking behavior and propose 1) key neural substrates and 2) the concept of environment prediction error as a framework to understand information-seeking processes. The cognitive aspect of information-seeking behavior, including the perception of salience and uncertainty, involves, in part, the pathways from the posterior hypothalamic supramammillary region to the hippocampal formation. The vigor of such behavior is modulated by the following: supramammillary glutamatergic neurons; their projections to medial septal glutamatergic neurons; and the projections of medial septal glutamatergic neurons to ventral tegmental dopaminergic neurons. Phasic responses of dopaminergic neurons are characterized as signaling potentially important stimuli rather than rewards. This paper describes how novel stimuli and uncertainty trigger seeking motivation and how these neural substrates modulate information-seeking behavior.

Effects of low-dose methylcyclopentadienyl manganese tricarbonyl-derived manganese on the development of diencephalic dopaminergic neurons in zebrafish

Fuel additive methylcyclopentadienyl manganese tricarbonyl (MMT) is counted as an organic manganese (Mn)-derived compound. The toxic effects of Mn (alone and complexed) on dopaminergic (DA) neurotransmission have been investigated in both cellular and animal models. However, the impact of environmentally relevant Mn exposure on DA neurodevelopment is rather poorly understood. In the present study, the MMT dose of 100 μM (about 5 mg Mn/L) caused up-regulation of DA-related genes in association with cell body swelling and increase in the number of DA neurons of the ventral diencephalon subpopulation DC2. Furthermore, our analysis identified significant brain Mn bioaccumulation and enhancement of total dopamine levels in association with locomotor hyperactivity. Although DA levels were restored at adulthood, we observed a deficit in the acquisition and consolidation of memory. Collectively, these findings suggest that developmental exposure to low-level MMT-derived Mn is responsible for the selective alteration of diencephalic DA neurons and with long-lasting effects on fish explorative behaviour in adulthood.

The modulation of brain network integration and arousal during exploration

There is growing interest in how neuromodulators shape brain networks. Recent neuroimaging studies provide evidence that brainstem arousal systems, such as the locus coeruleus-norepinephrine system (LC-NE), influence functional connectivity and brain network topology, suggesting they have a role in flexibly reconfiguring brain networks in order to adapt behavior and cognition to environmental demands. To date, however, the relationship between brainstem arousal systems and functional connectivity has not been assessed within the context of a task with an established relationship between arousal and behavior, with most prior studies relying on incidental variations in arousal or pharmacological manipulation and static brain networks constructed over long periods of time. These factors have likely contributed to a heterogeneity of effects across studies. To address these issues, we took advantage of the association between LC-NE-linked arousal and exploration to probe the relationships between exploratory choice, arousal—as measured indirectly via pupil diameter—and brain network dynamics. Exploration in a bandit task was associated with a shift toward fewer, more weakly connected modules that were more segregated in terms of connectivity and topology but more integrated with respect to the diversity of cognitive systems represented in each module. Functional connectivity strength decreased, and changes in connectivity were correlated with changes in pupil diameter, in line with the hypothesis that brainstem arousal systems influence the dynamic reorganization of brain networks. More broadly, we argue that carefully aligning dynamic network analyses with task designs can increase the temporal resolution at which behaviorally- and cognitively-relevant modulations can be identified, and offer these results as a proof of concept of this approach.

Attentional mechanisms drive systematic exploration in young children

Exploration is critical for discovering how the world works. Exploration should be particularly valuable for young children, who have little knowledge about the world. Theories of decision-making describe systematic exploration as being primarily driven by top-down cognitive control, which is immature in young children. Recent research suggests that a type of systematic exploration predominates in young children's choices, despite immature control, suggesting that it may be driven by different mechanisms. We hypothesize that young children's tendency to distribute attention widely promotes elevated exploration, and that interrupting distributed attention allocation through bottom-up attentional capture would also disrupt systematic exploration. We test this hypothesis by manipulating saliency of the options in a simple choice task. Saliency disrupted systematic exploration, thus indicating that attentional mechanisms may drive children's systematic exploratory behavior. We suggest that both may be part of a larger tendency toward broad information gathering in young children.

Systematic exploration and uncertainty dominate young children's choices

Organisms need to constantly balance the competing demands of gathering information and using previously acquired information to obtain rewarding outcomes (i.e., the "exploration-exploitation" dilemma). Exploration is critical to obtain information to discover how the world works, which should be particularly important for young children. While studies have shown that young children explore in response to surprising events, little is known about how they balance exploration and exploitation across multiple decisions or about how this process changes with development. In this study, we compare decision-making patterns of children and adults and evaluate the relative influences of reward seeking, random exploration, and systematic switching (which approximates uncertainty-directed exploration). In a second experiment, we directly test the effect of uncertainty on children's choices. Influential models of decision-making generally describe systematic exploration as a computationally refined capacity that relies on top-down cognitive control. We demonstrate that (a) systematic patterns dominate young children's behavior (facilitating exploration), despite protracted development of cognitive control; and (b) that uncertainty plays a major, but complicated, role in determining children's choices. We conclude that while young children's immature top-down control should hinder adult-like systematic exploration, other mechanisms may pick up the slack, facilitating broad information gathering in a systematic fashion to build a foundation of knowledge for use later in life.

Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

Involvement of dopamine in regulating exploration during decision-making has long been hypothesized, but direct causal evidence in humans is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging to address this issue. Thirty-one healthy male participants performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150 mg of the dopamine precursor L-dopa, 2 mg of the D2 receptor antagonist haloperidol, and placebo. Choices were best explained by an extension of an established Bayesian learning model accounting for perseveration, directed exploration and random exploration. Modeling revealed attenuated directed exploration under L-dopa, while neural signatures of exploration, exploitation and prediction error were unaffected. Instead, L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates how this circuit tracks accumulating uncertainty during decision-making.

Associations between the dopamine D4 receptor gene polymorphisms and personality traits in elite athletes

Personality traits and temperament may affect sports performance. Previous studies suggest that dopamine may play an important role in behavior regulation and physical exercise performance. The aim of this study is to determine associations between dopamine D4 receptor gene (DRD4 Ex3) polymorphisms and personality traits (such as neuroticism, extraversion, openness, agreeability and conscientiousness) in elite combat athletes. A total of 302 physically active, unrelated, self-reported Caucasian participants were recruited for this study. The participants consisted of 200 elite male combat athletes and 102 healthy male participants (control group). For personality trait measurements, the NEO Five-Factor Personality Inventory (NEO-FFI) and the State-Trait Anxiety Inventory questionnaires were used. For the genetic assays, blood was collected and all samples were genotyped using the real-time PCR method. A 2 x 3 factorial ANOVA revealed statistically significant differences on the Openness NEO Five Factor Inventory scale for both examined factors, i.e. sport status and genetics DTD4 Ex3. Combat athletes achieved higher scores on the Conscientiousness NEO-FFI scale when compared to controls (7.18 vs 5.98). On the other hand, combat athletes scored lower on the Openness scale in comparison with control group (4.42 vs. 4.63). Subjects with the DRD4 Ex3 s/s genotype had lower results on the openness scale in comparison with participants with the DRD4 Ex3 s/1 genotype (4.01 vs. 4.57) and higher DRD4 Ex3 1/1 genotype (4,01 vs. 3,50). In conclusion, we found an association between the dopamine D4 receptor gene in variable number tandem repeat (VNTR) polymorphisms and athletic status for two NEO-FFI factors: Openness and Conscientiousness. The DRD4 exon 3 polymorphism may be associated with the selected personality traits in combat athletes, thereby modulating athletes’ predisposition to participate in high risk sports.

The algorithmic architecture of exploration in the human brain

Balancing exploration and exploitation is one of the central problems in reinforcement learning. We review recent studies that have identified multiple algorithmic strategies underlying exploration. In particular, humans use a combination of random and uncertainty-directed exploration strategies, which rely on different brain systems, have different developmental trajectories, and are sensitive to different task manipulations. Humans are also able to exploit sophisticated structural knowledge to aid their exploration, such as information about correlations between options. New computational models, drawing inspiration from machine learning, have begun to formalize these ideas and offer new ways to understand the neural basis of reinforcement learning.

Dopaminergic genes are associated with both directed and random exploration

In order to maximize long-term rewards, agents must balance exploitation (choosing the option with the highest payoff) and exploration (gathering information about options that might have higher payoffs). Although the optimal solution to this trade-off is intractable, humans make use of two effective strategies: selectively exploring options with high uncertainty (directed exploration), and increasing the randomness of their choices when they are more uncertain (random exploration). Using a task that independently manipulates these two forms of exploration, we show that single nucleotide polymorphisms related to dopamine are associated with individual differences in exploration strategies. Variation in a gene linked to prefrontal dopamine (COMT) predicted the degree of directed exploration, as well as the overall randomness of responding. Variation in a gene linked to striatal dopamine (DARPP-32) predicted the degree of both directed and random exploration. These findings suggest that dopamine makes multiple contributions to exploration, depending on its afferent target.

Catechol-O-Methyltransferase Genotypes and Parenting Influence on Long-Term Executive Functioning After Moderate to Severe Early Childhood Traumatic Brain Injury: An Exploratory Study

OBJECTIVES

To examine catechol-O-methyltransferase (COMT) rs4680 genotypes as moderators of the effects of parenting style on postinjury changes in parent behavior ratings of executive dysfunction following moderate to severe early childhood traumatic brain injury.

SETTING

Research was conducted in an outpatient setting.

PARTICIPANTS

Participants included children admitted to hospital with moderate to severe traumatic brain injury (n = 55) or orthopedic injuries (n = 70) between ages 3 and 7 years.

DESIGN

Prospective cohort followed over 7 years postinjury.

MAIN MEASURES

Parenting Practices Questionnaire and the Behavior Rating Inventory of Executive Functioning obtained at baseline, 6, 12, and 18 months, and 3.5 and 6.8 years postinjury. DNA was collected from saliva samples, purified using the Oragene (DNA Genotek, Ottawa, Ontario, Canada) OG-500 self-collection tubes, and analyzed using TaqMan (Applied Biosystems, Thermo Fisher Scientific, Waltham, Massachusetts) assay protocols to identify the COMT rs4680 polymorphism.

RESULTS

Linear mixed models revealed a significant genotype × parenting style × time interaction (F = 5.72, P = .02), which suggested that the adverse effects of authoritarian parenting on postinjury development of executive functioning were buffered by the presence of the COMT AA genotype (lower enzyme activity, higher dopamine levels). There were no significant associations of executive functioning with the interaction between genotype and authoritative or permissive parenting ratings.

CONCLUSION

The lower activity COMT rs4680 genotype may buffer the negative effect of authoritarian parenting on long-term executive functioning following injury in early childhood. The findings provide preliminary evidence for associations of parenting style with executive dysfunction in children and for a complex interplay of genetic and environmental factors as contributors to decreases in these problems after traumatic injuries in children. Further investigation is warranted to understand the interplay among genetic and environmental factors related to recovery after traumatic brain injury in children.

A Primer on Foraging and the Explore/Exploit Trade-Off for Psychiatry Research

Foraging is a fundamental behavior, and many types of animals appear to have solved foraging problems using a shared set of mechanisms. Perhaps the most common foraging problem is the choice between exploiting a familiar option for a known reward and exploring unfamiliar options for unknown rewards-the so-called explore/exploit trade-off. This trade-off has been studied extensively in behavioral ecology and computational neuroscience, but is relatively new to the field of psychiatry. Explore/exploit paradigms can offer psychiatry research a new approach to studying motivation, outcome valuation, and effort-related processes, which are disrupted in many mental and emotional disorders. In addition, the explore/exploit trade-off encompasses elements of risk-taking and impulsivity-common behaviors in psychiatric disorders-and provides a novel framework for understanding these behaviors within an ecological context. Here we explain relevant concepts and some common paradigms used to measure explore/exploit decisions in the laboratory, review clinically relevant research on the neurobiology and neuroanatomy of explore/exploit decision making, and discuss how computational psychiatry can benefit from foraging theory.

Effect of stress on human biology: Epigenetics, adaptation, inheritance, and social significance

I present a brief introduction to epigenetics, focused primarily on methylation of the genome and various regulatory RNAs, modifications of associated histones, and their importance in enabling us to adapt to real and changing environmental, developmental, and social circumstances. Following this is a more extensive overview of how it impacts our inheritance, our entire life (which changes as we age), and how we interact with others. Throughout, I emphasize the critical influence that stress, of many varieties exerts, via epigenetic means, on much of how we live and survive, mostly in the brain. I end with a short section on multigenerational transmission, drugs, and the importance of both social life and early life experiences in the development of adult diseases. There will be nothing about cancer. Although epigenetics is critical in that field, it is a whole different cobweb of complications (some involving stress).

Acteoside and Isoacteoside Protect Amyloid β Peptide Induced Cytotoxicity, Cognitive Deficit and Neurochemical Disturbances In Vitro and In Vivo

Acteoside and isoacteoside, two phenylethanoid glycosides, coexist in some plants. This study investigates the memory-improving and cytoprotective effects of acteoside and isoacteoside in amyloid β peptide 1-42 (Aβ 1-42)-infused rats and Aβ 1-42-treated SH-SY5Y cells. It further elucidates the role of amyloid cascade and central neuronal function in these effects. Acteoside and isoacteoside ameliorated cognitive deficits, decreased amyloid deposition, and reversed central cholinergic dysfunction that were caused by Aβ 1-42 in rats. Acteoside and isoacteoside further decreased extracellular Aβ 1-40 production and restored the cell viability that was decreased by Aβ 1-42 in SH-SY5Y cells. Acteoside and isoacteoside also promoted Aβ 1-40 degradation and inhibited Aβ 1-42 oligomerization in vitro. However, the memory-improving and cytoprotective effects of isoacteoside exceeded those of acteoside. Isoacteoside promoted exploratory behavior and restored cortical and hippocampal dopamine levels, but acteoside did not. We suggest that acteoside and isoacteoside ameliorated the cognitive dysfunction that was caused by Aβ 1-42 by blocking amyloid deposition via preventing amyloid oligomerization, and reversing central neuronal function via counteracting amyloid cytotoxicity.

Dig1 protects against locomotor and biochemical dysfunctions provoked by Roundup

BACKGROUND

Plant medicinal extracts may be claimed to prevent or cure chemical intoxications. Few of these are tested for their mechanisms of actions in vivo and for their cellular impacts. In 2011, we demonstrated that hepatic cell mortality induced by environmentally realistic levels of the widely used herbicide Roundup (R) in vitro can be almost entirely prevented by plant extracts called Dig1 (D, Digeodren).

METHODS

We tested the in vivo effects of D alone (1.2 ml/kg bw/d), but also prior to and during 8 days of R intoxication (at 135 mg/kg bw/d) in a total of 4 groups of 40 adult Sprague-Dawley male rats each. After treatments, horizontal and vertical locomotor activities of the animals were measured by use of actimeters. Brain, liver, kidneys, heart and testes were collected and weighted. Body weights as well as feed and water consumption were recorded. Proteins, creatinine, urea, phosphate, potassium, sodium, calcium, chloride ions, testosterone, estradiol, AST and ALT were measured in serum. In liver S9 fractions, GST, GGT, and CYP450 (1A2, 2C9, 2C19, 2D6, 3A4) were assessed.

RESULTS

D did not have any physiological or biochemical observable impact alone at 2 %. Out of a total of 29 measured parameters, 8 were significantly affected by R absorption within only 8 days. On these 8 parameters, only 2 were not restored by D (GGT activity and plasmatic phosphate), 5 were totally restored (horizontal and vertical locomotor activities, CYP2D6 activity, plasmatic Na + and estradiol), and the 6th was almost restored (plasmatic K+). The specificities of the toxic effects of R and of the therapeutic effects of D treatment were thus demonstrated, both at the behavioural and biochemical levels.

CONCLUSIONS

D, without any side effect observable in these conditions, presented strong preventive and therapeutic properties in vivo after a short-term intoxication by the widely used pesticide Roundup.

Exploratory decision-making as a function of lifelong experience, not cognitive decline

Older adults perform worse than younger adults in some complex decision-making scenarios, which is commonly attributed to age-related declines in striatal and frontostriatal processing. Recently, this popular account has been challenged by work that considered how older adults' performance may differ as a function of greater knowledge and experience, and by work showing that, in some cases, older adults outperform younger adults in complex decision-making tasks. In light of this controversy, we examined the performance of older and younger adults in an exploratory choice task that is amenable to model-based analyses and ostensibly not reliant on prior knowledge. Exploration is a critical aspect of decision-making poorly understood across the life span. Across 2 experiments, we addressed (a) how older and younger adults differ in exploratory choice and (b) to what extent observed differences reflect processing capacity declines. Model-based analyses suggested that the strategies used by the 2 groups were qualitatively different, resulting in relatively worse performance for older adults in 1 decision-making environment but equal performance in another. Little evidence was found that differences in processing capacity drove performance differences. Rather the results suggested that older adults' performance might result from applying a strategy that may have been shaped by their wealth of real-word decision-making experience. While this strategy is likely to be effective in the real world, it is ill suited to some decision environments. These results underscore the importance of taking into account effects of experience in aging studies, even for tasks that do not obviously tap past experiences.