Behavioral reactions reflecting differential reward expectations in monkeys

Development of an assessment method for freely moving nonhuman primates' eating behavior using manual and deep learning analysis

Purpose

Although eating is imperative for survival, few comprehensive methods have been developed to assess freely moving nonhuman primates' eating behavior. In the current study, we distinguished eating behavior into appetitive and consummatory phases and developed nine indices to study them using manual and deep learning-based (DeepLabCut) techniques.

Method

The indices were utilized to three rhesus macaques by different palatability and hunger levels to validate their utility. To execute the experiment, we designed the eating behavior cage and manufactured the artificial food. The total number of trials was 3, with 1 trial conducted using natural food and 2 trials using artificial food.

Result

As a result, the indices of highest utility for hunger effect were approach frequency and consummatory duration. Appetitive composite score and consummatory duration showed the highest utility for palatability effect. To elucidate the effects of hunger and palatability, we developed 2D visualization plots based on manual indices. These 2D visualization methods could intuitively depict the palatability perception and hunger internal state. Furthermore, the developed deep learning-based analysis proved accurate and comparable with manual analysis. When comparing the time required for analysis, deep learning-based analysis was 24-times faster than manual analysis. Moreover, temporal and spatial dynamics were visualized via manual and deep learning-based analysis. Based on temporal dynamics analysis, the patterns were classified into four categories: early decline, steady decline, mid-peak with early incline, and late decline. Heatmap of spatial dynamics and trajectory-related visualization could elucidate a consumption posture and a higher spatial occupancy of food zone in hunger and with palatable food.

Discussion

Collectively, this study describes a newly developed and validated multi-phase method for assessing freely moving nonhuman primate eating behavior using manual and deep learning-based analyses. These effective tools will prove valuable in food reward (palatability effect) and homeostasis (hunger effect) research.

State-transition-free reinforcement learning in chimpanzees (Pan troglodytes)

The outcome of an action often occurs after a delay. One solution for learning appropriate actions from delayed outcomes is to rely on a chain of state transitions. Another solution, which does not rest on state transitions, is to use an eligibility trace (ET) that directly bridges a current outcome and multiple past actions via transient memories. Previous studies revealed that humans (Homo sapiens) learned appropriate actions in a behavioral task in which solutions based on the ET were effective but transition-based solutions were ineffective. This suggests that ET may be used in human learning systems. However, no studies have examined nonhuman animals with an equivalent behavioral task. We designed a task for nonhuman animals following a previous human study. In each trial, participants chose one of two stimuli that were randomly selected from three stimulus types: a stimulus associated with a food reward delivered immediately, a stimulus associated with a reward delivered after a few trials, and a stimulus associated with no reward. The presented stimuli did not vary according to the participants' choices. To maximize the total reward, participants had to learn the value of the stimulus associated with a delayed reward. Five chimpanzees (Pan troglodytes) performed the task using a touchscreen. Two chimpanzees were able to learn successfully, indicating that learning mechanisms that do not depend on state transitions were involved in the learning processes. The current study extends previous ET research by proposing a behavioral task and providing empirical data from chimpanzees.

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Expectations are often dynamic: sports fans know that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here we present behavioral and electrophysiological evidence of sub-second changes in expectations using slot machines as a case study. In Study 1, we demonstrate that EEG signal before the slot machine stops varies based on proximity to winning. Study 2 introduces a behavioral paradigm to measure dynamic expectations via betting, and shows that expectation trajectories vary as a function of winning proximity. Notably, these expectation trajectories parallel Study 1's EEG activity. Studies 3 (EEG) and 4 (behavioral) replicate these findings in the loss domain. These four studies provide compelling evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our research opens promising avenues for understanding the dynamic nature of reward expectations and their impact on cognitive processes.

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Expectations are often dynamic: any sports fan knows that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here, using slot machines as a case study, we provide parallel behavioral and electrophysiological evidence of sub-second moment-to-moment changes in expectations. In Study 1, we show that the dynamics of the EEG signal before the slot machine stopped differed depending on the nature of the outcome, including not only whether the participant won or lost, but also how close they came to winning. In line with our predictions, Near Win Before outcomes (the slot machine stops one item before a match) were similar to Wins, but different than Near Win After (the machine stops one item after a match) and Full Miss (the machine stops two or three items from a match). In Study 2, we designed a novel behavioral paradigm to measure moment-to-moment changes in expectations via dynamic betting. We found that different outcomes also elicited unique expectation trajectories in the deceleration phase. Notably, these behavioral expectation trajectories paralleled Study 1's EEG activity in the last second prior to the machine's stop. In Studies 3 (EEG) and 4 (behavior) we replicated these findings in the loss domain where a match entails a loss. Again, we found a significant correlation between behavioral and EEG results. These four studies provide the first evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our findings open up new avenues for studying the ongoing dynamics of reward expectations and their role in healthy and unhealthy cognition.

Glucagon-like peptide-1, a matter of taste?

Understanding of gustatory coding helps to predict, and perhaps even modulate the ingestive decision circuitry, especially when eating behaviour becomes dysfunctional. Preclinical research demonstrated that glucagon like peptide 1 (GLP-1) is locally synthesized in taste bud cells in the tongue and that GLP-1 receptor exists on the gustatory nerves in close proximity to GLP-1 containing taste bud cells. In humans, the tongue has not yet been addressed as clinically relevant target for GLP-1 based therapies. The primary aim of the current review was to elaborate on the role of GLP- 1 in mammalian gustatory system, in particular in the perception of sweet. Secondly, we aimed to explore what modulates gustatory coding and whether the GLP-1 based therapies might be involved in regulation of taste perception. We performed a series of PubMed, Medline and Embase databases systemic searches. The Population-Intervention-Comparison-Outcome (PICO) framework was used to identify interventional studies. Based on the available data, GLP-1 is specifically involved in the perception of sweet. Aging, diabetes and obesity are characterized by diminished taste and sweet perception. Calorie restriction and bariatric surgery are associated with a diminished appreciation of sweet food. GLP-1 receptor agonists (RAs) modulate food preference, yet its modulatory potential in gustatory coding is currently unknown. Future studies should explore whether GLP-1 RAs modulate taste perception to the extent that changes of food preference and consumption ensue.

Electrical stimulation of the macaque ventral tegmental area drives category-selective learning without attention

Perception improves by repeated practice with visual stimuli, a phenomenon known as visual perceptual learning (VPL). The interplay of attentional and neuromodulatory reward signals is hypothesized to cause these behavioral and associated neuronal changes, although VPL can occur without attention (i.e., task-irrelevant VPL). In addition, task-relevant VPL can be category-selective for simple attended oriented stimuli. Yet, it is unclear whether category-selective task-irrelevant VPL occurs and which brain centers mediate underlying forms of adult cortical plasticity. Here, we show that pairing subliminal complex visual stimuli (faces and bodies) with electrical microstimulation of the ventral tegmental area (VTA-EM) causes category-selective task-irrelevant VPL. These perceptual improvements are accompanied by fMRI signal changes in early and late visual and frontal areas, as well as the cerebellum, hippocampus, claustrum, and putamen. In conclusion, Pavlovian pairing of unattended complex stimuli with VTA-EM causes category-selective learning accompanied by changes of cortical and subcortical neural representations in macaques.

Glucagon-Like Peptide 1 and Taste Perception: From Molecular Mechanisms to Potential Clinical Implications

Preclinical studies provided some important insights into the action of glucagon-like peptide 1 (GLP-1) in taste perception. This review examines the literature to uncover some molecular mechanisms and connections between GLP-1 and the gustatory coding. Local GLP-1 production in the taste bud cells, the expression of GLP-1 receptor on the adjacent nerves, a functional continuum in the perception of sweet chemicals from the gut to the tongue and an identification of GLP-1 induced signaling pathways in peripheral and central gustatory coding all strongly suggest that GLP-1 is involved in the taste perception, especially sweet. However, the impact of GLP-1 based therapies on gustatory coding in humans remains largely unaddressed. Based on the molecular background we encourage further exploration of the tongue as a new treatment target for GLP-1 receptor agonists in clinical studies. Given that pharmacological manipulation of gustatory coding may represent a new potential strategy against obesity and diabetes, the topic is of utmost clinical relevance.

Glucagon-like peptide-1, a matter of taste?

Understanding of gustatory coding helps to predict, and perhaps even modulate the ingestive decision circuitry, especially when eating behaviour becomes dysfunctional. Preclinical research demonstrated that glucagon like peptide 1 (GLP-1) is locally synthesized in taste bud cells in the tongue and that GLP-1 receptor exists on the gustatory nerves in close proximity to GLP-1 containing taste bud cells. In humans, the tongue has not yet been addressed as clinically relevant target for GLP-1 based therapies. The primary aim of the current review was to elaborate on the role of GLP- 1 in mammalian gustatory system, in particular in the perception of sweet. Secondly, we aimed to explore what modulates gustatory coding and whether the GLP-1 based therapies might be involved in regulation of taste perception. We performed a series of PubMed, Medline and Embase databases systemic searches. The Population-Intervention-Comparison-Outcome (PICO) framework was used to identify interventional studies. Based on the available data, GLP-1 is specifically involved in the perception of sweet. Aging, diabetes and obesity are characterized by diminished taste and sweet perception. Calorie restriction and bariatric surgery are associated with a diminished appreciation of sweet food. GLP-1 receptor agonists (RAs) modulate food preference, yet its modulatory potential in gustatory coding is currently unknown. Future studies should explore whether GLP-1 RAs modulate taste perception to the extent that changes of food preference and consumption ensue.

Sound as a stimulus in associative learning for heat stress in Arabidopsis

Plants are analogous to animals by responding physiologically and phenotypically to environmental changes. Until recently, the meaning of sound in the plant’s life remains undiscovered. In this study, we investigated the role of music in response to heat stress and its application in memory and associative learning for stress tolerance in Arabidopsis. Significant upregulation of heat-responsive genes (HSFA3, SMXL7, and ATHSP101) in response to music suggests music has an advantage during heat stress. Moreover, the defensive conditioning experiment showed that plant learns to associate music with stress (heat) and elicit better response compared to music alone. Two heat-responsive genes, HSFA3 and ATCTL1, which are well known for their interaction and regulation of an array of heat shock proteins were found to play a key role in associative learning for heat stress in Arabidopsis. Our experiment highlights the application of sound in plant conditioning and as a stress reliever. Nonetheless, the persistence of memory awaits further experiments. We foresee the potential of artificial sound as an environment-friendly stimulus in conditioning the crops for upcoming stresses and reduce the yield loss, as an alternative to breeding and genetic modifications.

Food Craving, Seeking, and Consumption Behaviors: Conceptual Phases and Assessment Methods Used in Animal and Human Studies

What drives us to eat? It is one of the most fundamental questions in the obesity research field which have been investigated for centuries. Numerous novel in vivo technologies in the neuroscience field allows us to reevaluate the multiple components and phases of food-related behaviors. Focused on the cognitive, executive, behavioral and temporal aspects, food-related behaviors can be distinguished into appetitive phase (food craving→food seeking) and consummatory phase (food consumption). Food craving phase is an internal state or stage in which the animal has the motivation to eat the food but there is no actual food specific behaviors or actions. Food seeking phase entails repeated behaviors with a food searching purpose until the animal discovers the food (or food-related cue) and the approach behavior stage after the discovery of food. Food consumption phase is the step that the animal grabs, chews and intake the food. This review will specifically focus on characteristics and evaluation methods for each phase of food-related behavior in rodent, non-human primates and human.

The cost of obtaining rewards enhances the reward prediction error signal of midbrain dopamine neurons

Midbrain dopamine neurons are known to encode reward prediction errors (RPE) used to update value predictions. Here, we examine whether RPE signals coded by midbrain dopamine neurons are modulated by the cost paid to obtain rewards, by recording from dopamine neurons in awake behaving monkeys during performance of an effortful saccade task. Dopamine neuron responses to cues predicting reward and to the delivery of rewards were increased after the performance of a costly action compared to a less costly action, suggesting that RPEs are enhanced following the performance of a costly action. At the behavioral level, stimulus-reward associations are learned faster after performing a costly action compared to a less costly action. Thus, information about action cost is processed in the dopamine reward system in a manner that amplifies the following dopamine RPE signal, which in turn promotes more rapid learning under situations of high cost.

Rewards that require high effort tend to be preferred over those that require low effort. Here, the authors show how the effort of obtaining rewards affects reward-related activity of dopamine neurons, and in turn the speed of learning stimulus-reward associations.

When the simplest voluntary decisions appear patently suboptimal

Rahnev & Denison (R&D) catalog numerous experiments in which performance deviates, often in subtle ways, from the theoretical ideal. We discuss an extreme case, an elementary behavior (reactive saccades to single targets) for which a simple contextual manipulation results in responses that are dramatically different from those expected based on reward maximization - and yet are highly informative and amenable to mechanistic examination.

An investigation of variety effects during operant responding in the rat utilizing different reward flavors

Humans and nonhuman animals respond to food diversity by increasing intake and appetitive behaviors, reflecting enhanced valuation for items presented in the context of variety. Previous work on food variety effects has posited two main explanatory mechanisms. Variety could slow habituation processes by decreasing exposure to a single food item or could elicit contrast effects in which comparisons between items impact relative valuation. This study used three flavors of sucrose rewards to investigate rats' responses to qualitative reward variety in different variety contexts: low (2 flavors) and high (3 flavors) conditions. Control sessions used only a single flavored pellet (no variety). Animals were tested in low (10 trials), moderate (20 trials) and high consumption (30 trials) sessions. A trial within each session was defined as completion of the operant response and acquisition of the reward pellet. Cues associated with flavors were used to examine predictability and between-trial ('micro') variety. Indicators of a variety effect were found including faster responding for rewards during the variety context compared to an initial control (no variety) context. This decrease in response latency continued to be observed for some measures in post-variety control contexts. The most robust statistical finding of variety effects was found using trial-by-trial analysis, with shorter response latencies obtained for trials with outcomes differing from the preceding trial compared to successive trials with identical outcomes. These results have implications for understanding how a general reward context like variety impacts behavior, and for informing clinical approaches focusing on motivation and eating disorders.

Anticipatory Behavior of the Clonal Plant Fragaria vesca

Active foraging for patchy resources is a crucial feature of many clonal plant species. It has been recently shown that plants' foraging for resources can be facilitated by anticipatory behavior via association of resource position with other environmental cues. We therefore tested whether clones of Fragaria vesca are able to associate and memorize positions of soil nutrients with particular light intensity, which will consequently enable them anticipating nutrients in new environment. We trained clones of F. vesca for nutrients to occur either in shade or in light. Consequently, we tested their growth response to differing light intensity in the absence of soil nutrients. We also manipulated epigenetic status of a subset of the clones to test the role of DNA methylation in the anticipatory behavior. Clones of F. vesca were able to associate presence of nutrients with particular light intensity, which enabled them to anticipate nutrient positions in the new environment based on its light intensity. Clones that had been trained for nutrients to occur in shade increased placement of ramets to shade whereas clones trained for nutrients to occur in light increased biomass of ramets in light. Our study clearly shows that the clonal plant F. vesca is able to relate two environmental factors, light and soil nutrients, and use this connection in anticipatory behavior. We conclude that anticipatory behavior can substantially improve the ability of clonal plants to utilize scarce and unevenly distributed resources.

Using preferred fluids and different reward schedules to motivate rhesus macaques (Macaca mulatta) in cognitive tasks

Rhesus macaques (Macaca mulatta) used in behavioural neuroscience are often required to complete cognitively complex tasks, for which a high level of motivation is essential. To induce motivation, researchers may implement fluid-restriction protocols, whereby freely available water is limited, such that fluid can be used as a reward in the laboratory. A variety of different rewards and schedules are used, but there exists a lack of data assessing their effectiveness. In this study, we aimed to quantify fluid preference in rhesus macaques and to use these preferences to compare the motivational quality of different reward schedules: the monkey's previous reward (i.e. the fluid used to reward them in past studies), their new preferred reward, a variable schedule of previous and preferred reward, and a choice between the previous and preferred rewards. We found that it may be possible to reduce the level of restriction if an adequately motivating preferred reward is identified, but that this is dependent on the animal. Each monkey responded differently to both the fluid-preference assessments and to the different reward schedules. As such, monkeys should not be subject to 'blanket' protocols but should be assessed individually to maintain adequate scientific data collection at the least severe level of fluid restriction.

Paradigm Shift in Sensorimotor Control Research and Brain Machine Interface Control: The Influence of Context on Sensorimotor Representations

Neural activity in the primary motor cortex (M1) is known to correlate with movement related variables including kinematics and dynamics. Our recent work, which we believe is part of a paradigm shift in sensorimotor research, has shown that in addition to these movement related variables, activity in M1 and the primary somatosensory cortex (S1) are also modulated by context, such as value, during both active movement and movement observation. Here we expand on the investigation of reward modulation in M1, showing that reward level changes the neural tuning function of M1 units to both kinematic as well as dynamic related variables. In addition, we show that this reward-modulated activity is present during brain machine interface (BMI) control. We suggest that by taking into account these context dependencies of M1 modulation, we can produce more robust BMIs. Toward this goal, we demonstrate that we can classify reward expectation from M1 on a movement-by-movement basis under BMI control and use this to gate multiple linear BMI decoders toward improved offline performance. These findings demonstrate that it is possible and meaningful to design a more accurate BMI decoder that takes reward and context into consideration. Our next step in this development will be to incorporate this gating system, or a continuous variant of it, into online BMI performance.

Factors Influencing Individual Variation in Farm Animal Cognition and How to Account for These Statistically

For farmed species, good health and welfare is a win-win situation: both the animals and producers can benefit. In recent years, animal welfare scientists have embraced cognitive sciences to rise to the challenge of determining an animal's internal state in order to better understand its welfare needs and by extension, the needs of larger groups of animals. A wide range of cognitive tests have been developed that can be applied in farmed species to assess a range of cognitive traits. However, this has also presented challenges. Whilst it may be expected to see cognitive variation at the species level, differences in cognitive ability between and within individuals of the same species have frequently been noted but left largely unexplained. Not accounting for individual variation may result in misleading conclusions when the results are applied both at an individual level and at higher levels of scale. This has implications both for our fundamental understanding of an individual's welfare needs, but also more broadly for experimental design and the justification for sample sizes in studies using animals. We urgently need to address this issue. In this review, we will consider the latest developments on the causes of individual variation in cognitive outcomes, such as the choice of cognitive test, sex, breed, age, early life environment, rearing conditions, personality, diet, and the animal's microbiome. We discuss the impact of each of these factors specifically in relation to recent work in farmed species, and explore the future directions for cognitive research in this field, particularly in relation to experimental design and analytical techniques that allow individual variation to be accounted for appropriately.

Neural encoding of choice during a delayed response task in primate striatum and orbitofrontal cortex

Reward outcomes are available in many diverse situations and all involve choice. If there are multiple outcomes each rewarding, then decisions regarding relative value lead to choosing one over another. Important factors related to choice context should be encoded and utilized for this form of adaptive choosing. These factors can include the number of alternatives, the pacing of choice behavior and the possibility to reverse one's choice. An essential step in understanding if the context of choice is encoded is to directly compare choice with a context in which choice is absent. Neural activity in orbitofrontal cortex and striatum encodes potential value parameters related to reward quality and quantity as well as relative preference. We examined how neural activations in these brain regions are sensitive to choice situations and potentially involved in a prediction for the upcoming outcome selection. Neural activity was recorded and compared between a two-choice spatial delayed response task and an imperative 'one-option' task. Neural activity was obtained that extended from the instruction cue to the movement similar to previous work utilizing the identical imperative task. Orbitofrontal and striatal neural responses depended upon the decision about the choice of which reward to collect. Moreover, signals to predictive instruction cues that precede choice were selective for the choice situation. These neural responses could reflect chosen value with greater information on relative value of individual options as well as encode choice context itself embedded in the task as a part of the post-decision variable.

Differential effects of real versus hypothetical monetary reward magnitude on risk-taking behavior and brain activity

Human decisions are more easily affected by a larger amount of money than a smaller one. Although numerous studies have used hypothetical money as incentives to motivate human behavior, the validity of hypothetical versus real monetary rewards remains controversial. In the present study, we used event-related potential (ERP) with the balloon analogue risk task to investigate how magnitudes of real and hypothetical monetary rewards modulate risk-taking behavior and feedback-related negativity (FRN). Behavioral data showed that participants were more risk averse after negative feedback with increased magnitude of real monetary rewards, while no behavior differences were observed between large and small hypothetical monetary rewards. Similarly, ERP data showed a larger FRN in response to negative feedback during risk taking with large compared to small real monetary rewards, while no FRN differences were observed between large and small hypothetical monetary rewards. Moreover, FRN amplitude differences correlated with risk-taking behavior changes from small to large real monetary rewards, while such correlation was not observed for hypothetical monetary rewards. These findings suggest that the magnitudes of real and hypothetical monetary rewards have differential effects on risk-taking behavior and brain activity. Real and hypothetical money incentives may have different validity for modulating human decisions.

Effects of the serotonin 2C receptor agonist WAY163909 on the abuse-related effects and mesolimbic dopamine neurochemistry induced by abused stimulants in rhesus monkeys

RATIONALE

Accumulating evidence shows that the serotonergic system plays a major role in psychostimulant abuse through its interactions with the dopaminergic system. Studies indicate that serotonin 5-HT2C receptors are one of the main classes of receptors involved in mediating the influence of serotonin in drug abuse.

OBJECTIVE

The aim of the present study was to evaluate the effects of the selective serotonin 5-HT2C receptor agonist WAY163909 on the behavioral neuropharmacology of cocaine and methamphetamine in adult rhesus macaques.

METHODS

Cocaine or methamphetamine self-administration and reinstatement were evaluated under second-order and fixed-ratio schedules of reinforcement, respectively. Cocaine- and methamphetamine-induced increases in dopamine were assessed through in vivo microdialysis targeting the nucleus accumbens.

RESULTS

Pretreatment with WAY163909 dose-dependently attenuated cocaine and methamphetamine self-administration and drug-induced reinstatement of extinguished behavior previously maintained by cocaine or methamphetamine delivery. In an additional experiment, WAY163909 induced a dose-dependent attenuation of cocaine- or methamphetamine-induced dopamine overflow in the nucleus accumbens.

CONCLUSIONS

Our data indicate that selective 5-HT2C receptor activation decreases drug intake and drug-seeking behavior in nonhuman primate models of psychostimulant abuse through neurochemical mechanisms involved in the modulation of mesolimbic dopamine.

Judgement bias in pigs is independent of performance in a spatial holeboard task and conditional discrimination learning

Biases in judgement of ambiguous stimuli, as measured in a judgement bias task, have been proposed as a measure of the valence of affective states in animals. We recently suggested a list of criteria for behavioural tests of emotion, one of them stating that responses on the task used to assess emotionality should not be confounded by, among others, differences in learning capacity, i.e. must not simply reflect the cognitive capacity of an animal. We performed three independent studies in which pigs acquired a spatial holeboard task, a free choice maze which simultaneously assesses working memory and reference memory. Next, pigs learned a conditional discrimination between auditory stimuli predicting a large or small reward, a prerequisite for assessment of judgement bias. Once pigs had acquired the conditional discrimination task, optimistic responses to previously unheard ambiguous stimuli were measured in the judgement bias task as choices indicating expectation of the large reward. We found that optimism in the judgement bias task was independent of all three measures of learning and memory indicating that the performance is not dependent on the pig's cognitive abilities. These results support the use of biases in judgement as proxy indicators of emotional valence in animals.

A Dual-Mode Magnetic-Acoustic System for Monitoring Fluid Intake Behavior in Animals

We have developed an unobtrusive magnetic-acoustic fluid intake monitoring (MAFIM) system using a conventional stainless-steel roller-ball nipple to measure licking and drinking behavior in animals. Movements of a small permanent magnetic tracer attached to stainless-steel roller balls that operate as a tongue-actuated valve are sensed by a pair of three-axial magnetometers, and transformed into a time-series indicating the status of the ball (up or down), using a Gaussian mixture model based data-driven classifier. The sounds produced by the rise and fall of the roller balls are also recorded and classified to substantiate the magnetic data by an independent modality for a more robust solution. The operation of the magnetic and acoustic sensors is controlled by an embedded system, communicating via Universal Serial Bus (USB) with a custom-designed user interface, running on a PC. The MAFIM system has been tested in vivo with minipigs, accurately measuring various drinking parameters and licking patterns without constraints imposed by current lick monitoring systems, such as nipple access, animal-nipple contact, animal training, and complex parameter settings.

Learning by Association in Plants

In complex and ever-changing environments, resources such as food are often scarce and unevenly distributed in space and time. Therefore, utilizing external cues to locate and remember high-quality sources allows more efficient foraging, thus increasing chances for survival. Associations between environmental cues and food are readily formed because of the tangible benefits they confer. While examples of the key role they play in shaping foraging behaviours are widespread in the animal world, the possibility that plants are also able to acquire learned associations to guide their foraging behaviour has never been demonstrated. Here we show that this type of learning occurs in the garden pea, Pisum sativum. By using a Y-maze task, we show that the position of a neutral cue, predicting the location of a light source, affected the direction of plant growth. This learned behaviour prevailed over innate phototropism. Notably, learning was successful only when it occurred during the subjective day, suggesting that behavioural performance is regulated by metabolic demands. Our results show that associative learning is an essential component of plant behaviour. We conclude that associative learning represents a universal adaptive mechanism shared by both animals and plants.

Performance error-related activity in monkey striatum during social interactions

Monitoring our performance is fundamental to motor control while monitoring other’s performance is fundamental to social coordination. The striatum is hypothesized to play a role in action selection, action initiation, and action parsing, but we know little of its role in performance monitoring. Furthermore, the striatum contains neurons that respond to own and other’s actions. Therefore, we asked if striatal neurons signal own and conspecific’s performance errors. Two macaque monkeys sitting across a touch-sensitive table in plain view of each other took turns performing a simple motor task to obtain juice rewards while we recorded single striatal neurons from one monkey at a time. Both monkeys made more errors after individually making an error but made fewer errors after a conspecific error. Thus, monkeys’ behavior was influenced by their own and their conspecific’s past behavior. A population of striatal neurons responded to own and conspecific’s performance errors independently of a negative reward prediction error signal. Overall, these data suggest that monkeys are influenced by social errors and that striatal neurons signal performance errors. These signals might be important for social coordination, observational learning and adjusting to an ever-changing social landscape.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Primate amygdala neurons evaluate the progress of self-defined economic choice sequences

The amygdala is a prime valuation structure yet its functions in advanced behaviors are poorly understood. We tested whether individual amygdala neurons encode a critical requirement for goal-directed behavior: the evaluation of progress during sequential choices. As monkeys progressed through choice sequences toward rewards, amygdala neurons showed phasic, gradually increasing responses over successive choice steps. These responses occurred in the absence of external progress cues or motor preplanning. They were often specific to self-defined sequences, typically disappearing during instructed control sequences with similar reward expectation. Their build-up rate reflected prospectively the forthcoming choice sequence, suggesting adaptation to an internal plan. Population decoding demonstrated a high-accuracy progress code. These findings indicate that amygdala neurons evaluate the progress of planned, self-defined behavioral sequences. Such progress signals seem essential for aligning stepwise choices with internal plans. Their presence in amygdala neurons may inform understanding of human conditions with amygdala dysfunction and deregulated reward pursuit.

DOI:http://dx.doi.org/10.7554/eLife.18731.001

Fractionating choice: A study on reward discrimination, preference, and relative valuation in the rat (Rattus norvegicus)

Choice behavior combines discrimination between distinctive outcomes, preference for specific outcomes and relative valuation of comparable outcomes. Previous work has focused on 1 component (i.e., preference) disregarding other influential processes that might provide a more complete understanding. Animal models of choice have been explored primarily utilizing extensive training, limited freedom for multiple decisions and sparse behavioral measures constrained to a single phase of motivated action. The present study used a paradigm that combines different elements of previous methods with the goal to distinguish among components of choice and explore how well components match predictions based on risk-sensitive foraging strategies. In order to analyze discrimination and relative valuation, it was necessary to have an option that shifted and an option that remained constant. Shifting outcomes among weeks included a change in single-option outcome (0 to 1 to 2 pellets) or a change in mixed-option outcome (0 or 5 to 0 or 3 to 0 or 1 pellets). Constant outcomes among weeks were also mixed-option (0 or 3 pellets) or single-option (1 pellet). Shifting single-option outcomes among weeks led to better discrimination, more robust preference and significant incentive contrast effects for the alternative outcome. Shifting multioptions altered choice components and led to dissociations among discrimination, preference, and reduced contrast effects. During extinction, all components were impacted with the greatest deficits during the shifting mixed-option outcome sessions. Results suggest choice behavior can be optimized for 1 component but suboptimal for others depending upon the complexity of alterations in outcome value between options. (PsycINFO Database Record

A neuronal reward inequity signal in primate striatum

Primates are social animals, and their survival depends on social interactions with others. Especially important for social interactions and welfare is the observation of rewards obtained by other individuals and the comparison with own reward. The fundamental social decision variable for the comparison process is reward inequity, defined by an asymmetric reward distribution among individuals. An important brain structure for coding reward inequity may be the striatum, a component of the basal ganglia involved in goal-directed behavior. Two rhesus monkeys were seated opposite each other and contacted a touch-sensitive table placed between them to obtain specific magnitudes of reward that were equally or unequally distributed among them. Response times in one of the animals demonstrated differential behavioral sensitivity to reward inequity. A group of neurons in the striatum showed distinct signals reflecting disadvantageous and advantageous reward inequity. These neuronal signals occurred irrespective of, or in conjunction with, own reward coding. These data demonstrate that striatal neurons of macaque monkeys sense the differences between other's and own reward. The neuronal activities are likely to contribute crucial reward information to neuronal mechanisms involved in social interactions.

Modulation of neuronal activity by reward identity in the monkey subthalamic nucleus

The subthalamic nucleus (STN) has been argued to be an important component of reward-sensitive basal ganglia circuitry. This view is especially supported by the behavioral changes observed after STN inactivation, which could reflect impairments in the motivational control of action. However, it is still unclear how the STN integrates reward information and to what extent such integration correlates with behavior. In this study, the response properties of STN neurons in monkeys performing reaching movements with a cue predicting the identity of an upcoming liquid reward (juice or water) were investigated. Although the timing of movements reliably indicated that monkeys had greater motivation for juice than water, rarely did task-related changes in neuronal activity depend on the nature of the expected reward. Conversely, when presented with a choice of selecting a response that leads to juice or water delivery, animals showed a clear preference for juice and more than half of the neurons were differentially modulated dependent on the reward obtained, mostly after the monkeys's overt choice of action. Under such circumstances, an increase in activity specifically followed the action outcomes across the population of neurons when monkeys failed to choose the juice reward. These results indicate that STN neurons encode whether or not a preferred reward had been received when a choice between response alternatives is required. This differential neuronal activity might reflect the participation of the STN in evaluating the reward value of chosen actions, thus highlighting its contribution to decision-making processes.

Relative reward effects on operant behavior: Incentive contrast, induction and variety effects

Comparing different rewards automatically produces dynamic relative outcome effects on behavior. Each new outcome exposure is to an updated version evaluated relative to alternatives. Relative reward effects include incentive contrast, positive induction and variety effects. The present study utilized a novel behavioral design to examine relative reward effects on a chain of operant behavior using auditory cues. Incentive contrast is the most often examined effect and focuses on increases or decreases in behavioral performance after value upshifts (positive) or downshifts (negative) relative to another outcome. We examined the impact of comparing two reward outcomes in a repeated measures design with three sessions: a single outcome and a mixed outcome and a final single outcome session. Relative reward effects should be apparent when comparing trials for the identical outcome between the single and mixed session types. An auditory cue triggered a series of operant responses (nosepoke-leverpress-food retrieval), and we measured possible contrast effects for different reward magnitude combinations. We found positive contrast for trials with the greatest magnitude differential but positive induction or variety effects in other combinations. This behavioral task could be useful for analyzing environmental or neurobiological factors involved in reward comparisons, decision-making and choice during instrumental, goal-directed action.

Motivation and affective judgments differentially recruit neurons in the primate dorsolateral prefrontal and anterior cingulate cortex

The judgment of whether to accept or to reject an offer is determined by positive and negative affect related to the offer, but affect also induces motivational responses. Rewarding and aversive cues influence the firing rates of many neurons in primate prefrontal and cingulate neocortical regions, but it still is unclear whether neurons in these regions are related to affective judgment or to motivation. To address this issue, we recorded simultaneously the neuronal spike activities of single units in the dorsolateral prefrontal cortex (dlPFC) and the anterior cingulate cortex (ACC) of macaque monkeys as they performed approach-avoidance (Ap-Av) and approach-approach (Ap-Ap) decision-making tasks that can behaviorally dissociate affective judgment and motivation. Notably, neurons having activity correlated with motivational condition could be distinguished from neurons having activity related to affective judgment, especially in the Ap-Av task. Although many neurons in both regions exhibited similar, selective patterns of task-related activity, we found a larger proportion of neurons activated in low motivational conditions in the dlPFC than in the ACC, and the onset of this activity was significantly earlier in the dlPFC than in the ACC. Furthermore, the temporal onsets of affective judgment represented by neuronal activities were significantly slower in the low motivational conditions than in the other conditions. These findings suggest that motivation and affective judgment both recruit dlPFC and ACC neurons but with differential degrees of involvement and timing.

Planning activity for internally generated reward goals in monkey amygdala neurons

The best rewards are often distant and can only be achieved by planning and decision-making over several steps. We designed a multi-step choice task in which monkeys followed internal plans to save rewards toward self-defined goals. During this self-controlled behavior, amygdala neurons showed future-oriented activity that reflected the animal's plan to obtain specific rewards several trials ahead. This prospective activity encoded crucial components of the animal's plan, including value and length of the planned choice sequence. It began on initial trials when a plan would be formed, reappeared step by step until reward receipt, and readily updated with a new sequence. It predicted performance, including errors, and typically disappeared during instructed behavior. Such prospective activity could underlie the formation and pursuit of internal plans characteristic of goal-directed behavior. The existence of neuronal planning activity in the amygdala suggests that this structure is important in guiding behavior toward internally generated, distant goals.

Emotion and relative reward processing: an investigation on instrumental successive negative contrast and ultrasonic vocalizations in the rat

Incentive contrast effects include changes in behavioral responses after a reward upshift (positive contrast) or downshift (negative contrast). Proposed influences on these behavioral changes are emotional state reactions after experiencing or anticipating a change in reward outcome. Rat ultrasonic vocalizations have been shown to be indicators of emotional state during behavior and anticipatory periods. The objective of the present study was to monitor rodent ultrasounds during incentive contrast using a classical runway procedure called instrumental successive negative contrast. The procedure is one that has been used often to examine incentive relativity because of its reliability in measuring negative contrast effects. Rats were trained to run in the alleyway to receive a high (12 pellets) or low magnitude (1 pellet) outcome. The high magnitude was then shifted to the low and running speeds in the alleyway for the reward and USV emission were compared. Replicating previous work, a negative contrast effect was observed with postshift running speeds significantly slower in the shifted group compared to the unshifted group. USVs did not follow the same pattern with an apparent lack of significant differences between the groups following the reward downshift. We also tested another group of animals using a visual predictive cue in the same runway test. When visual cues predicted high or low magnitude outcome, no incentive contrast was found for the running speeds following an outcome downshift, but a weak contrast effect was observed for the USV emission. These results demonstrate a separation between USVs and behavioral indicators of incentive contrast suggesting that concomitant shifts in negative affect may not be necessary for anticipatory relative reward processes.

Training experience in gestures affects the display of social gaze in baboons' communication with a human

Gaze behaviour, notably the alternation of gaze between distal objects and social partners that accompanies primates' gestural communication is considered a standard indicator of intentionality. However, the developmental precursors of gaze behaviour in primates' communication are not well understood. Here, we capitalized on the training in gestures dispensed to olive baboons (Papio anubis) as a way of manipulating individual communicative experience with humans. We aimed to delineate the effects of such a training experience on gaze behaviour displayed by the monkeys in relation with gestural requests. Using a food-requesting paradigm, we compared subjects trained in requesting gestures (i.e. trained subjects) to naïve subjects (i.e. control subjects) for their occurrences of (1) gaze behaviour, (2) requesting gestures and (3) temporal combination of gaze alternation with gestures. We found that training did not affect the frequencies of looking at the human's face, looking at food or alternating gaze. Hence, social gaze behaviour occurs independently from the amount of communicative experience with humans. However, trained baboons-gesturing more than control subjects-exhibited most gaze alternation combined with gestures, whereas control baboons did not. By reinforcing the display of gaze alternation along with gestures, we suggest that training may have served to enhance the communicative function of hand gestures. Finally, this study brings the first quantitative report of monkeys producing requesting gestures without explicit training by humans (controls). These results may open a window on the developmental mechanisms (i.e. incidental learning vs. training) underpinning gestural intentional communication in primates.

Neural signal for counteracting pre-action bias in the centromedian thalamic nucleus

Most of our daily actions are selected and executed involuntarily under familiar situations by the guidance of internal drives, such as motivation. The behavioral tendency or biasing towards one over others reflects the action-selection process in advance of action execution (i.e., pre-action bias). Facing unexpected situations, however, pre-action bias should be withdrawn and replaced by an alternative that is suitable for the situation (i.e., counteracting bias). To understand the neural mechanism for the counteracting process, we studied the neural activity of the thalamic centromedian (CM) nucleus in monkeys performing GO-NOGO task with asymmetrical or symmetrical reward conditions. The monkeys reacted to GO signal faster in large-reward condition, indicating behavioral bias toward large reward. In contrast, they responded slowly in small-reward condition, suggesting a conflict between internal drive and external demand. We found that neurons in the CM nucleus exhibited phasic burst discharges after GO and NOGO instructions especially when they were associated with small reward. The small-reward preference was positively correlated with the strength of behavioral bias toward large reward. The small-reward preference disappeared when only NOGO action was requested. The timing of activation predicted the timing of action opposed to bias. These results suggest that CM signals the discrepancy between internal pre-action bias and external demand, and mediates the counteracting process—resetting behavioral bias and leading to execution of opposing action.

Motivation enhances visual working memory capacity through the modulation of central cognitive processes

Motivation is well known to enhance working memory (WM) capacity, but the mechanism underlying this effect remains unclear. The WM process can be divided into encoding, maintenance, and retrieval, and in a change detection visual WM paradigm, the encoding and retrieval processes can be subdivided into perceptual and central processing. To clarify which of these segments are most influenced by motivation, we measured ERPs in a change detection task with differential monetary rewards. The results showed that the enhancement of WM capacity under high motivation was accompanied by modulations of late central components but not those reflecting attentional control on perceptual inputs across all stages of WM. We conclude that the "state-dependent" shift of motivation impacted the central, rather than the perceptual functions in order to achieve better behavioral performances.

Goal-oriented, flexible use of numerical operations by monkeys

Previous studies have shown that elementary aspects of numerical abilities have developed in non-human primates. In the present study, we explored the potential for the development of a novel ability in the use of numerical operations by macaque monkeys (Macaca fuscata): adequate selection of a series of numerical actions toward achieving a behavioral goal. We trained monkeys to use a pair of devices to selectively add or subtract items to/from a digital array in order to match a previously viewed sample array. The monkeys determined whether to add or subtract on the basis of the feedback about numerosity given to the monkeys, which was displayed as an outcome of each step of the numerical operation. We also found that monkeys adapted flexibly to changes in the numerical rule that determined the relationship between device use and numerical operation. Our model analysis found that the numerosity-based model was a better fit for the monkeys' performance than was the reward-expectation-based model. Such a capacity for goal-oriented selection of numerical operations suggests a mechanism by which monkeys use numerical representations for purposeful behaviors.

Changes in brain activity in response to problem solving during the abstinence from online game play

BACKGROUND AND AIMS

Several studies have suggested that addictive disorders including substance abuse and pathologic gambling might be associated with dysfunction on working memory and prefrontal activity. We hypothesized that excessive online game playing is associated with deficits in prefrontal cortex function and that recovery from excessive online game playing might improve prefrontal cortical activation in response to working memory stimulation.

METHODS

Thirteen adolescents with excessive online game playing (AEOP) and ten healthy adolescents (HC) agreed to participate in this study. The severity of online game play and playing time were evaluated for a baseline measurement and again following four weeks of treatment. Brain activation in response to working memory tasks (simple and complex calculations) at baseline and subsequent measurements was assessed using BOLD functional magnetic resonance imaging (fMRI).

RESULTS

Compared to the HC subjects, the AEOP participants exhibited significantly greater activity in the right middle occipital gyrus, left cerebellum posterior lobe, left premotor cortex and left middle temporal gyrus in response to working memory tasks during baseline measurements. After four weeks of treatment, the AEOP subjects showed increased activity within the right dorsolateral prefrontal cortex and left occipital fusiform gyrus. After four weeks of treatment, changes in the severity of online game playing were negatively correlated with changes in the mean β value of the right dorsolateral prefrontal cortex in response to complex stimulation.

CONCLUSIONS

We suggest that the effects of online game addiction on working memory may be similar to those observed in patients with substance dependence.

From anticipation to action, the role of dopamine in perceptual decision making: an fMRI-tyrosine depletion study

During simple sensorimotor decision making, neurons in the parietal cortex extract evidence from sensory information provided by visual areas until a decision is reached. Contextual information can bias parietal activity during the task and change the decision-making parameters. One type of contextual information is the availability of reward for correct decisions. We tested the hypothesis that the frontal lobes and basal ganglia use contextual information to bias decision making to maximize reward. Human volunteers underwent functional MRI while making decisions about the motion of dots on a computer monitor. On rewarded trials, subjects responded more slowly by increasing the threshold to decision. Rewarded trials were associated with activation in the ventral striatum and prefrontal cortex in the period preceding coherent dot motion, and the degree of activation predicted the increased decision threshold. Decreasing dopamine transmission, using a tyrosine-depleting amino acid mixture, abolished the reward-related corticostriatal activation and eliminated the correlation between striatal activity and decision threshold. These observations provide direct evidence that some reward-related functional MRI signals in the striatum are the result of dopamine neuron activity and demonstrate that mesolimbic dopamine transmission can influence perceptual and decision-making neural processes engaged to maximize reward harvest.

Ventral striatum encodes past and predicted value independent of motor contingencies

The ventral striatum (VS) is thought to signal the predicted value of expected outcomes. However, it is still unclear whether VS can encode value independently from variables often yoked to value such as response direction and latency. Expectations of high value reward are often associated with a particular action and faster latencies. To address this issue we trained rats to perform a task in which the size of the predicted reward was signaled before the instrumental response was instructed. Instrumental directional cues were presented briefly at a variable onset to reduce accuracy and increase reaction time. Rats were more accurate and slower when a large versus small reward was at stake. We found that activity in VS was high during odors that predicted large reward even though reaction times were slower under these conditions. In addition to these effects, we found that activity before the reward predicting cue reflected past and predicted reward. These results demonstrate that VS can encode value independent of motor contingencies and that the role of VS in goal-directed behavior is not just to increase vigor of specific actions when more is at stake.

Frontal theta and beta synchronizations for monetary reward increase visual working memory capacity

Visual working memory (VWM) capacity is affected by motivational influences; however, little is known about how reward-related brain activities facilitate the VWM systems. To investigate the dynamic relationship between VWM- and reward-related brain activities, we conducted time-frequency analyses using electroencephalograph (EEG) data obtained during a monetary-incentive delayed-response task that required participants to memorize the position of colored disks. In case of a correct answer, participants received a monetary reward (0, 10 or 50 Japanese yen) announced at the beginning of each trial. Behavioral results showed that VWM capacity under high-reward condition significantly increased compared with that under low- or no-reward condition. EEG results showed that frontal theta (6 Hz) amplitudes enhanced during delay periods and positively correlated with VWM capacity, indicating involvement of theta local synchronizations in VWM. Moreover, frontal beta activities (24 Hz) were identified as reward-related activities, because delay-period amplitudes correlated with increases in VWM capacity between high-reward and no-reward conditions. Interestingly, cross-frequency couplings between frontal theta and beta phases were observed only under high-reward conditions. These findings suggest that the functional dynamic linking between VWM-related theta and reward-related beta activities on the frontal regions plays an integral role in facilitating increases in VWM capacity.

Modulation of working memory function by motivation through loss-aversion

Cognitive performance is affected by motivation. Few studies, however, have investigated the neural mechanisms of the influence of motivation through potential monetary punishment on working memory. We employed functional MRI during a delayed recognition task that manipulated top-down control demands with added monetary incentives to some trials in the form of potential losses of bonus money. Behavioral performance on the task was influenced by loss-threatening incentives in the form of faster and more accurate performance. As shown previously, we found enhancement of activity for relevant stimuli occurs throughout all task periods (e.g., stimulus encoding, maintenance, and response) in both prefrontal and visual association cortex. Further, these activation patterns were enhanced for trials with possible monetary loss relative to nonincentive trials. During the incentive cue, the amygdala and striatum showed significantly greater activation when money was at a possible loss on the trial. We also evaluated patterns of functional connectivity between regions responsive to monetary consequences and prefrontal areas responsive to the task. This analysis revealed greater delay period connectivity between and the left insula and prefrontal cortex with possible monetary loss relative to nonincentive trials. Overall, these results reveal that incentive motivation can modulate performance on working memory tasks through top-down signals via amplification of activity within prefrontal and visual association regions selective to processing the perceptual inputs of the stimuli to be remembered.

Cortico-basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors

Most of the human behaviors are executed automatically under familiar circumstances. These behaviors are prepotent in that they take precedence over any other potential alternatives. Yet, humans are also capable of engaging cognitive resources to inhibit such a prepotent behavior and replace it with an alternative controlled behavior in response to an unforeseen situation. This remarkable capability to switch behaviors in a short period of time is the hallmark of executive functions. In this article, we first argue that the prepotent automaticity could emerge at least in three different domains - innate, habitual and motivational. We then review neurophysiological findings on how the brain might realize its switching functions in each domain, primarily by focusing on the monkey oculomotor system as the experimental model. Emerging evidence now suggests that multiple neuronal populations in the shared cortico-basal ganglia network contribute to overriding prepotent eye movement, be its origin innate, habitual or motivational. This consideration suggests the general versatility of the cortico-basal ganglia network as the neural mechanism whereby humans and other animals keep themselves from becoming subservient to reflex, habit and motivational impulses.

Contribution of the amygdala, but not orbitofrontal or medial prefrontal cortices, to the expression of flavour preferences in marmoset monkeys

The development of food preferences contributes to a balanced diet, and involves both innate and learnt factors. By associating flavour cues with the reinforcing properties of the food (i.e. postingestive nutrient cues and innately preferred tastes, such as sweetness), animals acquire individual preferences. How the brain codes and guides selection when the subject has to choose between different palatable foods is little understood. To investigate this issue, we trained common marmoset monkeys (Callithrix jacchus) to respond to abstract visual patterns on a touch-sensitive computer screen to gain access to four different flavoured juices. After preferences were stable, animals received excitotoxic lesions of either the amygdala, the orbitofrontal cortex or the medial prefrontal cortex. Neither the orbitofrontal nor the medial prefrontal cortex lesions affected pre-surgery-expressed flavour preferences or the expression of preferences for novel flavours post-surgery. In contrast, amygdala lesions caused a shift in the preferences for juices expressed pre-surgery such that, post-surgery, juices were chosen according to their overall carbohydrate (simple sugars) content or 'sweetness'. Subsequent tests revealed that amygdala-lesioned animals only expressed juice preferences if they differed in 'sweetness'. Unlike controls, orbitofrontal cortex-lesioned and medial prefrontal cortex-lesioned animals, they were unable to display preferences between juices matched for 'sweetness' i.e. 5% sucrose solutions aromatised with different essential oils. The most parsimonious explanation is that the amygdala contributes to the expression of food preferences based on learnt cues but not those based on an innate preference for sweetness.

Human prosaccades and antisaccades under risk: effects of penalties and rewards on visual selection and the value of actions

Monkey studies report greater activity in the lateral intraparietal area and more efficient saccades when targets coincide with the location of prior reward cues, even when cue location does not indicate which responses will be rewarded. This suggests that reward can modulate spatial attention and visual selection independent of the "action value" of the motor response. Our goal was first to determine whether reward modulated visual selection similarly in humans, and next, to discover whether reward and penalty differed in effect, if cue effects were greater for cognitively demanding antisaccades, and if financial consequences that were contingent on stimulus location had spatially selective effects. We found that motivational cues reduced all latencies, more for reward than penalty. There was an "inhibition-of-return"-like effect at the location of the cue, but unlike the results in monkeys, cue valence did not modify this effect in prosaccades, and the inhibition-of-return effect was slightly increased rather than decreased in antisaccades. When financial consequences were contingent on target location, locations without reward or penalty consequences lost the benefits seen in noncontingent trials, whereas locations with consequences maintained their gains. We conclude that unlike monkeys, humans show reward effects not on visual selection but on the value of actions. The human saccadic system has both the capacity to enhance responses to multiple locations simultaneously, and the flexibility to focus motivational enhancement only on locations with financial consequences. Reward is more effective than penalty, and both interact with the additional attentional demands of the antisaccade task.

Reward expectations in honeybees

The study of expectations of reward helps to understand rules controlling goal-directed behavior as well as decision making and planning. I shall review a series of recent studies focusing on how the food gathering behavior of honeybees depends upon reward expectations. These studies document that free-flying honeybees develop long-term expectations of reward and use them to regulate their investment of energy/time during foraging. Also, they present a laboratory procedure suitable for analysis of neural substrates of reward expectations in the honeybee brain. I discuss these findings in the context of individual and collective foraging, on the one hand, and neurobiology of learning and memory of reward.

Apathy blunts neural response to money in Parkinson's disease

Apathy, defined as a primary deficit in motivation and manifested by the simultaneous diminution in the cognitive and emotional concomitants of goal-directed behavior, is a common and debilitating non-motor symptom of Parkinson's disease (PD). Despite the high prevalence and clinical significance of apathy, little is known about its pathophysiology, and in particular how apathy relates to alterations in the neural circuitry underpinning the cognitive and emotional components of goal-directed behavior. Here, we examined the neural coding of reward cues in patients with PD, with or without clinically significant levels of apathy, during performance of a spatial search task during H(2) (15)O PET (positron emission tomography) functional neuroimaging. By manipulating search outcome (money reward vs valueless token), while keeping the actions of the participants constant, we examined the influence of apathy on the neural coding of money reward cues. We found that apathy was associated with a blunted response to money in the ventromedial prefrontal cortex, amygdala, striatum, and midbrain, all part of a distributed neural circuit integral to the representation of the reward value of stimuli and actions, and the influence of reward cues on behavior. Disruption of this circuitry potentially underpins the expression of the various manifestations of apathy in PD, including reduced cognitive, emotional, and behavioral facets of goal-directed behavior.

Vigor in the face of fluctuating rates of reward: an experimental examination

Two fundamental questions underlie the expression of behavior, namely what to do and how vigorously to do it. The former is the topic of an overwhelming wealth of theoretical and empirical work particularly in the fields of reinforcement learning and decision-making, with various forms of affective prediction error playing key roles. Although vigor concerns motivation, and so is the subject of many empirical studies in diverse fields, it has suffered a dearth of computational models. Recently, Niv et al. [Niv, Y., Daw, N. D., Joel, D., & Dayan, P. Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berlin), 191, 507-520, 2007] suggested that vigor should be controlled by the opportunity cost of time, which is itself determined by the average rate of reward. This coupling of reward rate and vigor can be shown to be optimal under the theory of average return reinforcement learning for a particular class of tasks but may also be a more general, perhaps hard-wired, characteristic of the architecture of control. We, therefore, tested the hypothesis that healthy human participants would adjust their RTs on the basis of the average rate of reward. We measured RTs in an odd-ball discrimination task for rewards whose magnitudes varied slowly but systematically. Linear regression on the subjects' individual RTs using the time varying average rate of reward as the regressor of interest, and including nuisance regressors such as the immediate reward in a round and in the preceding round, showed that a significant fraction of the variance in subjects' RTs could indeed be explained by the rate of experienced reward. This validates one of the key proposals associated with the model, illuminating an apparently mandatory form of coupling that may involve tonic levels of dopamine.

Dorsolateral prefrontal cortex modulates striatal reward encoding during reappraisal of reward anticipation

Recent research showed that cognitive emotion regulation (ER) both increases activity in the dorsolateral prefrontal cortex (DLPFC) and decreases striatal responsivity to monetary rewards. Using a mixed monetary incentive delay/memory task as well as functional magnetic resonance imaging, we tested in healthy subjects whether ER effectively attenuates striatal reward encoding during the anticipation of reward (€1.00 vs. €0.05 reward cues) as well as subsequent target reaction times (RTs), which are an indicator of motivation to obtain reward. ER significantly diminished feelings of pleasant anticipation and slowed down €1.00 target RT. At the neural level, ER increased activity in the DLPFC and attenuated reward encoding in the left putamen. Analyses of psychophysiological interaction revealed that DLPFC activity correlated more positively with putamen activity during €0.05 than during €1.00 reward trials. Furthermore, parametric modulations showed that anticipatory left putamen activity correlated with target RT during nonregulation. No such correlation could be observed during ER, suggesting that ER had abolished preparatory target RT encoding. Our results provide evidence that ER can attenuate behavioral and striatal measures of reward-related motivation and motor preparation. Furthermore, the present findings suggest that the DLPFC might contribute to successful regulation of reward via increased promotion of low-reward responses.

Inactivation of the putamen selectively impairs reward history-based action selection

Behavioral decisions and actions are directed to achieve specific goals and to obtain rewards and escape punishments. Previous studies involving the recording of neuronal activity suggest the involvement of the cerebral cortex, basal ganglia, and midbrain dopamine system in these processes. The value signal of the action options is represented in the striatum, updated by reward prediction errors, and used for selecting higher-value actions. However, it remains unclear whether dysfunction of the striatum leads to impairment of value-based action selection. The present study examined the effect of inactivation of the putamen via local injection of the GABA(A) receptor agonist muscimol in monkeys engaged in a manual reward-based multi-step choice task. The monkeys first searched a reward target from three alternatives, based on the previous one or two choices and their outcomes, and obtained a large reward; they then earned an additional reward by choosing the last rewarded target. Inactivation of the putamen impaired the ability of monkeys to make optimal choices during third trial in which they were required to choose a target different from those selected in the two previous trials by updating the values of the three options. The monkeys normally changed options if the last choice resulted in small reward (lose-shift) and stayed with the last choice if it resulted in large reward (win-stay). Task start time and movement time during individual trials became longer after putamen inactivation. But monkeys could control the motivation level depending on the reward value of individual trial types before and after putamen inactivation. These results support a view that the putamen is involved selectively and critically in neuronal circuits for reward history-based action selection.