To walk or to fly? How birds choose among foraging modes

Disentangling the effects of metabolic cost and accuracy on movement speed

On any given day, we make countless reaching movements to objects around us. While such ubiquity may suggest uniformity, each movement's speed is unique-why is this? Reach speed is known to be influenced by accuracy; we slow down to sustain high accuracy. However, in other forms of movement like walking or running, metabolic cost is often the primary determinant of movement speed. Here we bridge this gap and ask: how do metabolic cost and accuracy interact to determine speed of reaching movements? First, we systematically measure the effect of increasing mass on the metabolic cost of reaching across a range of movement speeds. Next, in a sequence of three experiments, we examine how added mass affects preferred reaching speed across changing accuracy requirements. We find that, while added mass consistently increases metabolic cost thereby leading to slower metabolically optimal movement speeds, self-selected reach speeds are slower than those predicted by an optimization of metabolic cost alone. We then demonstrate how a single model that considers both accuracy and metabolic costs can explain preferred movement speeds. Together, our findings provide a unifying framework to illuminate the combined effects of metabolic cost and accuracy on movement speed and highlight the integral role metabolic cost plays in determining reach speed.

Slowing of Movements in Healthy Aging as a Rational Economic Response to an Elevated Effort Landscape

Why do we move slower as we grow older? The reward circuits of the brain, which tend to invigorate movements, decline with aging, raising the possibility that reduced vigor is due to the diminishing value that our brain assigns to movements. However, as we grow older, it also becomes more effortful to make movements. Is age-related slowing principally a consequence of increased effort costs from the muscles, or reduced valuation of reward by the brain? Here, we first quantified the cost of reaching via metabolic energy expenditure in human participants (male and female), and found that older adults consumed more energy than the young at a given speed. Thus, movements are objectively more costly for older adults. Next, we observed that when reward increased, older adults, like the young, responded by initiating their movements earlier. Yet, unlike the young, they were unwilling to increase their movement speed. Was their reluctance to reach quicker for rewards due to the increased effort costs, or because they ascribed less value to the movement? Motivated by a mathematical model, we next made the young experience a component of aging by making their movements more effortful. Now the young responded to reward by reacting faster but chose not to increase their movement speed. This suggests that slower movements in older adults are partly driven by an adaptive response to an elevated effort landscape. Moving slower may be a rational economic response the brain is making to mitigate the elevated effort costs that accompany aging.

Effort cost of harvest affects decisions and movement vigor of marmosets during foraging

Our decisions are guided by how we perceive the value of an option, but this evaluation also affects how we move to acquire that option. Why should economic variables such as reward and effort alter the vigor of our movements? In theory, both the option that we choose and the vigor with which we move contribute to a measure of fitness in which the objective is to maximize rewards minus efforts, divided by time. To explore this idea, we engaged marmosets in a foraging task in which on each trial they decided whether to work by making saccades to visual targets, thus accumulating food, or to harvest by licking what they had earned. We varied the effort cost of harvest by moving the food tube with respect to the mouth. Theory predicted that the subjects should respond to the increased effort costs by choosing to work longer, stockpiling food before commencing harvest, but reduce their movement vigor to conserve energy. Indeed, in response to an increased effort cost of harvest, marmosets extended their work duration, but slowed their movements. These changes in decisions and movements coincided with changes in pupil size. As the effort cost of harvest declined, work duration decreased, the pupils dilated, and the vigor of licks and saccades increased. Thus, when acquisition of reward became effortful, the pupils constricted, the decisions exhibited delayed gratification, and the movements displayed reduced vigor.

Worth the Work? Monkeys Discount Rewards by a Subjective Adapting Effort Cost

All life must solve how to allocate limited energy resources to maximize benefits from scarce opportunities. Economic theory posits decision makers optimize choice by maximizing the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here, we test how well the subtractive model of effort disutility explains the behavior of two male nonhuman primates (Macaca mulatta) in a binary choice task in which reward quantity and physical effort to obtain were varied. Applying random utility modeling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared with parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility depends on previous experience of effort: in analogy to work from behavioral labor economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choices can be explained by the subtractive cost model of effort. Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step.SIGNIFICANCE STATEMENT All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here, we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys' experience of effort in previous trials.

Foraging, Fear and Behavioral Variation in a Traplining Hummingbird

Traditionally, foraging behavior has been explained as the response to a trade-off between energetic gain from feeding resources and potential costs from concomitant risks. However, an increasing number of studies has shown that this view fails to explain an important fraction of the variation in foraging across a variety of taxa. One potential mechanism that may account for this variation is that various behavioral traits associated with foraging may have different fitness consequences, which may depend on the environmental context. Here, we explored this mechanism by evaluating the foraging efficiency of long-billed hermit hummingbirds (Phaethornis longirostris) with regard to three behavioral traits: (a) exploration (number of feeders used during the foraging visit), (b) risk avoidance (latency to start feeding) and (c) arousal (amount of movements during the foraging visit) in conditions at two different levels of perceived risk (low-control and high-experimental, with a threatening bullet ant model). Foraging efficiency decreased in response to threatening conditions. However, behavioral traits explained additional variation in foraging efficiency in a condition-dependent manner. More exploration was associated with a higher foraging efficiency under control conditions, but this was reversed when exposed to a threat. Regardless of the conditions, arousal was positively associated with foraging efficiency, while risk avoidance was negatively related. Importantly, exploratory behavior and risk avoidance were quite repeatable behaviors, suggesting that they may be related to the intrinsic traits of individuals. Our findings highlight the importance of taking into account additional behavioral dimensions to better understand the foraging strategies of individuals.

Worth the work? Monkeys discount rewards by a subjective adapting effort cost

All life must solve how to allocate limited energy resources to maximise benefits from scarce opportunities. Economic theory posits decision makers optimise choice by maximising the subjective benefit (utility) of reward minus the subjective cost (disutility) of the required effort. While successful in many settings, this model does not fully account for how experience can alter reward-effort trade-offs. Here we test how well the subtractive model of effort disutility explains the behavior of two non-human primates ( Macaca mulatta ) in a binary choice task in which reward quantity and physical effort to obtain were varied.Applying random utility modelling to independently estimate reward utility and effort disutility, we show the subtractive effort model better explains out-of-sample choice behavior when compared to parabolic and exponential effort discounting. Furthermore, we demonstrate that effort disutility is dependent on previous experience of effort: in analogy to work from behavioral labour economics, we develop a model of reference-dependent effort disutility to explain the increased willingness to expend effort following previous experience of effortful options in a session. The result of this analysis suggests that monkeys discount reward by an effort cost that is measured relative to an expected effort learned from previous trials. When this subjective cost of effort, a function of context and experience, is accounted for, trial-by-trial choice behavior can be explained by the subtractive cost model of effort.Therefore, in searching for net utility signals that may underpin effort-based decision-making in the brain, careful measurement of subjective effort costs is an essential first step.

Significance

All decision-makers need to consider how much effort they need to expend when evaluating potential options. Economic theories suggest that the optimal way to choose is by cost-benefit analysis of reward against effort. To be able to do this efficiently over many decision contexts, this needs to be done flexibly, with appropriate adaptation to context and experience. Therefore, in aiming to understand how this might be achieved in the brain, it is important to first carefully measure the subjective cost of effort. Here we show monkeys make reward-effort cost-benefit decisions, subtracting the subjective cost of effort from the subjective value of rewards. Moreover, the subjective cost of effort is dependent on the monkeys’ experience of effort in previous trials.

A road traversing a protected area has little effect on feeding and foraging behaviour of yellow baboons

The Tanzania-Zambia (TANZAM) Highway traversing Mikumi National Park (MINAPA) has been a concern for wildlife managers since it was first paved in 1973-1974. After its upgrade in 1989-1990, researchers have documented increasing traffic resulting in considerable animal injuries and mortalities. Yellow baboons (Papio cynocephalus) in MINAPA use the road as the bridge to and from foraging areas, therefore in addition to the risk of mortality road use could potentially have significant influence on their feeding behavior. However, knowledge on the influences of the TANZAM highway in the feeding behavior of yellow baboons is sparse. Using focal animal sampling techniques, we collected data on feeding and foraging behavior of two habituated troops of yellow baboons to examine to what extent the TANZAM highway is important in their feeding and foraging behavior. Results showed that in relation to habitat availability, visitation to habitat types reflect actual habitat choice of baboons. In general, yellow baboons less frequently visit and spent less time on the highway than natural habitats. Whenever they were on the highway, adult females and subadult males engage more into feeding, resting and socializing, while adult males were more vigilant. The major dietary compositions were fruits, seeds, leaves, sap, and invertebrates, almost exclusively collected from natural habitats, foods from the highway were opportunistically consumed. This study provides empirical evidence and concludes that yellow baboons do not directly depend on the highway for food, rather they use the TANZAM highway as normal part of their home range. However, its location near sleeping sites may have significant impact on baboons' activity budget. With these findings, we recommend strict implementation of rules against park littering and animal feeding in protected areas traversed by highways.

Nucleus accumbens D1-receptors regulate and focus transitions to reward-seeking action

It is well established that dopamine transmission is integral in mediating the influence of reward expectations on reward-seeking actions. However, the precise causal role of dopamine transmission in moment-to-moment reward-motivated behavioral control remains contentious, particularly in contexts where it is necessary to refrain from responding to achieve a beneficial outcome. To examine this, we manipulated dopamine transmission pharmacologically as rats performed a Go/No-Go task that required them to either make or withhold action to gain either a small or large reward. D1R Stimulation potentiated cue-driven action initiation, including fast impulsive actions on No-Go trials. By contrast, D1R blockade primarily disrupted the successful completion of Go trial sequences. Surprisingly, while after global D1R blockade this was characterized by a general retardation of reward-seeking actions, nucleus accumbens core (NAcC) D1R blockade had no effect on the speed of action initiation or impulsive actions. Instead, fine-grained analyses showed that this manipulation decreased the precision of animals' goal-directed actions, even though they usually still followed the appropriate response sequence. Strikingly, such "unfocused" responding could also be observed off-drug, particularly when only a small reward was on offer. These findings suggest that the balance of activity at NAcC D1Rs plays a key role in enabling the rapid activation of a focused, reward-seeking state to enable animals to efficiently and accurately achieve their goal.

Movement control, decision-making, and the building of Roman roads to link them

In science, as in life, one can only hope to both inform others, and be informed by them. The commentaries associated with our book Vigor have highlighted the many ways in which the theory that we proposed can be improved. For example, there are a myriad of factors that need to be considered in a fully encompassing objective function. The neural mechanisms underlying the links between movement and decision-making have yet to be unraveled. The implications of a two-way interaction between movement and decisions at both the individual and social levels remain to be understood. The commentaries outline future questions, and encouragingly highlight the diversity of science communities that may be linked via the concept of vigor.

Quantum decision corrections for the neuroeconomics of irrational movement control and goal attainment

Quantum decision theory corrects categorical and propositional logic pathologies common to classic statistical goal-oriented reasoning, such as rational neuroeconomics-based optimal foraging. Within this ecosalient framework, motivation, perception, learning, deliberation, brain computation, and conjunctive risk-order errors may be understood for subjective utility judgments underlying either rational or irrational canonical decisions-actions used to choose, procure, and consume rewarding nutrition with variable fitness.

Why we don't move: The importance of somatic maintenance and resting

A compelling ecological theory of movement and vigor must explain why humans and other animals spend so much time not moving. When we rest, our somatic maintenance systems continue to work. When our somatic maintenance requirements increase, we place greater subjective value on resting. To explain variation in movement and vigor, we must account for the subjective value of resting.

Visual acuity of budgerigars for moving targets

For a bird, it is often vital to visually detect food items, predators, or individuals from the same flock, i.e. moving stimuli of various shapes. Yet, behavioural tests of visual spatial acuity traditionally use stationary gratings as stimuli. We have behaviourally tested the ability of budgerigars (Melopsittacus undulatus) to detect a black circular target, moving semi-randomly at 1.69 degrees s⁻¹ against a brighter background. We found a detection threshold of 0.107±0.007 degrees of the visual field for a target size corresponding to a resolution of a grating with a spatial frequency of 4.68 cycles degree⁻¹. This detection threshold is lower than the resolution limit for gratings but similar to the threshold for stationary single objects of the same shape. We conclude that the target acuity of budgerigars for moving single targets, just as for stationary single targets, is lower than their acuity for gratings.

Summary: Movement does not improve detection: The detection threshold of budgerigars (Melopsittacus undulatus) for moving dark circular targets is similar to the detection threshold for stationary but otherwise similar targets.

Anticipatory Energization Revealed by Pupil and Brain Activity Guides Human Effort-Based Decision Making

An organism's fitness is determined by how it chooses to adapt to effort in response to challenges. Exertion of effort correlates with activity in dorsomedial prefrontal cortex (dmPFC) and noradrenergic pupil dilation, but little is known about the role of these neurophysiological processes for decisions about future efforts, they may provide anticipatory energization to help us accept the challenge or a cost representation that is weighted against the expected rewards. Here, we provide evidence for the former, by measuring pupil and functional magnetic resonance imaging (fMRI) brain responses while 52 human participants (29 females) chose whether to exert efforts to obtain rewards. Both pupil-dilation rate and dmPFC fMRI activity increased with anticipated effort level, and these increases differ depending on the choice outcome: they were stronger when participants chose to accept the challenge compared with when the challenge was declined. Crucially, the choice-dependent modulation of pupil and brain-activity effort representations were stronger in participants whose behavioral choices were more sensitive to effort. Our results identify a process involving the peripheral and central human nervous system that simulates the required energization before overt response, suggesting a role in guiding effort-based decisions.SIGNIFICANCE STATEMENT The brain's arousal system tracks the effort we engage in during strenuous activity. But much less is known about what role this effort signaling may play when we decide whether to exert effort in the future. Here, we characterize pupil-linked arousal and brain signals that guide decisions whether to engage in effort to gain money. During such choices, increases in brain activity and pupil dilation correlated with the effort involved in the chosen option, and these increases were stronger when people decided to accept the effort compared with when they rejected it. These results suggest that the brain arousal system guides decisions by energizing the organism for the prospective challenge.

Using metabolic energy to quantify the subjective value of physical effort

Economists have known for centuries that to understand an individual's decisions, we must consider not only the objective value of the goal at stake, but its subjective value as well. However, achieving that goal ultimately requires expenditure of effort. Surprisingly, despite the ubiquitous role of effort in decision-making and movement, we currently do not understand how effort is subjectively valued in daily movements. Part of the difficulty arises from the lack of an objective measure of effort. Here, we use a physiological approach to address this knowledge gap. We quantified objective effort costs by measuring metabolic cost via expired gas analysis as participants performed a reaching task against increasing resistance. We then used neuroeconomic methods to quantify each individual's subjective valuation of effort. Rather than the diminishing sensitivity observed in reward valuation, effort was valued objectively, on average. This is significantly less than the near-quadratic sensitivity to effort observed previously in force-based motor tasks. Moreover, there was significant inter-individual variability with many participants undervaluing or overvaluing effort. These findings demonstrate that in contrast with monetary decisions in which subjective value exhibits diminishing marginal returns, effort costs are valued more objectively in low-effort reaching movements common in daily life.

Precis of Vigor: Neuroeconomics of movement control

Why do we run toward people we love, but only walk toward others? Why do people in New York seem to walk faster than other cities? Why do our eyes linger longer on things we value more? There is a link between how the brain assigns value to things, and how it controls our movements. This link is an ancient one, developed through shared neural circuits that on one hand teach us how to value things, and on the other hand control the vigor with which we move. As a result, when there is damage to systems that signal reward, like dopamine and serotonin, that damage not only affects our mood and patterns of decision making, but how we move. In this book, we first ask why in principle evolution should have developed a shared system of control between valuation and vigor. We then focus on the neural basis of vigor, synthesizing results from experiments that have measured activity in various brain structures and neuromodulators, during tasks in which animals decide how patiently they should wait for reward, and how vigorously they should move to acquire it. Thus, the way we move unmasks one of our well-guarded secrets: how much we value the thing we are moving toward.

Constrained optimal foraging by marine bacterioplankton on particulate organic matter

Patch use theory predicts that organisms foraging in a heterogeneous resource landscape balance their residence time on a patch yielding diminishing returns with the sum of the metabolic, predation, and opportunity costs of foraging. By combining single-cell tracking with mathematical modeling, we show that bacteria foraging on seascapes of organic particles switch between attached and planktonic lifestyles and tune the time spent on particles to increase individual fitness. As predicted by patch use theory, bacteria remain longer on particles of higher quality and in poor environments, or when search times for fresh particles are longer. These results show that patch use theory can be a valuable framework to understand not only animals, but also microorganisms, and ultimately their ecosystem-level consequences.

Optimal foraging theory provides a framework to understand how organisms balance the benefits of harvesting resources within a patch with the sum of the metabolic, predation, and missed opportunity costs of foraging. Here, we show that, after accounting for the limited environmental information available to microorganisms, optimal foraging theory and, in particular, patch use theory also applies to the behavior of marine bacteria in particle seascapes. Combining modeling and experiments, we find that the marine bacterium Vibrio ordalii optimizes nutrient uptake by rapidly switching between attached and planktonic lifestyles, departing particles when their nutrient concentration is more than hundredfold higher than background. In accordance with predictions from patch use theory, single-cell tracking reveals that bacteria spend less time on nutrient-poor particles and on particles within environments that are rich or in which the travel time between particles is smaller, indicating that bacteria tune the nutrient concentration at detachment to increase their fitness. A mathematical model shows that the observed behavioral switching between exploitation and dispersal is consistent with foraging optimality under limited information, namely, the ability to assess the harvest rate of nutrients leaking from particles by molecular diffusion. This work demonstrates how fundamental principles in behavioral ecology traditionally applied to animals can hold right down to the scale of microorganisms and highlights the exquisite adaptations of marine bacterial foraging. The present study thus provides a blueprint for a mechanistic understanding of bacterial uptake of dissolved organic matter and bacterial production in the ocean—processes that are fundamental to the global carbon cycle.

Dopamine-Dependent Loss Aversion during Effort-Based Decision-Making

From psychology to economics, there has been substantial interest in how costs (e.g., delay, risk) are represented asymmetrically during decision-making when attempting to gain reward or avoid punishment. For example, in decision-making under risk, individuals show a tendency to prefer to avoid punishment rather than to acquire the equivalent reward (loss aversion). Although the cost of physical effort has recently received significant attention, it remains unclear whether loss aversion exists during effort-based decision-making. On the one hand, loss aversion may be hardwired due to asymmetric evolutionary pressure on losses and gains and therefore exists across decision-making contexts. On the other hand, distinct brain regions are involved with different decision costs, making it questionable whether similar asymmetries exist. Here, we demonstrate that young healthy human participants (females, 16; males, 6) exhibit loss aversion during effort-based decision-making by exerting more physical effort to avoid punishment than to gain a same-size reward. Next, we show that medicated Parkinson's disease (PD) patients (females, 9; males, 9) show a reduction in loss aversion compared with age-matched control subjects (females, 11; males, 9). Behavioral and computational analysis revealed that people with PD exerted similar physical effort in return for a reward but were less willing to produce effort to avoid punishment. Therefore, loss aversion is present during effort-based decision-making and can be modulated by altered dopaminergic state. This finding could have important implications for our understanding of clinical disorders that show a reduced willingness to exert effort in the pursuit of reward.SIGNIFICANCE STATEMENT Loss aversion-preferring to avoid punishment rather than to acquire equivalent reward-is an important concept in decision-making under risk. However, little is known about whether loss aversion also exists during decisions where the cost is physical effort. This is surprising given that motor cost shapes human behavior, and a reduced willingness to exert effort is a characteristic of many clinical disorders. Here, we show that healthy human individuals exert more effort to minimize punishment than to maximize reward (loss aversion). We also demonstrate that medicated Parkinson's disease patients exert similar effort to gain reward but less effort to avoid punishment when compared with healthy age-matched control subjects. This indicates that dopamine-dependent loss aversion is crucial for explaining effort-based decision-making.

The effect of learning on heart rate and behavior of European starlings (Sturnus vulgaris)

Wild-caught European starlings (Sturnus vulgaris) were exposed to a learning task to determine whether heart rate (HR) and behavior responses to the learning activated the sympathetic nervous system. Birds learned to discriminate between images of opposite convexity (concave and convex) based on shading cues in a closed economy (food only available through task completion). Once learned, the task was changed in three ways: (a) manipulating the angle and shape of the image; (b) altering the availability of the task; and (c) reversing the positive stimulus. HR, used as an index of catecholamine release, was measured during each change to determine whether having to alter previously established behaviors to learn new behaviors elicited a sympathetic response. Starlings decreased their HR during the initial discrimination training and did not alter their HR when presented with modified images or when the positive stimulus was reversed. However, HR increased when the task became unavailable and decreased upon its return, suggesting that preventing task performance was perceived as stressful. Birds also modified their behavior when tasks were changed. The number of trials per minute decreased during the reversal treatment, as did the success rate, suggesting that starlings may try to conserve energy when access to food diminishes. This is also supported by the decrease in perch hops per minute when the task was unavailable and the subsequent increase upon its return. Overall, these results suggest that learning per se does not activate the sympathetic nervous system and, therefore, is not a stressor for wild birds.

Neurons in the pigeon nidopallium caudolaterale, but not the corticoidea dorsolateralis, display value and effort discounting activity

We recorded from single neurons in two areas of the pigeon brain while birds were required to peck a stimulus indicating either a high effort task or a low effort task would follow. Upon completion of the task the birds received the same reward. We found that activity in the nidopallium caudolaterale, an area equivalent to the mammalian prefrontal cortex, was modulated by the value of the reward that would be received based on how much effort was required to obtain it. Value coding was most prominent during the presentation of the stimulus indicating a high or low effort task, and in the delay period immediately prior to carrying out the effort task. In contrast, activity in the corticoidea dorsolateralis was not modulated by value, however, population firing patterns suggest that it may be involved in associating actions with outcomes. Our findings support the view that activity in the nidopallium caudolaterale reflects value of reward as a function of effort discounting and as such may serve functions similar to the mammalian anterior cingulate cortex.

Contributions of metabolic and temporal costs to human gait selection

Humans naturally select several parameters within a gait that correspond with minimizing metabolic cost. Much less is understood about the role of metabolic cost in selecting between gaits. Here, we asked participants to decide between walking or running out and back to different gait specific markers. The distance of the walking marker was adjusted after each decision to identify relative distances where individuals switched gait preferences. We found that neither minimizing solely metabolic energy nor minimizing solely movement time could predict how the group decided between gaits. Of our twenty participants, six behaved in a way that tended towards minimizing metabolic energy, while eight favoured strategies that tended more towards minimizing movement time. The remaining six participants could not be explained by minimizing a single cost. We provide evidence that humans consider not just a single movement cost, but instead a weighted combination of these conflicting costs with their relative contributions varying across participants. Individuals who placed a higher relative value on time ran faster than individuals who placed a higher relative value on metabolic energy. Sensitivity to temporal costs also explained variability in an individual's preferred velocity as a function of increasing running distance. Interestingly, these differences in velocity both within and across participants were absent in walking, possibly due to a steeper metabolic cost of transport curve. We conclude that metabolic cost plays an essential, but not exclusive role in gait decisions.

Terrestrial locomotion of the Svalbard rock ptarmigan: comparing field and laboratory treadmill studies

Research into the terrestrial locomotion of birds is often based upon laboratory treadmill experiments. However, it is unclear how transposable these results are for birds moving in the wild. Here, using video recordings, we compared the kinematics of locomotion (stride frequency, stride length, stance phase, swing phase, duty factor) and speed range of Svalbard rock ptarmigan (Lagopus muta hyperborea) under field and laboratory treadmill conditions. Our findings indicate that the kinematics of walking and aerial running are conserved when moving on the treadmill and in the field. Differences, however, were found when grounded running under the two conditions, linked to substrate. Substrate effects were confirmed by analysing trials only moving over very hard snow. In line with laboratory treadmill energetic predictions, wild ptarmigan have a preferred speed during walking and to a lesser extent when aerial running but not when moving with a grounded running gait. The birds were also capable of a higher top speed in the field than that observed during treadmill studies. Our findings demonstrate that laboratory treadmill research provides meaningful information relevant to wild birds while highlighting the importance of understanding the substrate the animals are moving over.

Optimizing sequential decisions in the drift-diffusion model

To make decisions organisms often accumulate information across multiple timescales. However, most experimental and modeling studies of decision-making focus on sequences of independent trials. On the other hand, natural environments are characterized by long temporal correlations, and evidence used to make a present choice is often relevant to future decisions. To understand decision-making under these conditions we analyze how a model ideal observer accumulates evidence to freely make choices across a sequence of correlated trials. We use principles of probabilistic inference to show that an ideal observer incorporates information obtained on one trial as an initial bias on the next. This bias decreases the time, but not the accuracy of the next decision. Furthermore, in finite sequences of trials the rate of reward is maximized when the observer deliberates longer for early decisions, but responds more quickly towards the end of the sequence. Our model also explains experimentally observed patterns in decision times and choices, thus providing a mathematically principled foundation for evidence-accumulation models of sequential decisions.

What Is the Relationship between Dopamine and Effort?

The trade-off between reward and effort is at the heart of most behavioral theories, from ecology to economics. Compared to reward, however, effort remains poorly understood, both at the behavioral and neurophysiological levels. This is important because unwillingness to overcome effort to gain reward is a common feature of many neuropsychiatric and neurological disorders. A recent surge in interest in the neurobiological basis of effort has led to seemingly conflicting results regarding the role of dopamine. We argue here that, upon closer examination, there is actually striking consensus across studies: dopamine primarily codes for future reward but is less sensitive to anticipated effort cost. This strong association between dopamine and the incentive effects of rewards places dopamine in a key position to promote reward-directed action.

Control of movement vigor and decision making during foraging

During foraging, animals decide how long to stay at a patch and harvest reward, and then, they move with certain vigor to another location. How does the brain decide when to leave, and how does it determine the speed of the ensuing movement? Here, we considered the possibility that both the decision-making and the motor control problems aimed to maximize a single normative utility: the sum of all rewards acquired minus all efforts expended divided by total time. This optimization could be achieved if the brain compared a local measure of utility with its history. To test the theory, we examined behavior of people as they gazed at images: they chose how long to look at the image (harvesting information) and then moved their eyes to another image, controlling saccade speed. We varied reward via image content and effort via image eccentricity, and then, we measured how these changes affected decision making (gaze duration) and motor control (saccade speed). After a history of low rewards, people increased gaze duration and decreased saccade speed. In anticipation of future effort, they lowered saccade speed and increased gaze duration. After a history of high effort, they elevated their saccade speed and increased gaze duration. Therefore, the theory presented a principled way with which the brain may control two aspects of behavior: movement speed and harvest duration. Our experiments confirmed many (but not all) of the predictions, suggesting that harvest duration and movement speed, fundamental aspects of behavior during foraging, may be governed by a shared principle of control.

Infants' prosocial behavior is governed by cost-benefit analyses

Cost-benefit analyses are central to mature decision-making and behavior across a range of contexts. Given debates regarding the nature of infants' prosociality, we investigated whether 18-month-old infants' (N = 160) prosocial behavior is impacted by anticipated costs and benefits. Infants participated in a helping task in which they could carry either a heavy or light block across a room to help an experimenter. Infants' helping behavior was attenuated when the anticipated physical costs were high versus low (Experiment 1), and high-cost helping was enhanced under conditions of increased intrinsic motivational benefits (Experiments 2 and 3). High-cost helping was further predicted by infants' months of walking experience, presumably because carrying a heavy block across a room is more effortful for less experienced walkers than for more experienced walkers demonstrating that infants subjectively calibrate costs. Thus, infants' prosocial responding may be guided by a rational decision-making process that weighs and integrates costs and benefits.

High-resolution GPS tracking reveals sex differences in migratory behaviour and stopover habitat use in the Lesser Black-backed Gull Larus fuscus

Sex-, size- or age-dependent variation in migration strategies in birds is generally expected to reflect differences in competitive abilities. Theoretical and empirical studies thereby focus on differences in wintering areas, by which individuals may benefit from avoiding food competition during winter or ensuring an early return and access to prime nesting sites in spring. Here, we use GPS tracking to assess sex- and size-related variation in the spatial behaviour of adult Lesser Black-backed Gulls (Larus fuscus) throughout their annual cycle. We did not find sex- or size-dependent differences in wintering area or the timing of spring migration. Instead, sexual differences occurred prior to, and during, autumn migration, when females strongly focussed on agricultural areas. Females exhibited a more protracted autumn migration strategy, hence spent more time on stopover sites and arrived 15 days later at their wintering areas, than males. This shift in habitat use and protracted autumn migration coincided with the timing of moult, which overlaps with chick rearing and migration. Our results suggest that this overlap between energy-demanding activities may lead females to perform a more prolonged autumn migration, which results in spatiotemporal differences in foraging habitat use between the sexes.

Vigor of reaching movements: reward discounts the cost of effort

Making a movement may be thought of as an economic decision in which one spends effort to acquire reward. Time discounts reward, which predicts that the magnitude of reward should affect movement vigor: we should move faster, spending greater effort, when there is greater reward at stake. Indeed, saccade peak velocities are greater and reaction-times shorter when a target is paired with reward. In this study, we focused on human reaching and asked whether movement kinematics were affected by expectation of reward. Participants made out-and-back reaching movements to one of four quadrants of a 14-cm circle. During various periods of the experiment only one of the four quadrants was paired with reward, and the transition from reward to nonreward status of a quadrant occurred randomly. Our experiment design minimized dependence of reward on accuracy, granting the subjects wide latitude in self-selecting their movement speed, amplitude, and variability. When a quadrant was paired with reward, reaching movements had a shorter reaction time, higher peak velocity, and greater amplitude. Despite this greater vigor, movements toward the rewarded quadrant suffered from less variability: both reaction times and reach kinematics were less variable when there was expectation of reward. Importantly, the effect of reward on vigor was specific to the movement component that preceded the time of reward (outward reach), not the movement component that followed it (return reach). Our results suggest that expectation of reward not only increases vigor of human reaching but also decreases its variability. NEW & NOTEWORTHY Movements may be thought of as an economic transaction where the vigor of the movement represents the effort that the brain is willing to expend to acquire a rewarding state. We show that in reaching, reward discounts the cost of effort, producing movements with shorter reaction time, higher velocity, greater amplitude, and reduced reaction-time variability. These results complement earlier observations in saccades, suggesting a common principle of economics across modalities of motor control.

What Makes You Go Faster?: The Effect of Reward on Speeded Action under Risk

Evaluating the potential reward and risk associated with a choice of action plays an important role in everyday decision making. However, the details behind how reward and risk affect the decisions for actions remain unclear. The present study investigates the influence of reward and risk on a decision to make a speeded motor response. One hundred and ten college students performed a Speed-Rewarded Go-NoGo task during which they were rewarded proportionally based on the speed and accuracy of their response. On each trial, the magnitude of potential reward and the probability of a forthcoming Go signal (Go-probability) were presented prior to the Go or NoGo signal. Personality traits, such as risk taking and impulsive tendencies, were measured to determine their contribution in explaining individual differences in task performance. The results showed that larger amount of rewards can motivate people to respond faster, and this effect was modulated by the assessed risk, suggesting that decisions for actions are based on a systematic trade-off between rewards and risks. Moreover, when the assessed risk was high, individuals with greater risk taking and impulsive tendencies did not adequately adjust their behavior across different reward levels. These findings shed light on the mechanistic understanding of the effect of reward and risk on decisions for a speeded action.

Fatigue modulates dopamine availability and promotes flexible choice reversals during decision making

During decisions, animals balance goal achievement and effort management. Despite physical exercise and fatigue significantly affecting the levels of effort that an animal exerts to obtain a reward, their role in effort-based choice and the underlying neurochemistry are incompletely known. In particular, it is unclear whether fatigue influences decision (cost-benefit) strategies flexibly or only post-decision action execution and learning. To answer this question, we trained mice on a T-maze task in which they chose between a high-cost, high-reward arm (HR), which included a barrier, and a low-cost, low-reward arm (LR), with no barrier. The animals were parametrically fatigued immediately before the behavioural tasks by running on a treadmill. We report a sharp choice reversal, from the HR to LR arm, at 80% of their peak workload (PW), which was temporary and specific, as the mice returned to choose the HC when the animals were successively tested at 60% PW or in a two-barrier task. These rapid reversals are signatures of flexible choice. We also observed increased subcortical dopamine levels in fatigued mice: a marker of individual bias to use model-based control in humans. Our results indicate that fatigue levels can be incorporated in flexible cost-benefits computations that improve foraging efficiency.

A Representation of Effort in Decision-Making and Motor Control

Given two rewarding stimuli, animals tend to choose the more rewarding (or less effortful) option. However, they also move faster toward that stimulus [1-5]. This suggests that reward and effort not only affect decision-making, they also influence motor control [6, 7]. How does the brain compute the effort requirements of a task? Here, we considered data acquired during walking, reaching, flying, or isometric force production. In analyzing the decision-making and motor-control behaviors of various animals, we considered the possibility that the brain may estimate effort objectively, via the metabolic energy consumed to produce the action. We measured the energetic cost of reaching and found that, like walking, it was convex in time, with a global minimum, implying that there existed a movement speed that minimized effort. However, reward made it worthwhile to be energetically inefficient. Using a framework in which utility of an action depended on reward and energetic cost, both discounted in time, we found that it was possible to account for a body of data in which animals were free to choose how to move (reach slow or fast), as well as what to do (walk or fly, produce force F1 or F2). We suggest that some forms of decision-making and motor control may share a common utility in which the brain represents the effort associated with performing an action objectively via its metabolic energy cost and then, like reward, temporally discounts it as a function of movement duration.

Is it Worth the Effort? Novel Insights into Obesity-Associated Alterations in Cost-Benefit Decision-Making

Cost-benefit decision-making entails the process of evaluating potential actions according to the trade-off between the expected reward (benefit) and the anticipated effort (costs). Recent research revealed that dopaminergic transmission within the fronto-striatal circuitry strongly modulates cost-benefit decision-making. Alterations within the dopaminergic fronto-striatal system have been associated with obesity, but little is known about cost-benefit decision-making differences in obese compared with lean individuals. With a newly developed experimental task we investigate obesity-associated alterations in cost-benefit decision-making, utilizing physical effort by handgrip-force exertion and both food and non-food rewards. We relate our behavioral findings to alterations in local gray matter volume assessed by structural MRI. Obese compared with lean subjects were less willing to engage in physical effort in particular for high-caloric sweet snack food. Further, self-reported body dissatisfaction negatively correlated with the willingness to invest effort for sweet snacks in obese men. On a structural level, obesity was associated with reductions in gray matter volume in bilateral prefrontal cortex. Nucleus accumbens volume positively correlated with task induced implicit food craving. Our results challenge the common notion that obese individuals are willing to work harder to obtain high-caloric food and emphasize the need for further exploration of the underlying neural mechanisms regarding cost-benefit decision-making differences in obesity.

Behavioral modeling of human choices reveals dissociable effects of physical effort and temporal delay on reward devaluation

There has been considerable interest from the fields of biology, economics, psychology, and ecology about how decision costs decrease the value of rewarding outcomes. For example, formal descriptions of how reward value changes with increasing temporal delays allow for quantifying individual decision preferences, as in animal species populating different habitats, or normal and clinical human populations. Strikingly, it remains largely unclear how humans evaluate rewards when these are tied to energetic costs, despite the surge of interest in the neural basis of effort-guided decision-making and the prevalence of disorders showing a diminished willingness to exert effort (e.g., depression). One common assumption is that effort discounts reward in a similar way to delay. Here we challenge this assumption by formally comparing competing hypotheses about effort and delay discounting. We used a design specifically optimized to compare discounting behavior for both effort and delay over a wide range of decision costs (Experiment 1). We then additionally characterized the profile of effort discounting free of model assumptions (Experiment 2). Contrary to previous reports, in both experiments effort costs devalued reward in a manner opposite to delay, with small devaluations for lower efforts, and progressively larger devaluations for higher effort-levels (concave shape). Bayesian model comparison confirmed that delay-choices were best predicted by a hyperbolic model, with the largest reward devaluations occurring at shorter delays. In contrast, an altogether different relationship was observed for effort-choices, which were best described by a model of inverse sigmoidal shape that is initially concave. Our results provide a novel characterization of human effort discounting behavior and its first dissociation from delay discounting. This enables accurate modelling of cost-benefit decisions, a prerequisite for the investigation of the neural underpinnings of effort-guided choice and for understanding the deficits in clinical disorders characterized by behavioral inactivity.

One of the main functions of the brain is to select sequences of actions that lead to rewarding outcomes (e.g., food). However, such rewards are often not readily available; instead physical effort may be required to obtain them, or their arrival may be delayed. The ability to integrate the costs and benefits of potential courses of action is severely impaired in several common disorders, such as depression and schizophrenia. Mathematical models can describe how individuals depreciate rewards based on the costs associated with them. For example, models of how a reward loses appeal with increasing temporal delays can provide individual impulsivity scores, and can serve as a predictor of financial mismanagement. To date, there is no established model to describe accurately how humans depreciate rewards when obtaining them requires physical effort. This is surprising given the prevalence of disorders related to a diminished willingness to exert effort. Here we derive a biologically plausible mathematical model that can describe how healthy humans make decisions tied to physical efforts. We show that effort and delay influence reward valuation in different ways, contrary to common assumptions. Our model will be important for characterizing decision-making deficits in clinical disorders characterized by behavioral inactivity.

Captive chimpanzee foraging in a social setting: a test of problem solving, flexibility, and spatial discounting

In the wild, primates are selective over the routes that they take when foraging and seek out preferred or ephemeral food. Given this, we tested how a group of captive chimpanzees weighed the relative benefits and costs of foraging for food in their environment when a less-preferred food could be obtained with less effort than a more-preferred food. In this study, a social group of six zoo-housed chimpanzees (Pan troglodytes) could collect PVC tokens and exchange them with researchers for food rewards at one of two locations. Food preference tests had revealed that, for these chimpanzees, grapes were a highly-preferred food while carrot pieces were a less-preferred food. The chimpanzees were tested in three phases, each comprised of 30 thirty-minute sessions. In phases 1 and 3, if the chimpanzees exchanged a token at the location they collected them they received a carrot piece (no travel) or they could travel ≥10 m to exchange tokens for grapes at a second location. In phase 2, the chimpanzees had to travel for both rewards (≥10 m for carrot pieces, ≥15 m for grapes). The chimpanzees learned how to exchange tokens for food rewards, but there was individual variation in the time it took for them to make their first exchange and to discover the different exchange locations. Once all the chimpanzees were proficient at exchanging tokens, they exchanged more tokens for grapes (phase 3). However, when travel was required for both rewards (phase 2), the chimpanzees were less likely to work for either reward. Aside from the alpha male, all chimpanzees exchanged tokens for both reward types, demonstrating their ability to explore the available options. Contrary to our predictions, low-ranked individuals made more exchanges than high-ranked individuals, most likely because, in this protocol, chimpanzees could not monopolize the tokens or access to exchange locations. Although the chimpanzees showed a preference for exchanging tokens for their more-preferred food, they appeared to develop strategies to reduce the cost associated with obtaining the grapes, including scrounging rewards and tokens from group mates and carrying more than one token when travelling to the farther exchange location. By testing the chimpanzees in their social group we were able to tease apart the social and individual influences on their decision making and the interplay with the physical demands of the task, which revealed that the chimpanzees were willing to travel farther for better.

Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates

The mechanisms of reward maximization have been extensively studied at both the computational and neural levels. By contrast, little is known about how the brain learns to choose the options that minimize action cost. In principle, the brain could have evolved a general mechanism that applies the same learning rule to the different dimensions of choice options. To test this hypothesis, we scanned healthy human volunteers while they performed a probabilistic instrumental learning task that varied in both the physical effort and the monetary outcome associated with choice options. Behavioral data showed that the same computational rule, using prediction errors to update expectations, could account for both reward maximization and effort minimization. However, these learning-related variables were encoded in partially dissociable brain areas. In line with previous findings, the ventromedial prefrontal cortex was found to positively represent expected and actual rewards, regardless of effort. A separate network, encompassing the anterior insula, the dorsal anterior cingulate, and the posterior parietal cortex, correlated positively with expected and actual efforts. These findings suggest that the same computational rule is applied by distinct brain systems, depending on the choice dimension-cost or benefit-that has to be learned.

The evolutionary roots of human decision making

Humans exhibit a suite of biases when making economic decisions. We review recent research on the origins of human decision making by examining whether similar choice biases are seen in nonhuman primates, our closest phylogenetic relatives. We propose that comparative studies can provide insight into four major questions about the nature of human choice biases that cannot be addressed by studies of our species alone. First, research with other primates can address the evolution of human choice biases and identify shared versus human-unique tendencies in decision making. Second, primate studies can constrain hypotheses about the psychological mechanisms underlying such biases. Third, comparisons of closely related species can identify when distinct mechanisms underlie related biases by examining evolutionary dissociations in choice strategies. Finally, comparative work can provide insight into the biological rationality of economically irrational preferences.

Using vertebrate prey capture locations to identify cover type selection patterns of nocturnally foraging Burrowing Owls

Studies of habitat selection often measure an animal's use of space via radiotelemetry or GPS-based technologies. Such data tend to be analyzed using a resource selection function, despite the fact that the actual resources acquired are typically not recorded. Without explicit proof of resource use, conclusions from RSF models are based on assumptions regarding an animal's behavior and the resources gained. Conservation initiatives are often based on space-use models, and could be detrimental to the target species if these assumptions are incorrect. We used GPS dataloggers and digital video recorders to determine precise locations where nocturnally foraging Burrowing Owls acquired food resources (vertebrate prey). We compared land cover type selection patterns using a presence-only resource selection function (RSF) to a model that incorporated prey capture locations (CRSF). We also compared net prey returns in each cover type to better measure reward relative to foraging effort. The RSF method did not reflect prey capture patterns and cover-type rankings from this model were quite different from models that used only locations where prey was known to have been obtained. Burrowing Owls successfully foraged across all cover types; however, return vs. effort models indicate that different cover types were of higher quality than those identified using resource selection functions. Conclusions about the type of resources acquired should not be made from RSF-style models without evidence that the actual resource of interest was acquired. Conservation efforts based on RSF models alone may be ineffective or detrimental to the target species if the limiting resource and where it is acquired are not properly identified.

The anterior cingulate gyrus signals the net value of others' rewards

Evaluating the costs and benefits of our own choices is central to most forms of decision-making and its mechanisms in the brain are becoming increasingly well understood. To interact successfully in social environments, it is also essential to monitor the rewards that others receive. Previous studies in nonhuman primates have found neurons in the anterior cingulate cortex (ACC) that signal the net value (benefit minus cost) of rewards that will be received oneself and also neurons that signal when a reward will be received by someone else. However, little is understood about the way in which the human brain engages in cost-benefit analyses during social interactions. Does the ACC signal the net value (the benefits minus the costs) of rewards that others will receive? Here, using fMRI, we examined activity time locked to cues that signaled the anticipated reward magnitude (benefit) to be gained and the level of effort (cost) to be incurred either by a subject themselves or by a social confederate. We investigated whether activity in the ACC covaries with the net value of rewards that someone else will receive when that person is required to exert effort for the reward. We show that, although activation in the sulcus of the ACC signaled the costs on all trials, gyral ACC (ACC(g)) activity varied parametrically only with the net value of rewards gained by others. These results suggest that the ACC(g) plays an important role in signaling cost-benefit information by signaling the value of others' rewards during social interactions.

Intersection of effort and risk: ethological and neurobiological perspectives

The physical effort required to seek out and extract a resource is an important consideration for a foraging animal. A second consideration is the variability or risk associated with resource delivery. An intriguing observation from ethological studies is that animals shift their preference from stable to variable food sources under conditions of increased physical effort or falling energetic reserves. Although theoretical models for this effect exist, no exploration into its biological basis has been pursued. Recent advances in understanding the neural basis of effort- and risk-guided decision making suggest that opportunities exist for determining how effort influences risk preference. In this review, we describe the intersection between the neural systems involved in effort- and risk-guided decision making and outline two mechanisms by which effort-induced changes in dopamine release may increase the preference for variable rewards.

Neural substrates underlying effort computation in schizophrenia

The lack of initiative, drive or effort in patients with schizophrenia is linked to marked functional impairments. However, our assessment of effort and motivation is crude, relying on clinical rating scales based largely on patient recall. In order to better understand the neurobiology of effort in schizophrenia, we need more rigorous measurements of this construct. In the behavioural neuroscience literature, decades of work has been carried out developing various paradigms to examine the neural underpinnings of an animal's willingness to expend effort for a reward. Here, we shall review this literature on the nature of paradigms used in rodents to assess effort, as well as those used in humans. Next, the neurobiology of these effort-based decisions will be discussed. We shall then review what is known about effort in schizophrenia, and what might be inferred from experiments done in other human populations. Lastly, we shall discuss future directions of research that may assist in shedding light on the neurobiology of effort cost computations in schizophrenia.

Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task

Animals are thought to evaluate the desirability of action options using a unified scale that combines predicted benefits ("rewards"), costs, and the animal's internal motivational state. Midbrain dopamine neurons have long been associated with the reward part of this equation, but it is unclear whether these neurons also estimate the costs of taking an action. We studied the spiking activity of dopamine neurons in the substantia nigra pars compacta of monkeys (Macaca mulatta) during a reaching task in which the energetic costs incurred (friction loads) and the benefits gained (drops of food) were manipulated independently. Although the majority of dopamine neurons encoded the upcoming reward alone, a subset predicted net utility of a course of action by signaling the expected reward magnitude discounted by the invested cost in terms of physical effort. In addition, the tonic activity of some dopamine neurons was slowly reduced in conjunction with the accumulated trials, which is consistent with the hypothesized role for tonic dopamine in the invigoration or motivation of instrumental responding. The present results shed light on an often-hypothesized role for dopamine in the regulation of the balance in natural behaviors between the energy expended and the benefits gained, which could explain why dopamine disorders, such as Parkinson's disease, lead to a breakdown of that balance.

The basal ganglia, the ideal machinery for the cost-benefit analysis of action plans

Basal ganglia dysfunction causes profound movement disorders, often attributed to imbalance between direct and indirect pathway activity in the sensorimotor basal ganglia. In the classical view, the direct pathway facilitates movements, whereas the indirect pathway inhibits movements. However, the recent finding of co-activation of the two pathways during movement challenges this view. Reconciling the new finding with the body of evidence supporting the classical view, this perspective proposes that the direct pathway computes the expected benefits of motor plans entering the basal ganglia, while the indirect pathway computes their expected costs. Thus, basal ganglia output combining the two pathway signals in a subtraction manner weighs benefits against costs, and endorses the plan with the best prospective outcome via feedback projections to the cortex. The cost-benefit model, while retaining the antagonistic roles of the two pathways for movements, requires co-activation of the two pathways during movement as both benefit and cost are computed for every movement. The cost-benefit model, though simple, accounts for a number of confounding results, and generates new focus for future research with testable predictions.

How costs influence decision values for mixed outcomes

The things that we hold dearest often require a sacrifice, as epitomized in the maxim “no pain, no gain.” But how is the subjective value of outcomes established when they consist of mixtures of costs and benefits? We describe theoretical models for the integration of costs and benefits into a single value, drawing on both the economic and the empirical literatures, with the goal of rendering them accessible to the neuroscience community. We propose two key assays that go beyond goodness of fit for deciding between the dominant additive model and four varieties of interactive models. First, how they model decisions between costs when reward is not on offer; and second, whether they predict changes in reward sensitivity when costs are added to outcomes, and in what direction. We provide a selective review of relevant neurobiological work from a computational perspective, focusing on those studies that illuminate the underlying valuation mechanisms. Cognitive neuroscience has great potential to decide which of the theoretical models is actually employed by our brains, but empirical work has yet to fully embrace this challenge. We hope that future research improves our understanding of how our brain decides whether mixed outcomes are worthwhile.

Dopamine and effort-based decision making

Motivational theories of choice focus on the influence of goal values and strength of reinforcement to explain behavior. By contrast relatively little is known concerning how the cost of an action, such as effort expended, contributes to a decision to act. Effort-based decision making addresses how we make an action choice based on an integration of action and goal values. Here we review behavioral and neurobiological data regarding the representation of effort as action cost, and how this impacts on decision making. Although organisms expend effort to obtain a desired reward there is a striking sensitivity to the amount of effort required, such that the net preference for an action decreases as effort cost increases. We discuss the contribution of the neurotransmitter dopamine (DA) toward overcoming response costs and in enhancing an animal's motivation toward effortful actions. We also consider the contribution of brain structures, including the basal ganglia and anterior cingulate cortex, in the internal generation of action involving a translation of reward expectation into effortful action.

Spatial discounting of food and social rewards in guppies (poecilia reticulata)

In temporal discounting, animals trade off the time to obtain a reward against the quality of a reward, choosing between a smaller reward available sooner versus a larger reward available later. Similar discounting can apply over space, when animals choose between smaller and closer versus larger and more distant rewards. Most studies of temporal and spatial discounting in non-human animals use food as the reward, and it is not established whether animals trade off other preferred stimuli in similar ways. Here, we offered female guppies (Poecilia reticulata) a spatial discounting task in which we measured preferences for a larger reward as the distance to it increased relative to a closer but smaller reward. We tested whether the fish discounted reward types differently by offering subjects either food items or same-sex conspecifics as rewards. Before beginning the discounting tasks, we conducted validation tests to ensure that subjects equally valued the food and social stimuli in the quantities provided. In the discounting task, subjects switched their preferences from the larger to the smaller reward as the distance to the larger reward increased (spatial discounting), but the pattern and magnitude of discounting did not differ across the two reward types. These findings indicate that guppies show similar patterns of discounting for food and social rewards in a spatial task. In an examination of travel times, however, the fish swam faster to food rewards than to shoaling partners. Analysis of travel times suggests that fish temporally discounted social rewards less steeply than food rewards. Thus, reward type influences temporal discounting, suggesting a dissociation between temporal and spatial discounting. Our results illustrate how animals adjust choices and travel times depending on both the type of cost (time, distance) and benefits (food, social partners).

Neural processing of reward magnitude under varying attentional demands

Central to the organization of behavior is the ability to represent the magnitude of a prospective reward and the costs related to obtaining it. Therein, reward-related neural activations are discounted in dependence of the effort required to resolve a given task. Varying attentional demands of the task might however affect reward-related neural activations. Here we employed fMRI to investigate the neural representation of expected values during a monetary incentive delay task with varying attentional demands. Following a cue, indicating at the same time the difficulty (hard/easy) and the reward magnitude (high/low) of the upcoming trial, subjects performed an attention task and subsequently received feedback about their monetary reward. Consistent with previous results, activity in anterior-cingulate, insular/orbitofrontal and mesolimbic regions co-varied with the anticipated reward-magnitude, but also with the attentional requirements of the task. These activations occurred contingent on action-execution and resembled the response time pattern of the subjects. In contrast, cue-related activations, signaling the forthcoming task-requirements, were only observed within attentional control structures. These results suggest that anticipated reward-magnitude and task-related attentional demands are concurrently processed in partially overlapping neural networks of anterior-cingulate, insular/orbitofrontal, and mesolimbic regions.

Effort-based decision making in the rat: an [18F]fluorodeoxyglucose micro positron emission tomography study

Decision making refers to the process by which subjects choose between competing courses of action based on the expected costs and benefits of their consequences. Lesion studies in rats suggest that the anterior cingulate cortex and the nucleus accumbens are key structures of a neural system that subserves effort-based decision making. Little is known about brain activation associated with effort-based decisions in intact rats. Using an open hypothesis approach, we used 2-deoxy-2[(18)F]fluoro-D-glucose positron emission tomography (FDG-PET) to assess regional metabolic changes in two conditions of an effort-based decision making task. In the "same effort" condition, male rats could choose between two response options associated with the same effort but different reward sizes, i.e., decision making was simply a function of reward size. By contrast, in the "different effort" condition, an integration of different efforts and reward sizes associated with the two response options was necessary before making a decision. Separate PET scans were performed from each condition. Subtractive analysis revealed that metabolic activity was increased in the different effort relative to the same effort condition in the left anterior cingulate, left orbitofrontal and prelimbic cortex region. Metabolic activity was decreased in the infralimbic cortex and septum region. Our findings suggest that making decisions on how much effort to invest to obtain greater rewards evokes changes of metabolic activity in multiple brain areas associated with cognitive, limbic, motor and autonomic functions. This study demonstrates that FDG-PET provides a tool to determine in rats regional brain metabolic activity in cognitive tasks.