Neural mechanisms underlying motivation of mental versus physical effort

Examining dorsal striatum in cognitive effort using Parkinson's disease and fMRI

OBJECTIVE

Understanding cognition mediated by the striatum can clarify cognitive deficits in Parkinson's disease (PD). Previously, we claimed that dorsal striatum (DS) mediates cognitive flexibility. To refute the possibility that variation in cognitive effort confounded our observations, we reexamined our data to dissociate cognitive flexibility from effort. PD provides a model for exploring DS-mediated functions. In PD, dopamine-producing cells supplying DS are significantly degenerated. DS-mediated functions are impaired off and improved on dopamine replacement medication. Functional magnetic resonance imaging (fMRI) can confirm striatum-mediated functions.

METHODS

Twenty-two PD patients, off-on dopaminergic medication, and 22 healthy age-matched controls performed a number selection task. Numerical distance between number pairs varied systematically. Selecting between two numbers that are closer versus distant in magnitude is more effortful: the symbolic distance effect. However, selecting between closer versus distant number pairs is equivalent in the need to alter attention or response strategies (i.e., cognitive flexibility). In Experiment 2, 28 healthy participants performed the same task with simultaneous measurement of brain activity with fMRI.

RESULTS

The symbolic distance effect was equivalent for PD versus control participants and across medication sessions. Furthermore, symbolic distance did not correlate with DS activation using fMRI. In this dataset, we showed previously that integrating conflicting influences on decision making is (1) impaired in PD and improved by dopaminergic therapy and (2) associated with preferential DS activation using fMRI.

INTERPRETATION

These findings support the notion that DS mediates cognitive flexibility specifically, not merely cognitive effort, accounting for some cognitive deficits in PD and informing treatment.

A common currency for the computation of motivational values in the human striatum

Reward comparison in the brain is thought to be achieved through the use of a 'common currency', implying that reward value representations are computed on a unique scale in the same brain regions regardless of the reward type. Although such a mechanism has been identified in the ventro-medial prefrontal cortex and ventral striatum in the context of decision-making, it is less clear whether it similarly applies to non-choice situations. To answer this question, we scanned 38 participants with fMRI while they were presented with single cues predicting either monetary or erotic rewards, without the need to make a decision. The ventral striatum was the main brain structure to respond to both cues while showing increasing activity with increasing expected reward intensity. Most importantly, the relative response of the striatum to monetary vs erotic cues was correlated with the relative motivational value of these rewards as inferred from reaction times. Similar correlations were observed in a fronto-parietal network known to be involved in attentional focus and motor readiness. Together, our results suggest that striatal reward value signals not only obey to a common currency mechanism in the absence of choice but may also serve as an input to adjust motivated behaviour accordingly.

How the brain decides when to work and when to rest: dissociation of implicit-reactive from explicit-predictive computational processes

A pervasive case of cost-benefit problem is how to allocate effort over time, i.e. deciding when to work and when to rest. An economic decision perspective would suggest that duration of effort is determined beforehand, depending on expected costs and benefits. However, the literature on exercise performance emphasizes that decisions are made on the fly, depending on physiological variables. Here, we propose and validate a general model of effort allocation that integrates these two views. In this model, a single variable, termed cost evidence, accumulates during effort and dissipates during rest, triggering effort cessation and resumption when reaching bounds. We assumed that such a basic mechanism could explain implicit adaptation, whereas the latent parameters (slopes and bounds) could be amenable to explicit anticipation. A series of behavioral experiments manipulating effort duration and difficulty was conducted in a total of 121 healthy humans to dissociate implicit-reactive from explicit-predictive computations. Results show 1) that effort and rest durations are adapted on the fly to variations in cost-evidence level, 2) that the cost-evidence fluctuations driving the behavior do not match explicit ratings of exhaustion, and 3) that actual difficulty impacts effort duration whereas expected difficulty impacts rest duration. Taken together, our findings suggest that cost evidence is implicitly monitored online, with an accumulation rate proportional to actual task difficulty. In contrast, cost-evidence bounds and dissipation rate might be adjusted in anticipation, depending on explicit task difficulty.

Imagine that ahead of you is a long time of work: when will you take a break? This sort of issue – how to allocate effort over time – has been addressed by distinct theoretical fields, with different emphasis on reactive and predictive processes. An intuitive view is that you start working, stop when you are tired, and start again when fatigue goes away. Biologically, this means that decisions are taken when some physiological variable reaches a given bound on the risk of homeostatic failure. In a more economic perspective, fatigue translates into effort cost, which must be anticipated and compared to expected benefit before engaging an action. We proposed a computational model that bridges these perspectives from sport physiology and decision theory. Decisions are made in reaction to bounds being reached by an implicit cost variable that accumulates during effort, at a rate proportional to task difficulty, and dissipates during rest. However, some latent parameters (bounds and dissipation rate) are adjusted in anticipation, depending on explicit costs and benefits. This model was supported by behavioral data obtained using a paradigm where participants squeeze a handgrip to win a monetary payoff proportional to effort duration.

The role of the striatum in effort-based decision-making in the absence of reward

Decision-making involves weighing costs against benefits, for instance, in terms of the effort it takes to obtain a reward of a given magnitude. This evaluation process has been linked to the dorsal anterior cingulate cortex (dACC) and the striatum, with activation in these brain structures reflecting the discounting effect of effort on reward. Here, we investigate how cognitive effort influences neural choice processes in the absence of an extrinsic reward. Using functional magnetic resonance imaging in humans, we used an effort-based decision-making task in which participants were required to choose between two options for a subsequent flanker task that differed in the amount of cognitive effort. Cognitive effort was manipulated by varying the proportion of incongruent trials associated with each choice option. Choice-locked activation in the striatum was higher when participants chose voluntarily for the more effortful alternative but displayed the opposite trend on forced-choice trials. The dACC revealed a similar, yet only trend-level significant, activation pattern. Our results imply that activation levels in the striatum reflect a cost-benefit analysis, in which a balance is made between effort discounting and the intrinsic motivation to choose a cognitively challenging task. Moreover, our findings indicate that it matters whether this challenge is voluntarily chosen or externally imposed. As such, the present findings contrast with classical findings on effort discounting that found reductions in striatum activation for higher effort by finding enhancements of the same neural circuits when a cognitively challenging task is voluntarily selected and does not entail the danger of losing reward.

Overlapping neural systems represent cognitive effort and reward anticipation

Anticipating a potential benefit and how difficult it will be to obtain it are valuable skills in a constantly changing environment. In the human brain, the anticipation of reward is encoded by the Anterior Cingulate Cortex (ACC) and Striatum. Naturally, potential rewards have an incentive quality, resulting in a motivational effect improving performance. Recently it has been proposed that an upcoming task requiring effort induces a similar anticipation mechanism as reward, relying on the same cortico-limbic network. However, this overlapping anticipatory activity for reward and effort has only been investigated in a perceptual task. Whether this generalizes to high-level cognitive tasks remains to be investigated. To this end, an fMRI experiment was designed to investigate anticipation of reward and effort in cognitive tasks. A mental arithmetic task was implemented, manipulating effort (difficulty), reward, and delay in reward delivery to control for temporal confounds. The goal was to test for the motivational effect induced by the expectation of bigger reward and higher effort. The results showed that the activation elicited by an upcoming difficult task overlapped with higher reward prospect in the ACC and in the striatum, thus highlighting a pivotal role of this circuit in sustaining motivated behavior.

Better get back to work: a role for motor beta desynchronization in incentive motivation

Much research has been devoted to characterizing brain representations of reward and movement. However, the mechanisms allowing expected rewards to influence motor commands remain poorly understood. Unraveling such mechanisms is crucial to providing explanations of how behavior can be driven by goals, hence accounting for apathy cases in clinics. Here, we propose that the reduction of motor beta synchrony (MBS) before movement onset could participate in this incentive motivation process. To test this hypothesis, we recorded brain activity using magnetoencenphalography (MEG) while human participants were exerting physical effort to win monetary incentives. Knowing that the payoff was proportional to the time spent above a target force, subjects spontaneously took breaks when exhausted and resumed effort production when repleted. Behavioral data indicated that the rest periods were shorter when higher incentives were at stake. MEG data showed that the amplitude of MBS reduction correlated to both incentive level and rest duration. Moreover, the time of effort initiation could be predicted by MBS reduction measured at the beginning of rest periods. Incentive effects on MBS reduction and rest duration were also correlated across subjects. Finally, Bayesian comparison between possible causal models suggested that MBS reduction mediates the impact of incentive level on rest duration. We conclude that MBS reduction could represent a neural mechanism that speeds the initiation of effort production when the effort is more rewarded.

Imaging social motivation: distinct brain mechanisms drive effort production during collaboration versus competition

Collaborative and competitive interactions have been investigated extensively so as to understand how the brain makes choices in the context of strategic games, yet such interactions are known to influence a more basic dimension of behavior: the energy invested in the task. The cognitive mechanisms that motivate effort production in social situations remain poorly understood, and their neural counterparts have not been explored so far. A dominant idea is that the motivation provided by the social context is reducible to the personal utility of effort production, which decreases in collaboration and increases in competition. Using functional magnetic resonance imaging, we scanned human participants while they produced a physical effort in a collaborative or competitive context. We found that motivation was indeed primarily driven by personal utility, which was reflected in brain regions devoted to reward processing (the ventral basal ganglia). However, subjects who departed from utility maximization, working more in collaborative situations, showed greater functional activation and anatomical volume in a brain region implicated previously in social cognition (the temporoparietal junction). Therefore, this region might mediate a purely pro-social motivation to produce greater effort in the context of collaboration. More generally, our findings suggest that the individual propensity to invest energy in collaborative work might have an identifiable counterpart in the brain functional architecture.

Neural correlates of the divergence of instrumental probability distributions

Flexible action selection requires knowledge about how alternative actions impact the environment: a "cognitive map" of instrumental contingencies. Reinforcement learning theories formalize this map as a set of stochastic relationships between actions and states, such that for any given action considered in a current state, a probability distribution is specified over possible outcome states. Here, we show that activity in the human inferior parietal lobule correlates with the divergence of such outcome distributions-a measure that reflects whether discrimination between alternative actions increases the controllability of the future-and, further, that this effect is dissociable from those of other information theoretic and motivational variables, such as outcome entropy, action values, and outcome utilities. Our results suggest that, although ultimately combined with reward estimates to generate action values, outcome probability distributions associated with alternative actions may be contrasted independently of valence computations, to narrow the scope of the action selection problem.

Neural substrates underlying effort computation in schizophrenia

The lack of initiative, drive or effort in patients with schizophrenia is linked to marked functional impairments. However, our assessment of effort and motivation is crude, relying on clinical rating scales based largely on patient recall. In order to better understand the neurobiology of effort in schizophrenia, we need more rigorous measurements of this construct. In the behavioural neuroscience literature, decades of work has been carried out developing various paradigms to examine the neural underpinnings of an animal's willingness to expend effort for a reward. Here, we shall review this literature on the nature of paradigms used in rodents to assess effort, as well as those used in humans. Next, the neurobiology of these effort-based decisions will be discussed. We shall then review what is known about effort in schizophrenia, and what might be inferred from experiments done in other human populations. Lastly, we shall discuss future directions of research that may assist in shedding light on the neurobiology of effort cost computations in schizophrenia.

What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference

It has long been assumed that people treat cognitive effort as costly, but also that such effort costs may vary greatly across individuals. Individual differences in subjective effort could present a major and pervasive confound in behavioral and neuroscience assessments, by conflating cognitive ability with cognitive motivation. Self-report cognitive effort scales have been developed, but objective measures are lacking. In this study, we use the behavioral economic approach of revealed preferences to quantify subjective effort. Specifically, we adapted a well-established discounting paradigm to measure the extent to which cognitive effort causes participants to discount monetary rewards. The resulting metrics are sensitive to both within-individual factors, including objective load and reward amount, and between-individual factors, including age and trait cognitive engagement. We further validate cognitive effort discounting by benchmarking it against well-established measures of delay discounting. The results highlight the promise and utility of behavioral economic tools for assessing trait and state influences on cognitive motivation.

Parsing Anhedonia: Translational Models of Reward-Processing Deficits in Psychopathology

The term anhedonia has long been used in the psychiatric literature to describe reward-processing dysfunction in psychopathology, especially depression and schizophrenia. Although anhedonia literally describes a lack of pleasurable experiences in everyday life, recent advances in both the basic science and clinical literatures indicate that reward deficits in these disorders are much broader than hedonic responses. In this article, we summarize some of the recent theoretical and empirical advances in understanding deficits in reward processing and their neurobehavioral mechanisms, with a particular focus on the neural underpinnings of motivation and effort-based decision making. We also highlight the potential of translational neuroscience to enhance diagnostic clarity by defining clinical symptoms in terms of underlying pathophysiology.

Effort and valuation in the brain: the effects of anticipation and execution

Neural representations of the effort deployed in performing actions, and the valence of the outcomes they yield, form the foundation of action choice. To discover whether brain areas represent effort and outcome valence together or if they represent one but not the other, we examined these variables in an explicitly orthogonal way. We did this by asking human subjects to exert one of two levels of effort to improve their chances of either winning or avoiding the loss of money. Subjects responded faster both when exerting greater effort and when exerting effort in anticipation of winning money. Using fMRI, we inspected BOLD responses during anticipation (before any action was executed) and when the outcome was delivered. In this way, we indexed BOLD signals associated with an anticipated need to exert effort and its affective consequences, as well as the effect of executed effort on the representation of outcomes. Anterior cingulate cortex and dorsal striatum (dorsal putamen) signaled the anticipation of effort independently of the prospect of winning or losing. Activity in ventral striatum (ventral putamen) was greater for better-than-expected outcomes compared with worse-than-expected outcomes, an effect attenuated in the context of having exerted greater effort. Our findings provide evidence that neural representations of anticipated actions are sensitive to the expected demands, but not to the expected value of their consequence, whereas representations of outcome value are discounted by exertion, commensurate with an integration of cost and benefit so as to approximate net value.

Neurocomputational account of how the human brain decides when to have a break

No pain, no gain: cost-benefit trade-off has been formalized in classical decision theory to account for how we choose whether to engage effort. However, how the brain decides when to have breaks in the course of effort production remains poorly understood. We propose that decisions to cease and resume work are triggered by a cost evidence accumulation signal reaching upper and lower bounds, respectively. We developed a task in which participants are free to exert a physical effort knowing that their payoff would be proportional to their effort duration. Functional MRI and magnetoencephalography recordings conjointly revealed that the theoretical cost evidence accumulation signal was expressed in proprioceptive regions (bilateral posterior insula). Furthermore, the slopes and bounds of the accumulation process were adapted to the difficulty of the task and the money at stake. Cost evidence accumulation might therefore provide a dynamical mechanistic account of how the human brain maximizes benefits while preventing exhaustion.

On the influence of reward on action-effect binding

Ideomotor theory states that the formation of anticipatory representations about the perceptual consequences of an action [i.e., action-effect (A-E) binding] provides the functional basis of voluntary action control. A host of studies have demonstrated that A-E binding occurs fast and effortlessly, yet little is known about cognitive and affective factors that influence this learning process. In the present study, we sought to test whether the motivational value of an action modulates the acquisition of A-E associations. To this end, we linked specific actions with monetary incentives during the acquisition of novel A-E mappings. In a subsequent test phase, the degree of binding was assessed by presenting the former effect stimuli as task-irrelevant response primes in a forced-choice response task, absent reward. Binding, as indexed by response priming through the former action-effects, was only found for reward-related A-E mappings. Moreover, the degree to which reward associations modulated the binding strength was predicted by individuals’ trait sensitivity to reward. These observations indicate that the association of actions and their immediate outcomes depends on the motivational value of the action during learning, as well as on the motivational disposition of the individual. On a larger scale, these findings also highlight the link between ideomotor theories and reinforcement-learning theories, providing an interesting perspective for future research on anticipatory regulation of behavior.

The Sweet Taste of Success

According to the resource-depletion model, self-control is a limited resource that is depleted after a period of exertion. Evidence consistent with this model indicates that self-control relies on glucose metabolism and glucose supplementation to depleted individuals replenishes self-control resources. In five experiments, we tested an alternative hypothesis that glucose in the oral cavity counteracts the deleterious effects of self-control depletion. We predicted a glucose mouth rinse, as opposed to an artificially sweetened placebo rinse, would lead to better self-control after depletion. In Studies 1 to 3, participants engaging in a depleting task performed significantly better on a subsequent self-control task after receiving a glucose mouth rinse, as opposed to participants rinsing with a placebo. Studies 4 and 5 replicated these findings and demonstrated that the glucose mouth rinse had no effect on self-control in nondepleted participants. Results are consistent with a neural rather than metabolic mechanism for the effect of glucose supplementation on self-control.

Longitudinal Evaluation of Resting-State fMRI After Acute Stroke With Hemiparesis

Background. Functional magnetic resonance imaging (fMRI) of motor impairment after stroke strongly depends on patient effort and capacity to make a movement. Hence fMRI has had limited use in clinical management. Alternatively, resting-state fMRI (ie, with no task) can elucidate the brain’s functional connections by determining temporal synchrony between brain regions. Objective. The authors examined whether resting-state fMRI can elucidate the disruption of functional connections within hours of ischemic stroke as well as during recovery. Methods. A total of 51 ischemic stroke patients—31 with mild-to-moderate hand deficits (National Institutes of Health Stroke Scale [NIHSS] motor score ≥1) and 20 with NIHSS score of 0—underwent resting-state fMRI at <24 hours, 7 days, and 90 days poststroke; 15 age-matched healthy individuals participated in 1 session. Using the resting-state fMRI signal from the ipsilesional motor cortex, the strength of functional connections with the contralesional motor cortex was computed. Whole-brain maps of the resting-state motor network were also generated and compared between groups and sessions. Results. Within hours poststroke, patients with motor deficits exhibited significantly lower connectivity than controls ( P = .02) and patients with no motor impairment ( P = .03). Connectivity was reestablished after 7 days in recovered (ie, NIHSS score = 0) participants. After 90 days, recovered patients exhibited normal motor connectivity; however, reduced connectivity with subcortical regions associated with effort and cognitive processing remained. Conclusion. Resting-state fMRI within hours of ischemic stroke can demonstrate the impact of stroke on functional connections throughout the brain. This tool has the potential to help select appropriate stroke therapies in an acute imaging setting and to monitor the efficacy of rehabilitation.

Contributions of the striatum to learning, motivation, and performance: an associative account

It has long been recognized that the striatum is composed of distinct functional sub-units that are part of multiple cortico-striatal-thalamic circuits. Contemporary research has focused on the contribution of striatal sub-regions to three main phenomena: learning of associations between stimuli, actions and rewards; selection between competing response alternatives; and motivational modulation of motor behavior. Recent proposals have argued for a functional division of the striatum along these lines, attributing, for example, learning to one region and performance to another. Here, we consider empirical data from human and animal studies, as well as theoretical notions from both the psychological and computational literatures, and conclude that striatal sub-regions instead differ most clearly in terms of the associations being encoded in each region.

When unconscious rewards boost cognitive task performance inefficiently: the role of consciousness in integrating value and attainability information

Research has shown that high vs. low value rewards improve cognitive task performance independent of whether they are perceived consciously or unconsciously. However, efficient performance in response to high value rewards also depends on whether or not rewards are attainable. This raises the question of whether unconscious reward processing enables people to take into account such attainability information. Building on a theoretical framework according to which conscious reward processing is required to enable higher level cognitive processing, the present research tested the hypothesis that conscious but not unconscious reward processing enables integration of reward value with attainability information. In two behavioral experiments, participants were exposed to mask high and low value coins serving as rewards on a working memory (WM) task. The likelihood for conscious processing was manipulated by presenting the coins relatively briefly (17 ms) or long and clearly visible (300 ms). Crucially, rewards were expected to be attainable or unattainable. Requirements to integrate reward value with attainability information varied across experiments. Results showed that when integration of value and attainability was required (Experiment 1), long reward presentation led to efficient performance, i.e., selectively improved performance for high value attainable rewards. In contrast, in the short presentation condition, performance was increased for high value rewards even when these were unattainable. This difference between the effects of long and short presentation time disappeared when integration of value and attainability information was not required (Experiment 2). Together these findings suggest that unconsciously processed reward information is not integrated with attainability expectancies, causing inefficient effort investment. These findings are discussed in terms of a unique role of consciousness in efficient allocation of effort to cognitive control processes.

Stochastic dynamic causal modelling of fMRI data: should we care about neural noise?

Dynamic causal modelling (DCM) was introduced to study the effective connectivity among brain regions using neuroimaging data. Until recently, DCM relied on deterministic models of distributed neuronal responses to external perturbation (e.g., sensory stimulation or task demands). However, accounting for stochastic fluctuations in neuronal activity and their interaction with task-specific processes may be of particular importance for studying state-dependent interactions. Furthermore, allowing for random neuronal fluctuations may render DCM more robust to model misspecification and finesse problems with network identification. In this article, we examine stochastic dynamic causal models (sDCM) in relation to their deterministic counterparts (dDCM) and highlight questions that can only be addressed with sDCM. We also compare the network identification performance of deterministic and stochastic DCM, using Monte Carlo simulations and an empirical case study of absence epilepsy. For example, our results demonstrate that stochastic DCM can exploit the modelling of neural noise to discriminate between direct and mediated connections. We conclude with a discussion of the added value and limitations of sDCM, in relation to its deterministic homologue.

Anything you can do, you can do better: neural substrates of incentive-based performance enhancement

Performance-based pay schemes in many organizations share the fundamental assumption that the performance level for a given task will increase as a function of the amount of incentive provided. Consistent with this notion, psychological studies have demonstrated that expectations of reward can improve performance on a plethora of different cognitive and physical tasks, ranging from problem solving to the voluntary regulation of heart rate. However, much less is understood about the neural mechanisms of incentivized performance enhancement. In particular, it is still an open question how brain areas that encode expectations about reward are able to translate incentives into improved performance across fundamentally different cognitive and physical task requirements.