The opportunity cost of time modulates cognitive effort

Neural basis of cognitive control signals in anterior cingulate cortex during delay discounting

Cognitive control involves allocating cognitive effort according to internal needs and task demands and the Anterior Cingulate Cortex (ACC) is hypothesized to play a central role in this process. We investigated the neural basis of cognitive control in the ACC of rats performing an adjusting-amount delay discounting task. Decision-making in this this task can be guided by using either a lever-value tracking strategy, requiring a 'resource-based' form of cognitive effort or a lever-biased strategy requiring a 'resistance-based' form of cognitive effort. We found that ACC ensembles always tightly tracked lever value on each trial, indicative of a resource-based control signal. These signals were prevalent in the neural recordings and were influenced by the delay. A shorter delay was associated with devaluing of the immediate option and a longer delay was associated with overvaluing of the immediate option. In addition, ACC theta (6-12Hz) oscillations were observed at the choice point of rats exhibiting a resistance-based strategy. These data provide candidates of neural activity patterns in the ACC that underlie the use of 'resource-based' and 'resistance-based' cognitive effort. Furthermore, these data illustrate how strategies can be engaged under different conditions in individual subjects.

The roles of intrinsic motivation and capability-related factors in cognitive effort-based decision-making

Individual differences in cognitive effort-based decision-making can be used to reveal human motivations to invest effort into a given task. Preferences among options that differ by dimensions related to demand levels (i.e., the interaction of task characteristics and performance measures) are also heavily influenced by how likely a person can succeed at a given option. However, most existing cognitive effort-based research has focused primarily on demand-related factors, leading to confounding inferences about the motivation behind these choices. This study used an adaptive algorithm to adjust relative demand levels for three cognitive tasks to investigate general and individual differences in demand preferences. The results highlight an overall pattern of individual differences in intrinsic motivation to perform challenging tasks, supporting research that found cognitive effort aversive to some but attractive to others. These results suggest that relative demand levels and intrinsic task factors drive the motivation to select an action.

Control adjustment costs limit goal flexibility: Empirical evidence and a computational account

A cornerstone of human intelligence is the ability to flexibly adjust our cognition and behavior as our goals change. For instance, achieving some goals requires efficiency, while others require caution. Adapting to these changing goals require corresponding adjustments in cognitive control (e.g., levels of attention, response thresholds). However, adjusting our control to meet new goals comes at a cost: we are better at achieving a goal in isolation than when transitioning between goals. The source of these control adjustment costs remains poorly understood, and the bulk of our understanding of such costs comes from settings in which participants transition between discrete task sets, rather than performance goals. Across four experiments, we show that adjustments in continuous control states incur a performance cost, and that a dynamical systems model can explain the source of these costs. Participants performed a single cognitively demanding task under varying performance goals (e.g., to be fast or to be accurate). We modeled control allocation to include a dynamic process of adjusting from one's current control state to a target state for a given performance goal. By incorporating inertia into this adjustment process, our model accounts for our empirical findings that people under-shoot their target control state more (i.e., exhibit larger adjustment costs) when (a) goals switch rather than remain fixed over a block (Study 1); (b) target control states are more distant from one another (Study 2); (c) less time is given to adjust to the new goal (Study 3); and (d) when anticipating having to switch goals more frequently (Study 4). Our findings characterize the costs of adjusting control to meet changing goals, and show that these costs can emerge directly from cognitive control dynamics. In so doing, they shed new light on the sources of and constraints on flexibility in human goal-directed behavior.

Wagers for work: Decomposing the costs of cognitive effort

Some aspects of cognition are more taxing than others. Accordingly, many people will avoid cognitively demanding tasks in favor of simpler alternatives. Which components of these tasks are costly, and how much, remains unknown. Here, we use a novel task design in which subjects request wages for completing cognitive tasks and a computational modeling procedure that decomposes their wages into the costs driving them. Using working memory as a test case, our approach revealed that gating new information into memory and protecting against interference are costly. Critically, other factors, like memory load, appeared less costly. Other key factors which may drive effort costs, such as error avoidance, had minimal influence on wage requests. Our approach is sensitive to individual differences, and could be used in psychiatric populations to understand the true underlying nature of apparent cognitive deficits.

Striatal and Behavioral Responses to Reward Vary by Socioeconomic Status in Adolescents

Disparities in socioeconomic status (SES) lead to unequal access to financial and social support. These disparities are believed to influence reward sensitivity, which in turn are hypothesized to shape how individuals respond to and pursue rewarding experiences. However, surprisingly little is known about how SES shapes reward sensitivity in adolescence. Here, we investigated how SES influenced adolescent responses to reward, both in behavior and the striatum-a brain region that is highly sensitive to reward. We examined responses to both immediate reward (tracked by phasic dopamine) and average reward rate fluctuations (tracked by tonic dopamine) as these distinct signals independently shape learning and motivation. Adolescents (n = 114; 12-14 years; 58 female) performed a gambling task during functional magnetic resonance imaging. We manipulated trial-by-trial reward and loss outcomes, leading to fluctuations between periods of reward scarcity and abundance. We found that a higher reward rate hastened behavioral responses, and increased guess switching, consistent with the idea that reward abundance increases response vigor and exploration. Moreover, immediate reward reinforced previously rewarding decisions (win-stay, lose-switch) and slowed responses (postreward pausing), particularly when rewards were scarce. Notably, lower-SES adolescents slowed down less after rare rewards than higher-SES adolescents. In the brain, striatal activations covaried with the average reward rate across time and showed greater activations during rewarding blocks. However, these striatal effects were diminished in lower-SES adolescents. These findings show that the striatum tracks reward rate fluctuations, which shape decisions and motivation. Moreover, lower SES appears to attenuate reward-driven behavioral and brain responses.

What Is Mental Effort: A Clinical Perspective

Although mental effort is a frequently used term, it is poorly defined and understood. Consequently, its usage is frequently loose and potentially misleading. In neuroscience research, the term is used to mean both the cognitive work that is done to meet task demands and the subjective experience of performing that work. We argue that conflating these two meanings hampers progress in understanding cognitive impairments in neuropsychiatric conditions because cognitive work and the subjective experience of it have distinct underlying mechanisms. We suggest that the most coherent and clinically useful perspective on mental effort is that it is a subjective experience. This makes a clear distinction between cognitive impairments that arise from changes in the cognitive apparatus, as in dementia and brain injury, and those that arise from subjective difficulties in carrying out the cognitive work, as in attention-deficit/hyperactivity disorder, depression, and other motivational disorders. We review recent advances in neuroscience research that suggests that the experience of effort has emerged to control task switches so as to minimize costs relative to benefits. We consider how these advances can contribute to our understanding of the experience of increased effort perception in clinical populations. This more specific framing of mental effort will offer a deeper understanding of the mechanisms of cognitive impairments in differing clinical groups and will ultimately facilitate better therapeutic interventions.

Effort Foraging Task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans

Effort-based decisions, in which people weigh potential future rewards against effort costs required to achieve those rewards involve both cognitive and physical effort, though the mechanistic relationship between them is not yet understood. Here, we use an individual differences approach to isolate and measure the computational processes underlying effort-based decisions and test the association between cognitive and physical domains. Patch foraging is an ecologically valid reward rate maximization problem with well-developed theoretical tools. We developed the Effort Foraging Task, which embedded cognitive or physical effort into patch foraging, to quantify the cost of both cognitive and physical effort indirectly, by their effects on foraging choices. Participants chose between harvesting a depleting patch, or traveling to a new patch that was costly in time and effort. Participants' exit thresholds (reflecting the reward they expected to receive by harvesting when they chose to travel to a new patch) were sensitive to cognitive and physical effort demands, allowing us to quantify the perceived effort cost in monetary terms. The indirect sequential choice style revealed effort-seeking behavior in a minority of participants (preferring high over low effort) that has apparently been missed by many previous approaches. Individual differences in cognitive and physical effort costs were positively correlated, suggesting that these are perceived and processed in common. We used canonical correlation analysis to probe the relationship of task measures to self-reported affect and motivation, and found correlations of cognitive effort with anxiety, cognitive function, behavioral activation, and self-efficacy, but no similar correlations with physical effort.

Attentional Strategies and the Transition From Subitizing to Estimation in Numerosity Perception

The common view of the transition between subitizing and numerosity estimation regimes is that there is a hard bound on the subitizing range, and beyond this range, people estimate. However, this view does not adequately address the behavioral signatures of enumeration under conditions of attentional load or in the immediate post-subitizing range. The possibility that there might exist a numerosity range where both processes of subitizing and estimation operate in conjunction has so far been ignored. Here, we investigate this new proposal, that people strategically combine the processes of subitizing and estimation to maximize accuracy and precision, given time or attentional constraints. We present a process-level account of how subitizing and estimation can be combined through strategic deployment of attention to maximize the precision of perceived numerosity given time constraints. We then describe a computational model of this account and apply it in two experimental simulations to demonstrate how it can explain key findings in prior enumeration research. While recent modeling work has argued that the behavioral signatures of enumeration can best be explained through a single numerosity system with a single form of representation, we argue that our model demonstrates how the traditional two-systems view of numerical representation accounts for behavioral data through coordination with a unified attentional mechanism, rather than a unified representation.

Evidence for a common mechanism supporting invigoration of action selection and action execution

The speed, or vigor, of our movements can vary depending on circumstances. For instance, the promise of a reward leads to faster movements. Reward also leads us to move with a lower reaction time, suggesting that the process of action selection can also be invigorated by reward. It has been proposed that invigoration of action selection and of action execution might occur through a common mechanism, and thus these aspects of behavior might be coupled. To test this hypothesis, we asked participants to make reaching movements to "shoot" through a target at varying speeds to assess whether moving more quickly was also associated with more rapid action selection. We found that, when participants were required to move with a lower velocity, the speed of their action selection was also significantly slowed. This finding was recapitulated in a further dataset in which participants determined their own movement speed, but had to move slowly to stop their movement inside the target. By reanalyzing a previous dataset, we also found evidence for the converse relationship between action execution and action selection; when pressured to select actions more rapidly, people also executed movements with higher velocity. Our results establish that invigoration of action selection and action execution vary in tandem with one another, supporting the hypothesis of a common underlying mechanism.NEW & NOTEWORTHY We show that voluntary increases in the vigor of action execution lead action selection to also occur more rapidly. Conversely, hastening action selection by imposing a deadline to act also leads to increases in movement speed. These findings provide evidence that these two distinct aspects of behavior are modulated by a common underlying mechanism.

Neurophysiological mechanisms underlying the differential effect of reward prospect on response selection and inhibition

Reward and cognitive control play crucial roles in shaping goal-directed behavior. Yet, the behavioral and neural underpinnings of interactive effects of both processes in driving our actions towards a particular goal have remained rather unclear. Given the importance of inhibitory control, we investigated the effect of reward prospect on the modulatory influence of automatic versus controlled processes during response inhibition. For this, a performance-contingent monetary reward for both correct response selection and response inhibition was added to a Simon NoGo task, which manipulates the relationship of automatic and controlled processes in Go and NoGo trials. A neurophysiological approach was used by combining EEG temporal signal decomposition and source localization methods. Compared to a non-rewarded control group, rewarded participants showed faster response execution, as well as overall lower response selection and inhibition accuracy (shifted speed-accuracy tradeoff). Interestingly, the reward group displayed a larger interference of the interactive effects of automatic versus controlled processes during response inhibition (i.e., a larger Simon NoGo effect), but not during response selection. The reward-specific behavioral effect was mirrored by the P3 amplitude, underlining the importance of stimulus-response association processes in explaining variability in response inhibition performance. The selective reward-induced neurophysiological modulation was associated with lower activation differences in relevant structures spanning the inferior frontal and parietal cortex, as well as higher activation differences in the somatosensory cortex. Taken together, this study highlights relevant neuroanatomical structures underlying selective reward effects on response inhibition and extends previous reports on the possible detrimental effect of reward-triggered performance trade-offs on cognitive control processes.

Integrated control of non-motor and motor efforts during perceptual decision-making and action execution: a pilot study

Humans daily life is characterized by a succession of voluntary actions. Since energy resources are limited, the ability to invest the appropriate amount of effort for selecting and executing these actions is a hallmark of adapted behavior. Recent studies indicate that decisions and actions share important principles, including the optimization of their duration when the context requires it. In the present pilot study, we test the hypothesis that the management of effort-related energy resources is shared between decision and action too. Healthy human subjects performed a perceptual decision task where they had to choose between two levels of effort to invest in making the decision (i.e. two levels of perceptual difficulty), and report it with a reaching movement. Crucially, the movement accuracy requirement gradually increased from trial to trial depending on participants' decision performance. Results indicate an overall moderate and non-significant impact of the increasing motor difficulty on the choice of the non-motor (decision) effort to invest in each trial and on decision performance. By contrast, motor performance strongly decreased depending on both the motor and decisional difficulties. Together, the results support the hypothesis of an integrated management of the effort-related energy resources between decision and action. They also suggest that in the present task, the mutualized resources are primarily allocated to the decision-making process to the detriment of movements.

More than a feeling: physiological measures of affect index the integration of effort costs and rewards during anticipatory effort evaluation

The notion that humans avoid effortful action is one of the oldest and most persistent in psychology. Influential theories of effort propose that effort valuations are made according to a cost-benefit trade-off: we tend to invest mental effort only when the benefits outweigh the costs. While these models provide a useful conceptual framework, the affective components of effort valuation remain poorly understood. Here, we examined whether primitive components of affective response-positive and negative valence, captured via facial electromyography (fEMG)-can be used to better understand valuations of cognitive effort. Using an effortful arithmetic task, we find that fEMG activity in the corrugator supercilii-thought to index negative valence-1) tracks the anticipation and exertion of cognitive effort and 2) is attenuated in the presence of high rewards. Together, these results suggest that activity in the corrugator reflects the integration of effort costs and rewards during effortful decision-making.

Is all mental effort equal? The role of cognitive demand-type on effort avoidance

Humans are often termed "cognitive misers" for their aversion to mental effort. Both in and outside the laboratory people often show preferences for low-effort tasks and are willing to forgo financial reward to avoid more demanding alternatives. Mental effort, however, does not seem to be ubiquitously avoided: people play crosswords, board games, and read novels, all as forms of leisure. While such activities undoubtedly require effort, the type of cognitive demands they impose appear markedly different from the tasks typically used in psychological research on mental effort (e.g., N-Back, Stroop Task, vigilance tasks). We investigate the effect disparate demands, such as tasks which require problem solving (e.g., solve the missing number: 1, 3, 7, 15, 31,?) compared to those which require rule-implementation (e.g., N-Back task), have on people's aversion to or preference for increased mental effort. Across four experiments using three different tasks, and a mixture of online and lab-based settings, we find that aversion to effort remains largely stable regardless of the types of cognitive demands a task imposes. The results are discussed in terms of other factors that might induce the pursuit of mental effort over and above the type of cognitive demands imposed by a task.

Learning when effort matters: neural dynamics underlying updating and adaptation to changes in performance efficacy

To determine how much cognitive control to invest in a task, people need to consider whether exerting control matters for obtaining rewards. In particular, they need to account for the efficacy of their performance-the degree to which rewards are determined by performance or by independent factors. Yet it remains unclear how people learn about their performance efficacy in an environment. Here we combined computational modeling with measures of task performance and EEG, to provide a mechanistic account of how people (i) learn and update efficacy expectations in a changing environment and (ii) proactively adjust control allocation based on current efficacy expectations. Across 2 studies, subjects performed an incentivized cognitive control task while their performance efficacy (the likelihood that rewards are performance-contingent or random) varied over time. We show that people update their efficacy beliefs based on prediction errors-leveraging similar neural and computational substrates as those that underpin reward learning-and adjust how much control they allocate according to these beliefs. Using computational modeling, we show that these control adjustments reflect changes in information processing, rather than the speed-accuracy tradeoff. These findings demonstrate the neurocomputational mechanism through which people learn how worthwhile their cognitive control is.

Phantom controllers: Misspecified models create the false appearance of adaptive control during value-based choice

Decision scientists have grown increasingly interested in how people adaptively control their decision making. Researchers have demonstrated that parameters governing the accumulation of evidence towards a choice, such as the decision threshold, are shaped by information available prior to or in parallel with one's evaluation of an option set (e.g., recent outcomes or choice conflict). A recent account has taken a bold leap forward in this approach, suggesting that adjustments in decision parameters can be motivated by the value of the options under consideration. This motivated control account predicts that when faced with difficult choices (similarly valued options) under time pressure, people will adaptively lower their decision threshold to ensure that they make a choice in time. This account was supported by drift diffusion modeling of a deadlined choice task, demonstrating that decision thresholds decrease for difficult relative to easy choices. Here, we reanalyze the data from this experiment, and show that evidence for this novel account does not hold up to further scrutiny. Using a more systematic and comprehensive modeling approach, we show that this previously observed threshold adjustment disappears (or even reverses) under a more complete model of the data. Importantly, we further show how this and other apparent evidence for motivated control arises as an artifact of model (mis)specification, where one model's putatively controlled decision process (e.g., value-driven threshold adjustments) can mimic another model's stimulus-driven decision processes (e.g., accumulator competition or collapsing bounds). Collectively, this work reveals crucial insights and constraints in the pursuit of understanding how control guides decision-making, and when it doesn't.

The effect of smartphone use on mental effort, learning, and creativity

We argue that scientific studies have not directly assessed the key cognitive processes affected by smartphone use. We propose that smartphone use can be disruptively habitual, with the main detrimental consequence being an inability to exert prolonged mental effort. This inability might negatively affect real-life creativity and domain-specific knowledge acquisition.

Dopaminergic medication increases motivation to exert cognitive control by reducing subjective effort costs in Parkinson's patients

Engaging in demanding mental activities requires the allocation of cognitive control, which can be effortful and aversive. Individuals thus tend to avoid exerting cognitive effort if less demanding behavioral options are available. Recent accounts propose a key role for dopamine in motivating behavior by increasing the sensitivity to rewards associated with effort exertion. Whether dopamine additionally plays a specific role in modulating the sensitivity to the costs of cognitive effort, even in the absence of any incentives, is much less clear. To address this question, we assessed cognitive effort avoidance in patients (n = 38) with Parkinson's disease, a condition characterized by loss of midbrain dopaminergic neurons, both ON and OFF dopaminergic medication and compared them to healthy controls (n = 24). Effort avoidance was assessed using the Demand Selection Task (DST), in which participants could freely choose between performing a high-demand or a low-demand version of a task-switching paradigm. Critically, participants were not offered any incentives to choose the more effortful option, nor for good performance. While healthy controls and patients OFF their dopaminergic medications consistently preferred the low-demand option, effort avoidance in patients ON dopaminergic medications was reduced compared to patients OFF, a difference which seems to lessen over trials. These differences in preference could not be explained by altered task-switching performance. Although patients ON were less accurate at detecting the different effort levels, as measured during instructed forced-choice blocks, their detection ability was not associated with effort avoidance, unlike in the healthy controls and the patients OFF. Our findings provide evidence that dopamine replacement in Parkinson's patients increases the willingness to engage in cognitively demanding behavior, and that this cannot be explained by possible effects of dopamine replacement on performance nor on the ability to detect effort demands. These results suggest that dopamine plays a role in reducing the sensitivity to effort costs that is independent of its role in enhancing the sensitivity to the benefits of effort exertion.

Choices favoring cognitive effort in a foraging environment decrease when multiple forms of effort and delay are interleaved

Cognitive and physical effort are typically regarded as costly, but demands for effort also seemingly boost the appeal of prospects under certain conditions. One contextual factor that might influence choices for or against effort is the mix of different types of demand a decision maker encounters in a given environment. In two foraging experiments, participants encountered prospective rewards that required equally long intervals of cognitive effort, physical effort, or unfilled delay. Monetary offers varied per trial, and the two experiments differed in whether the type of effort or delay cost was the same on every trial, or varied across trials. When each participant faced only one type of cost, cognitive effort persistently produced the highest acceptance rate compared to trials with an equivalent period of either physical effort or unfilled delay. We theorized that if cognitive effort were intrinsically rewarding, we would observe the same pattern of preferences when participants foraged for varying cost types in addition to rewards. Contrary to this prediction, in the second experiment, an initially higher acceptance rate for cognitive effort trials disappeared over time amid an overall decline in acceptance rates as participants gained experience with all three conditions. Our results indicate that cognitive demands may reduce the discounting effect of delays, but not because decision makers assign intrinsic value to cognitive effort. Rather, the results suggest that a cognitive effort requirement might influence contextual factors such as subjective delay duration estimates, which can be recalibrated if multiple forms of demand are interleaved.

Performance-gated deliberation: A context-adapted strategy in which urgency is opportunity cost

Finding the right amount of deliberation, between insufficient and excessive, is a hard decision making problem that depends on the value we place on our time. Average-reward, putatively encoded by tonic dopamine, serves in existing reinforcement learning theory as the opportunity cost of time, including deliberation time. Importantly, this cost can itself vary with the environmental context and is not trivial to estimate. Here, we propose how the opportunity cost of deliberation can be estimated adaptively on multiple timescales to account for non-stationary contextual factors. We use it in a simple decision-making heuristic based on average-reward reinforcement learning (AR-RL) that we call Performance-Gated Deliberation (PGD). We propose PGD as a strategy used by animals wherein deliberation cost is implemented directly as urgency, a previously characterized neural signal effectively controlling the speed of the decision-making process. We show PGD outperforms AR-RL solutions in explaining behaviour and urgency of non-human primates in a context-varying random walk prediction task and is consistent with relative performance and urgency in a context-varying random dot motion task. We make readily testable predictions for both neural activity and behaviour.

Cognitive Control as a Multivariate Optimization Problem

A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take-including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses-and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.

Reward Value Enhances Sequence Monitoring Ramping Dynamics as Ending Rewards Approach in the Rostrolateral Prefrontal Cortex

Many fundamental human behaviors contain multiple sequences performed to reach a desired outcome, such as cooking. Reward is inherently associated with sequence completion and has been shown to generally enhance cognitive control. However, the impact of reward on cognitive sequence processing remains unexplored. To address this key question, we focused on the rostrolateral prefrontal cortex (RLPFC). This area is necessary and exhibits increasing (“ramping”) activation during sequences, a dynamic that may be related to reward processing in other brain regions. To separate these dynamics, we designed a task where reward was only provided after multiple four-item sequences (“iterations”), rather than each individual sequence. Using fMRI in humans, we investigated three possible interactions of reward and sequential control signals in RLPFC: (1) with the visibility of sequential cues, i.e., memory; (2) equally across individual sequence iterations; and (3) differently across individual sequence iterations (e.g., increasing as reward approaches). Evidence from previous, nonsequential cognitive control experiments suggested that reward would uniformly change RLPFC activity across iterations and may depend on the visibility of cues. However, we found the influence of reward on RLPFC ramping increased across sequence iterations and did not interact with memory. These results suggest an active, predictive, and distinctive role for RLPFC in sequence monitoring and integration of reward information, consistent with extant literature demonstrating similar accelerating reward-related dopamine dynamics in regions connected to RLPFC. These results have implications for understanding sequential behavior in daily life, and when they go awry in disorders such as addiction.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.

Cognitive Effort Exertion Enhances Electrophysiological Responses to Rewarding Outcomes

Recent work has highlighted neural mechanisms underlying cognitive effort-related discounting of anticipated rewards. However, findings on whether effort exertion alters the subjective value of obtained rewards are inconsistent. Here, we provide a more nuanced account of how cognitive effort affects subsequent reward processing in a novel task designed to assess effort-induced modulations of the Reward Positivity, an event-related potential indexing reward-related neural activity. We found that neural responses to both gains and losses were significantly elevated in trials requiring more versus less cognitive effort. Moreover, time-frequency analysis revealed that these effects were mirrored in gain-related delta, but not in loss-related theta band activity, suggesting that people ascribed more value to high-effort outcomes. In addition, we also explored whether individual differences in behavioral effort discounting rates and reward sensitivity in the absence of effort may affect the relationship between effort exertion and subsequent reward processing. Together, our findings provide evidence that cognitive effort exertion can increase the subjective value of subsequent outcomes and that this effect may primarily rely on modulations of delta band activity.

Effects of average reward rate on vigor as a function of individual variation in striatal dopamine

RATIONALE

We constantly need to decide not only which actions to perform, but also how vigorously to perform them. In agreement with an earlier theoretical model, it has been shown that a significant portion of the variance in our action vigor can be explained by the average rate of rewards received for that action. Moreover, this invigorating effect of average reward rate was shown to vary with within-subject changes in dopamine, both in human individuals and experimental rodents.

OBJECTIVES

Here, we assessed whether individual differences in the effect of average reward rate on vigor are related to individual variation in a stable measure of striatal dopamine function in healthy, unmedicated participants.

METHODS

Forty-four participants performed a discrimination task to test the effect of average reward rate on response times to index vigor and completed an [¹⁸F]-DOPA PET scan to index striatal dopamine synthesis capacity.

RESULTS

We did not find an interaction between dopamine synthesis capacity and average reward rate across the entire group. However, a post hoc analysis revealed that participants with higher striatal dopamine synthesis capacity, particularly in the nucleus accumbens, exhibited a stronger invigorating effect of average reward rate among the 30 slowest participants.

CONCLUSIONS

Our findings provide converging evidence for a role of striatal dopamine in average reward rate signaling, thereby extending the current literature on the mechanistic link between average reward rate, vigor, and dopamine.

Dissociable influences of reward and punishment on adaptive cognitive control

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Reward at encoding but not retrieval modulates memory for detailed events

Much of the evidence suggesting that rewards improve memory performance has focused on how explicit rewards facilitate encoding of simplistic stimuli. To expand beyond this focus, the current study tested how explicit rewards presented at encoding as well as retrieval facilitate memory for information contained within complex events. In a single experimental session, participants (N = 88) encoded videos depicting naturalistic events (e.g., getting dressed) and then completed a recognition test probing their memory for different detail types (i.e., event, perceptual, or contextual) from the video stimuli. We manipulated the explicit reward associated with each video, such that accurate memory responses for half the videos were associated with high monetary incentives and half were associated with low monetary incentives. This reward manipulation was presented at either encoding or retrieval during a recognition memory test. The reward manipulation only affected memory when presented at encoding and this effect did not depend on the type of detail probed. Drift Diffusion Modelling further revealed that presenting reward information at encoding engendered greater encoding fidelity-indexed by an increase in drift rate-but did not change response caution at the time of retrieval-indexed by response threshold. Together, our results suggest that presenting reward information when encoding but not retrieving complex events has a general facilitatory effect, likely via attentional processing, on the ability to later remember precise details from the event.

The impact of pandemic-related worry on cognitive functioning and risk-taking

Here, we sought to quantify the effects of experienced fear and worry, engendered by the COVID-19 pandemic, on both cognitive abilities-speed of information processing, task-set shifting, and proactive control-as well as economic risk-taking. Leveraging a repeated-measures cross-sectional design, we examined the performance of 1517 participants, collected during the early phase of the pandemic in the US (April-June 2020), finding that self-reported pandemic-related worry predicted deficits in information processing speed and maintenance of goal-related contextual information. In a classic economic risk-taking task, we observed that worried individuals' choices were more sensitive to the described outcome probabilities of risky actions. Overall, these results elucidate the cognitive consequences of a large-scale, unpredictable, and uncontrollable stressor, which may in turn play an important role in individuals' understanding of, and adherence to safety directives both in the current crisis and future public health emergencies.

Magnitude-sensitivity: rethinking decision-making

Magnitude-sensitivity refers to the result that performance in decision-making, across domains and organisms, is affected by the total value of the possible alternatives. This simple result offers a window into fundamental issues in decision-making and has led to a reconsideration of ecological decision-making, prominent computational models of decision-making, and optimal decision-making. Moreover, magnitude-sensitivity has inspired the design of new robotic systems that exploit natural solutions and apply optimal decision-making policies. In this article, we review the key theoretical and empirical results about magnitude-sensitivity and highlight the importance that this phenomenon has for the understanding of decision-making. Furthermore, we discuss open questions and ideas for future research.

The neural basis of effort valuation: A meta-analysis of functional magnetic resonance imaging studies

Choosing how much effort to expend is critical for everyday decisions. While several neuroimaging studies have examined effort-based decision-making, results have been highly heterogeneous, leaving unclear which brain regions process effort-related costs and integrate them with rewards. We conducted two meta-analyses of functional magnetic resonance imaging data to examine consistent neural correlates of effort demands (23 studies, 15 maps, 549 participants) and net value (15 studies, 11 maps, 428 participants). The pre-supplementary motor area (pre-SMA) scaled positively with pure effort demand, whereas the ventromedial prefrontal cortex (vmPFC) showed the opposite effect. Moreover, regions that have been previously implicated in value integration in other cost domains, such as the vmPFC and ventral striatum, were consistently involved in signaling net value. The opposite response patterns of the pre-SMA and vmPFC imply that they are differentially involved in the representation of effort costs and value integration. These findings provide conclusive evidence that the vmPFC is a central node for net value computation and reveal potential brain targets to treat motivation-related disorders.

Context-sensitive valuation and learning

A variety of behavioral and neural phenomena suggest that organisms evaluate outcomes not on an absolute utility scale, but relative to some dynamic and context-sensitive reference or scale. Sometimes, as in foraging tasks, this results in sensible choices; in other situations, like choosing between options learned in different contexts, irrational choices can result. We argue that what unites and demystifies these various phenomena is that the brain's goal is not assessing utility as an end in itself, but rather comparing different options to choose the better one. In the presence of uncertainty, noise, or costly computation, adjusting options to the context can produce more accurate choices.

Acute Psychosocial Stress Increases Cognitive-Effort Avoidance

Adverse effects following acute stress are traditionally thought to reflect functional impairments of central executive-dependent cognitive-control processes. However, recent evidence demonstrates that cognitive-control application is perceived as effortful and aversive, indicating that stress-related decrements in cognitive performance could denote decreased motivation to expend effort instead. To investigate this hypothesis, we tested 40 young, healthy individuals (20 female, 20 male) under both stress and control conditions in a 2-day study that had a within-subjects design. Cognitive-effort avoidance was assessed using the demand-selection task, in which participants chose between performing low-demand and high-demand variants of a task-switching paradigm. We found that acute stress indeed increased participants' preference for less demanding behavior, whereas task-switching performance remained intact. Additional Bayesian and multiverse analyses confirmed the robustness of this effect. Our findings provide novel insights into how stressful experiences shape behavior by modulating our motivation to employ cognitive control.

Seizing the opportunity: Lifespan differences in the effects of the opportunity cost of time on cognitive control

Previous work suggests that lifespan developmental differences in cognitive control reflect maturational and aging-related changes in prefrontal cortex functioning. However, complementary explanations exist: It could be that children and older adults differ from younger adults in how they balance the effort of engaging in control against its potential benefits. Here we test whether the degree of cognitive effort expenditure depends on the opportunity cost of time (average reward rate per unit time): if the average reward rate is high, participants should withhold cognitive effort whereas if it is low, they should invest more. In Experiment 1, we examine this hypothesis in children, adolescents, younger, and older adults, by applying a reward rate manipulation in two cognitive control tasks: a modified Erikson Flanker and a task-switching paradigm. We found that young adults and adolescents reflexively withheld effort when the opportunity cost of time was high, whereas older adults and, to a lesser degree children, invested more resources to accumulate reward as quickly as possible. We tentatively interpret these results in terms of age- and task-specific differences in the processing of the opportunity cost of time. We qualify our findings in a second experiment in younger adults in which we address an alternative explanation of our results and show that the observed age differences in effort expenditure may not result from differences in task difficulty. To conclude, we think that our results present an interesting first step at relating opportunity costs to motivational processes across the lifespan. We frame the implications of further work in this area within a recent developmental model of resource-rationality, which points to developmental sweet spots in cognitive control.

The computational cost of active information sampling before decision-making under uncertainty

Humans often seek information to minimize the pervasive effect of uncertainty on decisions. Current theories explain how much knowledge people should gather before a decision, based on the cost-benefit structure of the problem at hand. Here, we demonstrate that this framework omits a crucial agent-related factor: the cognitive effort expended while collecting information. Using an active sampling model, we unveil a speed-efficiency trade-off whereby more informative samples take longer to find. Crucially, under sufficient time pressure, humans can break this trade-off, sampling both faster and more efficiently. Computational modelling demonstrates the existence of a cost of cognitive effort which, when incorporated into theoretical models, provides a better account of people's behaviour and also relates to self-reported fatigue accumulated during active sampling. Thus, the way people seek knowledge to guide their decisions is shaped not only by task-related costs and benefits, but also crucially by the quantifiable computational costs incurred.

Knockout of the Serotonin Transporter in the Rat Mildly Modulates Decisional Anhedonia

Serotonin transporter gene variance has long been considered an essential factor contributing to depression. However, meta-analyses yielded inconsistent findings recently, asking for further understanding of the link between the gene and depression-related symptoms. One key feature of depression is anhedonia. While data exist on the effect of serotonin transporter gene knockout (5-HTT^-/-) in rodents on consummatory and anticipatory anhedonia, with mixed outcomes, the effect on decisional anhedonia has not been investigated thus far. Here, we tested whether 5-HTT^-/- contributes to decisional anhedonia. To this end, we established a novel touchscreen-based go/go task of visual decision-making. During the learning of stimulus discrimination, 5-HTT^+/+ rats performed more optimal decision-making compared to 5-HTT^-/- rats at the beginning, but this difference did not persist throughout the learning period. During stimulus generalization, the generalization curves were similar between both genotypes and did not alter as the learning progress. Interestingly, the response time in 5-HTT^+/+ rats increased as the session increased in general, while 5-HTT^-/- rats tended to decrease. The response time difference might indicate that 5-HTT^-/- rats altered willingness to exert cognitive effort to the categorization of generalization stimuli. These results suggest that the effect of 5-HTT ablation on decisional anhedonia is mild and interacts with learning, explaining the discrepant findings on the link between 5-HTT gene and depression.

A mosaic of cost-benefit control over cortico-striatal circuitry

Dopamine contributes to cognitive control through well-established effects in both the striatum and cortex. Although earlier work suggests that dopamine affects cognitive control capacity, more recent work suggests that striatal dopamine may also impact on cognitive motivation. We consider the emerging perspective that striatal dopamine boosts control by making people more sensitive to the benefits versus the costs of cognitive effort, and we discuss how this sensitivity shapes competition between controlled and prepotent actions. We propose that dopamine signaling in distinct cortico-striatal subregions mediates different types of cost-benefit tradeoffs, and also discuss mechanisms for the local control of dopamine release, enabling selectivity among cortico-striatal circuits. In so doing, we show how this cost-benefit mosaic can reconcile seemingly conflicting findings about the impact of dopamine signaling on cognitive control.

Search for solutions, learning, simulation, and choice processes in suicidal behavior

Suicide may be viewed as an unfortunate outcome of failures in decision processes. Such failures occur when the demands of a crisis exceed a person's capacity to (i) search for options, (ii) learn and simulate possible futures, and (iii) make advantageous value‐based choices. Can individual‐level decision deficits and biases drive the progression of the suicidal crisis? Our overview of the evidence on this question is informed by clinical theory and grounded in reinforcement learning and behavioral economics. Cohort and case–control studies provide strong evidence that limited cognitive capacity and particularly impaired cognitive control are associated with suicidal behavior, imposing cognitive constraints on decision‐making. We conceptualize suicidal ideation as an element of impoverished consideration sets resulting from a search for solutions under cognitive constraints and mood‐congruent Pavlovian influences, a view supported by mostly indirect evidence. More compelling is the evidence of impaired learning in people with a history of suicidal behavior. We speculate that an inability to simulate alternative futures using one's model of the world may undermine alternative solutions in a suicidal crisis. The hypothesis supported by the strongest evidence is that the selection of suicide over alternatives is facilitated by a choice process undermined by randomness. Case–control studies using gambling tasks, armed bandits, and delay discounting support this claim. Future experimental studies will need to uncover real‐time dynamics of choice processes in suicidal people. In summary, the decision process framework sheds light on neurocognitive mechanisms that facilitate the progression of the suicidal crisis.

This article is categorized under:

Economics > Individual Decision‐Making

Psychology > Emotion and Motivation

Psychology > Learning

Neuroscience > Behavior

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.

An Electromyographic Analysis of the Effects of Cognitive Fatigue on Online and Anticipatory Action Control

Cognitive fatigue is a problem for the safety of critical systems (e.g., aircraft) as it can lead to accidents, especially during unexpected events. In order to determine the extent to which it disrupts adaptive capabilities, we evaluated its effect on online and anticipatory control. Despite numerous studies conducted to determine its effects, the exact mechanism(s) affected by fatigue remains to be clarified. In this study, we used distribution and electromyographic analysis to assess whether cognitive fatigue increases the capture of the incorrect automatic response or if it impairs its suppression (online control), and whether the conflict adaptation effect is reduced (anticipatory control). To this end, we evaluated the evolution of the performance over time during the Simon task, a classic conflict task that elicits incorrect automatic responses. To accentuate the presence of fatigue during the Simon task, two groups previously performed a dual-task with two different cognitive load levels to create two different levels of fatigue. The results revealed that time on task impaired online control by disrupting the capacity to suppress the incorrect response but leaving unaffected the expression of the automatic response. Furthermore, participants emphasized speed rather than accuracy with time on task, with in addition more fast guesses, suggesting that they opted for a less effortful response strategy. As the implementation of the suppression mechanism requires cognitive effort, the conjunction of these results suggests that the deficits observed may be due to disengagement of effort over time rather than reflecting an incapacity to make an effort.

Methylphenidate boosts choices of mental labor over leisure depending on striatal dopamine synthesis capacity

The cognitive enhancing effects of methylphenidate are well established, but the mechanisms remain unclear. We recently demonstrated that methylphenidate boosts cognitive motivation by enhancing the weight on the benefits of a cognitive task in a manner that depended on striatal dopamine. Here, we considered the complementary hypothesis that methylphenidate might also act by changing the weight on the opportunity cost of a cognitive task, that is, the cost of foregoing alternative opportunity. To this end, 50 healthy participants (25 women) completed a novel cognitive effort-discounting task that required choices between task and leisure. They were tested on methylphenidate, placebo, as well as the selective D2-receptor agent sulpiride, the latter to strengthen inference about dopamine receptor selectivity of methylphenidate's effects. Furthermore, they also underwent an [18F]DOPA PET scan to quantify striatal dopamine synthesis capacity. Methylphenidate boosted choices of cognitive effort over leisure across the group, and this effect was greatest in participants with more striatal dopamine synthesis capacity. The effects of sulpiride did not reach significance. This study strengthens the motivational account of methylphenidate's effects on cognition, and suggests that methylphenidate reduces the cost of mental labor by increasing striatal dopamine.

Forced choices reveal a trade-off between cognitive effort and physical pain

Cognitive effort is described as aversive, and people will generally avoid it when possible. This aversion to effort is believed to arise from a cost-benefit analysis of the actions available. The comparison of cognitive effort against other primary aversive experiences, however, remains relatively unexplored. Here, we offered participants choices between performing a cognitively demanding task or experiencing thermal pain. We found that cognitive effort can be traded off for physical pain and that people generally avoid exerting high levels of cognitive effort. We also used computational modelling to examine the aversive subjective value of effort and its effects on response behaviours. Applying this model to decision times revealed asymmetric effects of effort and pain, suggesting that cognitive effort may not share the same basic influences on avoidance behaviour as more primary aversive stimuli such as physical pain.

Dopamine promotes instrumental motivation, but reduces reward-related vigour

We can be motivated when reward depends on performance, or merely by the prospect of a guaranteed reward. Performance-dependent (contingent) reward is instrumental, relying on an internal action-outcome model, whereas motivation by guaranteed reward may minimise opportunity cost in reward-rich environments. Competing theories propose that each type of motivation should be dependent on dopaminergic activity. We contrasted these two types of motivation with a rewarded saccade task, in patients with Parkinson’s disease (PD). When PD patients were ON dopamine, they had greater response vigour (peak saccadic velocity residuals) for contingent rewards, whereas when PD patients were OFF medication, they had greater vigour for guaranteed rewards. These results support the view that reward expectation and contingency drive distinct motivational processes, and can be dissociated by manipulating dopaminergic activity. We posit that dopamine promotes goal-directed motivation, but dampens reward-driven vigour, contradictory to the prediction that increased tonic dopamine amplifies reward expectation.

What is optimal in optimal inference?

Inferring hidden structure from noisy observations is a problem addressed by Bayesian statistical learning, which aims to identify optimal models of the process that generated the observations given assumptions that constrain the space of potential solutions. Animals and machines face similar "model-selection" problems to infer latent properties and predict future states of the world. Here we review recent attempts to explain how intelligent agents address these challenges and how their solutions relate to Bayesian principles. We focus on how constraints on available information and resources affect inference and propose a general framework that uses benefit(accuracy) and accuracy(cost) curves to assess optimality under these constraints.

Predicting an Outcome Less Probable yet More Desirable than the Other

Extant research suggests that the desirability of an outcome influences the way an individual makes a prediction. The current research investigated how an outcome’s desirability influences the extent to which an individual evaluates its probability when making a prediction. Two studies were conducted using a single binary prediction based on the urn model. Individuals predicted which color—red or blue—a ball drawn from a bag would be, while being aware of the proportion of each color in the bag. The results of the first study indicated that individuals predicted the more probable outcome regardless of the probabilities of two outcomes. However, when the less probable outcome was more desirable, the proportion of predictions became significantly correlated and better calibrated to the actual probability. This result was interpreted as showing that, when motivated to predict the more desirable but less probable outcome, individuals evaluate its probability more effortfully. This interpretation was tested in the second study. When the probabiity- matching motivation was implemented, the proportion of individuals who predicted the less probable outcome increased significantly. However, when the less probable outcome was more desirable, the same motivation did not significantly increase the proportion of such individuals. Taken together, these results imply that individuals likely process the same probability informatio differently based on whether this information is useful for predicting a desirable outcome.

Biased belief updating and suboptimal choice in foraging decisions

Deciding which options to engage, and which to forego, requires developing accurate beliefs about the overall distribution of prospects. Here we adapt a classic prey selection task from foraging theory to examine how individuals keep track of an environment’s reward rate and adjust choices in response to its fluctuations. Preference shifts were most pronounced when the environment improved compared to when it deteriorated. This is best explained by a trial-by-trial learning model in which participants estimate the reward rate with upward vs. downward changes controlled by separate learning rates. A failure to adjust expectations sufficiently when an environment becomes worse leads to suboptimal choices: options that are valuable given the environmental conditions are rejected in the false expectation that better options will materialize. These findings offer a previously unappreciated parallel in the serial choice setting of observations of asymmetric updating and resulting biased (often overoptimistic) estimates in other domains.

In some types of decision-making, people must accept or forego an option without knowing what prospects might later be available. Here, the authors reveal how a key bias– asymmetric learning from negative versus positive outcomes – emerges in this type of decision.

What is bipolar disorder? A disease model of dysregulated energy expenditure

Advances in the understanding and management of bipolar disorder (BD) have been slow to emerge. Despite notable recent developments in neurosciences, our conceptualization of the nature of this mental disorder has not meaningfully progressed. One of the key reasons for this scenario is the continuing lack of a comprehensive disease model. Within the increasing complexity of modern research methods, there is a clear need for an overarching theoretical framework, in which findings are assimilated and predictions are generated. In this review and hypothesis article, we propose such a framework, one in which dysregulated energy expenditure is a primary, sufficient cause for BD. Our proposed model is centered on the disruption of the molecular and cellular network regulating energy production and expenditure, as well its potential secondary adaptations and compensatory mechanisms. We also focus on the putative longitudinal progression of this pathological process, considering its most likely periods for onset, such as critical periods that challenges energy homeostasis (e.g. neurodevelopment, social isolation), and the resulting short and long-term phenotypical manifestations.

Catecholaminergic modulation of the cost of cognitive control in healthy older adults

Catecholamines have long been associated with cognitive control and value-based decision-making. More recently, we have shown that catecholamines also modulate value-based decision-making about whether or not to engage in cognitive control. Yet it is unclear whether catecholamines influence these decisions by altering the subjective value of control. Thus, we tested whether tyrosine, a catecholamine precursor altered the subjective value of performing a demanding working memory task among healthy older adults (60-75 years). Contrary to our prediction, tyrosine administration did not significantly increase the subjective value of conducting an N-back task for reward, as a main effect. Instead, in line with our previous study, exploratory analyses indicated that drug effects varied as a function of participants' trait impulsivity scores. Specifically, tyrosine increased the subjective value of conducting an N-back task in low impulsive participants, while reducing its value in more impulsive participants. One implication of these findings is that the over-the-counter tyrosine supplements may be accompanied by an undermining effect on the motivation to perform demanding cognitive tasks, at least in certain older adults. Taken together, these findings indicate that catecholamines can alter cognitive control by modulating motivation (rather than just the ability) to exert cognitive control.