Effort discounts reward-based control allocation: A neurodynamic perspective

The amount of cognitive and neural resources allocated to a task is largely determined by the reward we can expect. However, it remains under-appreciated how this reward-expectation-based control allocation is modulated by effort expenditure. The present event-related potential study investigated this issue through the lens of neural dynamics. Thirty-four participants completed an effort-based monetary incentive delay task while their EEG was recorded. Effort demand was manipulated by adding no (low effort) or much (high effort) noise to the target. Behaviorally, participants exhibited reward-related speeding regardless of effort expenditure, as revealed by faster RTs for reward than neutral trials. Our ERP results demonstrated a widespread facilitatory influence of reward expectation on neural dynamics extending from cue evaluation as indexed by the cue-P3, to control preparation as indexed by the contingent negative variation (CNV), and finally to control engagement as indexed by the target-P3. Critically, the neural facilitation was discounted by effort expenditure during both the control-preparation and control-engagement stages instead of the cue-evaluation stage. Overall, this study provides neurodynamic evidence that control allocation is determined by reward and effort via a cost-benefit analysis.

Reward Expectation Reduces Representational Drift in the Hippocampus

Spatial memory in the hippocampus involves dynamic neural patterns that change over days, termed representational drift. While drift may aid memory updating, excessive drift could impede retrieval. Memory retrieval is influenced by reward expectation during encoding, so we hypothesized that diminished reward expectation would exacerbate representational drift. We found that high reward expectation limited drift, with CA1 representations on one day gradually re-emerging over successive trials the following day. Conversely, the absence of reward expectation resulted in increased drift, as the gradual re-emergence of the previous day's representation did not occur. At the single cell level, lowering reward expectation caused an immediate increase in the proportion of place-fields with low trial-to-trial reliability. These place fields were less likely to be reinstated the following day, underlying increased drift in this condition. In conclusion, heightened reward expectation improves memory encoding and retrieval by maintaining reliable place fields that are gradually reinstated across days, thereby minimizing representational drift.

Purkinje cells translate subjective salience into readiness to act and choice performance

The brain selectively allocates attention from a continuous stream of sensory input. This process is typically attributed to computations in distinct regions of the forebrain and midbrain. Here, we explore whether cerebellar Purkinje cells encode information about the selection of sensory inputs and could thereby contribute to non-motor forms of learning. We show that complex spikes of individual Purkinje cells change the sensory modality they encode to reflect changes in the perceived salience of sensory input. Comparisons with mouse models deficient in cerebellar plasticity suggest that changes in complex spike activity instruct potentiation of Purkinje cells simple spike firing, which is required for efficient learning. Our findings suggest that during learning, climbing fibers do not directly guide motor output, but rather contribute to a general readiness to act via changes in simple spike activity, thereby bridging the sequence from non-motor to motor functions.

Reward Expectation Differentially Modulates Global and Local Spatial Working Memory Accuracy

Although it has been suggested that reward expectation affects the performance of spatial working memory tasks, controversial results have been found in previous experiments. Hence, it is still unclear to what extent reward expectation has an effect on working memory. To clarify this question, a memory-guided saccade task was applied, in which participants were instructed to retain and reconstruct a temporospatial sequence of four locations by moving their eyes in each trial. The global- and local-level spatial working memory accuracies were calculated to determine the reward effect on the global and local level of processing in spatial working memory tasks. Although high reward expectation enhanced the encoding of spatial information, the percentage of trials in which the cued location was correctly fixated decreased with increment of reward expectation. The reconstruction of the global temporospatial sequence was enhanced by reward expectation, whereas the local reconstruction performance was not affected by reward. Furthermore, the improvements in local representations of uncued locations and local sequences were at the cost of the representation of cued locations. The results suggest that the reward effect on spatial working memory is modulated by the level of processing, which supports the flexible resource theory during maintenance.

Reward expectation modulates N2pc for target selection: Electrophysiological evidence

In an electrophysiological experiment, we investigated the effect of reward expectation on the localized attentional interference effect using a cue-target paradigm, while event-related potentials (ERPs) were recorded. A cue indicating the reward condition of each trial (incentive vs. non-incentive) was followed by the presentation of a search array containing two target items. Participants were asked to decide whether the two shape singletons (two triangles, two rectangles, or one triangle and one rectangle) among a set of circles were the same shape. Moreover, we manipulated the distance between the two targets to be adjacent to each other (Separation 1) or further apart (Separation 3 and Separation 5). Behavioral results revealed a larger reward facilitation effect for the larger target separation conditions. The N2pc component locked to the target display exhibited an interaction between reward expectation and the distance between the two targets. For non-incentive trials, the N2pc amplitude increased as the separation between the two targets increased; however, for incentive trials, the N2pc showed comparable amplitudes in the different target separation conditions. These results indicate that reward expectation regulated attentional focus to better resolve the competition between representation and selection of the two targets for acquiring possible reward outcomes.

Emergence of β-Band Oscillations in the Aged Rat Amygdala during Discrimination Learning and Decision Making Tasks

Older adults tend to use strategies that differ from those used by young adults to solve decision-making tasks. MRI experiments suggest that altered strategy use during aging can be accompanied by a change in extent of activation of a given brain region, inter-hemispheric bilateralization or added brain structures. It has been suggested that these changes reflect compensation for less effective networks to enable optimal performance. One way that communication can be influenced within and between brain networks is through oscillatory events that help structure and synchronize incoming and outgoing information. It is unknown how aging impacts local oscillatory activity within the basolateral complex of the amygdala (BLA). The present study recorded local field potentials (LFPs) and single units in old and young rats during the performance of tasks that involve discrimination learning and probabilistic decision making. We found task- and age-specific increases in power selectively within the β range (15-30 Hz). The increased β power occurred after lever presses, as old animals reached the goal location. Periods of high-power β developed over training days in the aged rats, and was greatest in early trials of a session. β Power was also greater after pressing for the large reward option. These data suggest that aging of BLA networks results in strengthened synchrony of β oscillations when older animals are learning or deciding between rewards of different size. Whether this increased synchrony reflects the neural basis of a compensatory strategy change of old animals in reward-based decision-making tasks, remains to be verified.

Manipulating the Placebo Response in Experimental Pain by Altering Doctor's Performance Style

Background: Performance is paramount in traditional healing rituals. From a Western perspective, such performative behavior can be understood principally as inducing patients’ faith in the performer’s supernatural healing powers and effecting positive changes through the same mechanisms attributed to the placebo response, which is defined as improvement of clinical outcome in individuals receiving inactive treatment. Here we examined the possibility of using theatrical performance tools, including stage directions and scripting, to reproducibly manipulate the style and content of a simulated doctor–patient encounter and influence the placebo response in experimental pain.

Methods: A total of 122 healthy volunteers (18–45 years, 76 men) exposed to experimental pain (the cold pressor test) were assessed for pain threshold and tolerance before and after receiving a placebo cream from a “doctor” impersonated by a trained actor. The actor alternated between two distinct scripts and stage directions, i.e., performance styles created by a theater director/playwright, one emulating a standard doctor–patient encounter (scenario A) and the other emphasizing attentiveness and strong suggestion, elements also present in ritual healing (scenario B). The placebo response size was calculated as the %difference in pain threshold and tolerance after exposure relative to baseline. In addition, subjects demonstrating a ≥30% increase in pain threshold or tolerance relative to baseline were defined as responders. Each encounter was videotaped in its entirety.

Results: Inspection of the videotapes confirmed the reproducibility and consistency of the distinct scenarios enacted by the “doctor”-performer. Furthermore, scenario B resulted in a significant increase in pain threshold relative to scenario A. Interestingly, this increase derived from the placebo responder subgroup; as shown by two-way analysis of variance (performance style, F = 4.30; p = 0.040; η² = 0.035; style × responder status interaction term, F = 5.21; p = 0.024) followed by post hoc analysis showing a ∼60% increase in pain threshold in responders exposed to scenario B (p = 0.020).

Conclusion: These results support the hypothesis that structured manipulation of physician’s verbal and non-verbal performance, designed to build rapport and increase faith in treatment, is feasible and may have a significant beneficial effect on the size of the response to placebo analgesia. They also demonstrate that subjects, who are not susceptible to placebo, are also not susceptible to performance style.

Timing in reward and decision processes

Sensitivity to time, including the time of reward, guides the behaviour of all organisms. Recent research suggests that all major reward structures of the brain process the time of reward occurrence, including midbrain dopamine neurons, striatum, frontal cortex and amygdala. Neuronal reward responses in dopamine neurons, striatum and frontal cortex show temporal discounting of reward value. The prediction error signal of dopamine neurons includes the predicted time of rewards. Neurons in the striatum, frontal cortex and amygdala show responses to reward delivery and activities anticipating rewards that are sensitive to the predicted time of reward and the instantaneous reward probability. Together these data suggest that internal timing processes have several well characterized effects on neuronal reward processing.

Linking reward processing to behavioral output: motor and motivational integration in the primate subthalamic nucleus

The expectation and detection of motivationally relevant events is a major determinant of goal-directed behavior and there is a strong interest in the contribution of basal ganglia in the integration of motivational processes into behavioral output. Recent research has focused on the role of the subthalamic nucleus (STN) in the motivational control of action, but it remains to be determined how information about reward is encoded in this nucleus. We recorded the activity of single neurons in the STN of two behaving monkeys to examine whether activity was influenced by the delivery of reward in an instrumental task, a Pavlovian stimulus-reward association, or outside of a task context. We confirmed preliminary findings indicating that STN neurons were sensitive not only to rewards obtained during task performance, but also to the expectation of reward when its delivery was delayed in time. Most of the modulations at the onset of reaching movement were combined with modulations following reward delivery, suggesting the convergence of signals related to the animal's movement and its outcome in the same neurons. Some neurons were also influenced by the visuomotor contingencies of the task, i.e., target location and/or movement direction. In addition, modulations were observed under conditions where reward delivery was not contingent on an instrumental response, even in the absence of a reward predictive cue. Taken as a whole, these results demonstrate a potential contribution of the STN to motivational control of behavior in the non-human primate, although problems in distinguishing neuronal signals related to reward from those related to motor behavior should be considered. Characterizing the specificity of reward processing in the STN remains challenging and could have important implications for understanding the influence of this key component of basal ganglia circuitry on emotional and motivated behaviors under normal and pathological conditions.

Oscillatory interaction between amygdala and hippocampus coordinates behavioral modulation based on reward expectation

The aim of this study is to examine how the amygdala and hippocampus interact for behavioral performance modulated by different Reward-expectations (REs). We simultaneously recorded neuronal spikes and local field potential from the basolateral amygdala and hippocampal CA1 while rats were performing a light-side discrimination task with different expectations of a high or low probability of reward delivery. Here, we report the following results. First, the rats actually modulated their behavioral performance on their expectations of a high or low probability of reward. Second, we found more neurons related to RE in the amygdala and more neurons related to task performance in the hippocampus. Third, a prominent increase in the coherence of high-frequency oscillations (HFOs) (90–150 Hz) between the amygdala and the hippocampus was present during high RE. Fourth, coherent HFOs during inter-trial intervals and theta coherence during trials had significant correlations with the behavioral goal-selection time. Finally, cross-frequency couplings of LFPs within and across the amygdala and hippocampus occurred during ITI. These results suggest that the amygdala and hippocampus have different functional roles in the present task with different REs, and the distinctive band of coherence between the amygdala and the hippocampus contributes to behavioral modulation on the basis of REs. We propose that the amygdala influences firing rates and the strength of synchronization of hippocampal neurons through coherent oscillation, which is a part of the mechanism of how reward expectations modulate goal-directed behavior.

Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement

To test the hypothesis that the basal ganglia are related to reward-oriented saccades, we examined activity of substantia nigra pars reticulata (SNr) neurons by using a one-direction-rewarded version of the memory-guided saccade task (1DR). Many SNr neurons changed (decreased or increased) their activity after and before a visual cue (post-cue and pre-cue activity). Post-cue decreases or increases tended to be larger to a contralateral cue. They were often modulated prospectively by the presence or absence of reward, either positively (enhanced in the rewarded condition) or negatively (enhanced in the nonrewarded condition). The positive reward modulation was more common among decreasing type neurons, whereas no such preference was observed among increasing type neurons. The reward-contingent decrease in SNr neuronal activity would facilitate rewarded saccades by inducing disinhibition in superior colliculus (SC) neurons. In contrast, the increase in SNr activity would suppress a saccade less selectively (rewarded or nonrewarded) by augmenting inhibition of SC neurons. The post-cue activity was often preceded by anticipatory pre-cue activity. Most typically, post-cue decrease was preceded by pre-cue decrease, selectively when the contralateral side was rewarded. This would reinforce the reward-oriented nature of SNr neuronal activity. The decreases and increases in SNr activity may be derived directly and indirectly, respectively, from the caudate (CD), where neurons show reward-contingent pre-cue and post-cue activity. These results suggest that the CD-SNr-SC mechanism would promote saccades oriented to reward.