Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex

Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.

Visual, delay, and oculomotor timing and tuning in macaque dorsal pulvinar during instructed and free choice memory saccades

Causal perturbations suggest that primate dorsal pulvinar plays a crucial role in target selection and saccade planning, though its basic neuronal properties remain unclear. Some functional aspects of dorsal pulvinar and interconnected frontoparietal areas-e.g. ipsilesional choice bias after inactivation-are similar. But it is unknown if dorsal pulvinar shares oculomotor properties of cortical circuitry, in particular delay and choice-related activity. We investigated such properties in macaque dorsal pulvinar during instructed and free-choice memory saccades. Most recorded units showed visual (12%), saccade-related (30%), or both types of responses (22%). Visual responses were primarily contralateral; diverse saccade-related responses were predominantly post-saccadic with a weak contralateral bias. Memory delay and pre-saccadic enhancement was infrequent (11-9%)-instead, activity was often suppressed during saccade planning (25%) and further during execution (15%). Surprisingly, only few units exhibited classical visuomotor patterns combining cue and continuous delay activity or pre-saccadic ramping; moreover, most spatially-selective neurons did not encode the upcoming decision during free-choice delay. Thus, in absence of a visible goal, the dorsal pulvinar has a limited role in prospective saccade planning, with patterns partially complementing its frontoparietal partners. Conversely, prevalent visual and post-saccadic responses imply its participation in integrating spatial goals with processing across saccades.

Intraparietal stimulation disrupts negative distractor effects in human multi-alternative decision-making

There has been debate about whether addition of an irrelevant distractor option to an otherwise binary decision influences which of the two choices is taken. We show that disparate views on this question are reconciled if distractors exert two opposing but not mutually exclusive effects. Each effect predominates in a different part of decision space: (1) a positive distractor effect predicts high-value distractors improve decision-making; (2) a negative distractor effect, of the type associated with divisive normalisation models, entails decreased accuracy with increased distractor values. Here, we demonstrate both distractor effects coexist in human decision making but in different parts of a decision space defined by the choice values. We show disruption of the medial intraparietal area (MIP) by transcranial magnetic stimulation (TMS) increases positive distractor effects at the expense of negative distractor effects. Furthermore, individuals with larger MIP volumes are also less susceptible to the disruption induced by TMS. These findings also demonstrate a causal link between MIP and the impact of distractors on decision-making via divisive normalisation.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Multiphasic value biases in fast-paced decisions

Perceptual decisions are biased toward higher-value options when overall gains can be improved. When stimuli demand immediate reactions, the neurophysiological decision process dynamically evolves through distinct phases of growing anticipation, detection, and discrimination, but how value biases are exerted through these phases remains unknown. Here, by parsing motor preparation dynamics in human electrophysiology, we uncovered a multiphasic pattern of countervailing biases operating in speeded decisions. Anticipatory preparation of higher-value actions began earlier, conferring a 'starting point' advantage at stimulus onset, but the delayed preparation of lower-value actions was steeper, conferring a value-opposed buildup-rate bias. This, in turn, was countered by a transient deflection toward the higher-value action evoked by stimulus detection. A neurally-constrained process model featuring anticipatory urgency, biased detection, and accumulation of growing stimulus-discriminating evidence, successfully captured both behavior and motor preparation dynamics. Thus, an intricate interplay of distinct biasing mechanisms serves to prioritise time-constrained perceptual decisions.

Neuronal Response to Reward and Luminance in Macaque LIP During Saccadic Choice

Recent work in decision neuroscience suggests that visual saliency can interact with reward-based choice, and the lateral intraparietal cortex (LIP) is implicated in this process. In this study, we recorded from LIP neurons while monkeys performed a two alternative choice task in which the reward and luminance associated with each offer were varied independently. We discovered that the animal's choice was dictated by the reward amount while the luminance had a marginal effect. In the LIP, neuronal activity corresponded well with the animal's choice pattern, in that a majority of reward-modulated neurons encoded the reward amount in the neuron's preferred hemifield with a positive slope. In contrast, compared to their responses to low luminance, an approximately equal proportion of luminance-sensitive neurons responded to high luminance with increased or decreased activity, leading to a much weaker population-level response. Meanwhile, in the non-preferred hemifield, the strength of encoding for reward amount and luminance was positively correlated, suggesting the integration of these two factors in the LIP. Moreover, neurons encoding reward and luminance were homogeneously distributed along the anterior-posterior axis of the LIP. Overall, our study provides further evidence supporting the neural instantiation of a priority map in the LIP in reward-based decisions.

A conflict between spatial selection and evidence accumulation in area LIP

The lateral intraparietal area (LIP) contains spatially selective neurons that help guide eye movements and, according to numerous studies, do so by accumulating sensory evidence in favor of one choice (e.g., look left) or another (look right). To examine this functional link, we trained two monkeys on an urgent motion discrimination task, a task with which the evolution of both the recorded neuronal activity and the subject's choice can be tracked millisecond by millisecond. We found that while choice accuracy increased steeply with increasing sensory evidence, at the same time, the LIP selection signal became progressively weaker, as if it hindered performance. This effect was consistent with the transient deployment of spatial attention to disparate locations away from the relevant sensory cue. The results demonstrate that spatial selection in LIP is dissociable from, and may even conflict with, evidence accumulation during informed saccadic choices.

Deficits in decision-making induced by parietal cortex inactivation are compensated at two timescales

Perceptual decisions arise through the transformation of samples of evidence into a commitment to a proposition or plan of action. Such transformation is thought to involve cortical circuits capable of computation over timescales associated with working memory, attention, and planning. Neurons in the lateral intraparietal area (LIP) play a role in these functions, and much of what is known about the neurobiology of decision-making has been influenced by studies of LIP and its network of connections. However, the causal role of LIP remains controversial. In this study, we used pharmacological and chemogenetic methods to inactivate LIP in one brain hemisphere of four rhesus monkeys. This inactivation produced biases in decisions, but the effects dissipated despite persistent neural inactivation, implying compensation by unaffected areas. Compensation occurred rapidly within an experimental session and more gradually across sessions. These findings resolve disparate studies and inform the interpretation of focal perturbations of brain function.

Neuronal correlates of selective attention and effort in visual area V4 are invariant of motivational context

Task demands can differentially engage two fundamental attention components: selectivity (spatial bias) and effort (total nonselective attentional intensity). The relative contributions and interactions of these components in modulating neuronal signals remain unknown. We recorded V4 neurons while monkeys' spatially selective attention and effort were independently controlled by adjusting either task difficulty or reward size at two locations. Neurons were robustly modulated by either selective attention or effort. Notably, increasing overall effort to improve performance at a distant site reduced neuronal responses even when performance was unchanged for receptive field stimuli. This interaction between attentional selectivity and effort was evident in single-trial spiking and can be explained by divisive normalization of spatially distributed behavioral performance at the single-neuron level. Changing motivation using task difficulty or reward produced indistinguishable effects. These results provide a cellular-level mechanism of how attention components integrate to modulate sensory processing in different motivational contexts.

Urgent Decision Making: Resolving Visuomotor Interactions at High Temporal Resolution

Measuring when exactly perceptual decisions are made is crucial for defining how the activation of specific neurons contributes to behavior. However, in traditional, nonurgent visuomotor tasks, the uncertainty of this temporal measurement is very large. This is a problem not only for delimiting the capacity of perception, but also for correctly interpreting the functional roles ascribed to choice-related neuronal responses. In this article, we review psychophysical, neurophysiological, and modeling work based on urgent visuomotor tasks in which this temporal uncertainty can be effectively overcome. The cornerstone of this work is a novel behavioral metric that describes the evolution of the subject's perceptual judgment moment by moment, allowing us to resolve numerous perceptual events that unfold within a few tens of milliseconds. In this framework, the neural distinction between perceptual evaluation and motor selection processes becomes particularly clear, as the conclusion of one is not contingent on that of the other. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Neurophysiology of Human Perceptual Decision-Making

The discovery of neural signals that reflect the dynamics of perceptual decision formation has had a considerable impact. Not only do such signals enable detailed investigations of the neural implementation of the decision-making process but they also can expose key elements of the brain's decision algorithms. For a long time, such signals were only accessible through direct animal brain recordings, and progress in human neuroscience was hampered by the limitations of noninvasive recording techniques. However, recent methodological advances are increasingly enabling the study of human brain signals that finely trace the dynamics of the unfolding decision process. In this review, we highlight how human neurophysiological data are now being leveraged to furnish new insights into the multiple processing levels involved in forming decisions, to inform the construction and evaluation of mathematical models that can explain intra- and interindividual differences, and to examine how key ancillary processes interact with core decision circuits.

Keeping in time with social and non-social stimuli: Synchronisation with auditory, visual, and audio-visual cues

Everyday social interactions require us to closely monitor, predict, and synchronise our movements with those of an interacting partner. Experimental studies of social synchrony typically examine the social-cognitive outcomes associated with synchrony, such as affiliation. On the other hand, research on the sensorimotor aspects of synchronisation generally uses non-social stimuli (e.g. a moving dot). To date, the differences in sensorimotor aspects of synchronisation to social compared to non-social stimuli remain largely unknown. The present study aims to address this gap using a verbal response paradigm where participants were asked to synchronise a 'ba' response in time with social and non-social stimuli, which were presented auditorily, visually, or audio-visually combined. For social stimuli a video/audio recording of an actor performing the same verbal 'ba' response was presented, whereas for non-social stimuli a moving dot, an auditory metronome or both combined were presented. The impact of autistic traits on participants' synchronisation performance was examined using the Autism Spectrum Quotient (AQ). Our results revealed more accurate synchronisation for social compared to non-social stimuli, suggesting that greater familiarity with and motivation in attending to social stimuli may enhance our ability to better predict and synchronise with them. Individuals with fewer autistic traits demonstrated greater social learning, as indexed through an improvement in synchronisation performance to social vs non-social stimuli across the experiment.

A Hierarchical Attractor Network Model of perceptual versus intentional decision updates

Changes of Mind are a striking example of our ability to flexibly reverse decisions and change our own actions. Previous studies largely focused on Changes of Mind in decisions about perceptual information. Here we report reversals of decisions that require integrating multiple classes of information: 1) Perceptual evidence, 2) higher-order, voluntary intentions, and 3) motor costs. In an adapted version of the random-dot motion task, participants moved to a target that matched both the external (exogenous) evidence about dot-motion direction and a preceding internally-generated (endogenous) intention about which colour to paint the dots. Movement trajectories revealed whether and when participants changed their mind about the dot-motion direction, or additionally changed their mind about which colour to choose. Our results show that decision reversals about colour intentions are less frequent in participants with stronger intentions (Exp. 1) and when motor costs of intention pursuit are lower (Exp. 2). We further show that these findings can be explained by a hierarchical, multimodal Attractor Network Model that continuously integrates higher-order voluntary intentions with perceptual evidence and motor costs. Our model thus provides a unifying framework in which voluntary actions emerge from a dynamic combination of internal action tendencies and external environmental factors, each of which can be subject to Change of Mind.

Distinct temporal difference error signals in dopamine axons in three regions of the striatum in a decision-making task

Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral, dorsomedial, and dorsolateral striatum, while mice performed a perceptual and value-based decision task. Surprisingly, dopamine axon activity was similar across all three areas. At a glance, the activity multiplexed different variables such as stimulus-associated values, confidence, and reward feedback at different phases of the task. Our modeling demonstrates, however, that these modulations can be inclusively explained by moment-by-moment changes in the expected reward, that is the temporal difference error. A major difference between areas was the overall activity level of reward responses: reward responses in dorsolateral striatum were positively shifted, lacking inhibitory responses to negative prediction errors. The differences in dopamine signals put specific constraints on the properties of behaviors controlled by dopamine in these regions.

Frontal eye field and caudate neurons make different contributions to reward-biased perceptual decisions

Many decisions require trade-offs between sensory evidence and internal preferences. Potential neural substrates include the frontal eye field (FEF) and caudate nucleus, but their distinct roles are not understood. Previously we showed that monkeys’ decisions on a direction-discrimination task with asymmetric rewards reflected a biased accumulate-to-bound decision process (Fan et al., 2018) that was affected by caudate microstimulation (Doi et al., 2020). Here we compared single-neuron activity in FEF and caudate to each other and to accumulate-to-bound model predictions derived from behavior. Task-dependent neural modulations were similar in both regions. However, choice-selective neurons in FEF, but not caudate, encoded behaviorally derived biases in the accumulation process. Baseline activity in both regions was sensitive to reward context, but this sensitivity was not reliably associated with behavioral biases. These results imply distinct contributions of FEF and caudate neurons to reward-biased decision-making and put experimental constraints on the neural implementation of accumulation-to-bound-like computations.

Confluence of Timing and Reward Biases in Perceptual Decision-Making Dynamics

Although the decisions of our daily lives often occur in the context of temporal and reward structures, the impact of such regularities on decision-making strategy is poorly understood. Here, to explore how temporal and reward context modulate strategy, we trained 2 male rhesus monkeys to perform a novel perceptual decision-making task with asymmetric rewards and time-varying evidence reliability. To model the choice and response time patterns, we developed a computational framework for fitting generalized drift-diffusion models, which flexibly accommodate diverse evidence accumulation strategies. We found that a dynamic urgency signal and leaky integration, in combination with two independent forms of reward biases, best capture behavior. We also tested how temporal structure influences urgency by systematically manipulating the temporal structure of sensory evidence, and found that the time course of urgency was affected by temporal context. Overall, our approach identified key components of cognitive mechanisms for incorporating temporal and reward structure into decisions.SIGNIFICANCE STATEMENT In everyday life, decisions are influenced by many factors, including reward structures and stimulus timing. While reward and timing have been characterized in isolation, ecologically valid decision-making involves a multiplicity of factors acting simultaneously. This raises questions about whether the same decision-making strategy is used when these two factors are concurrently manipulated. To address these questions, we trained rhesus monkeys to perform a novel decision-making task with both reward asymmetry and temporal uncertainty. In order to understand their strategy and hint at its neural mechanisms, we used the new generalized drift diffusion modeling framework to model both reward and timing mechanisms. We found two of each reward and timing mechanisms are necessary to explain our data.

Centroparietal activity mirrors the decision variable when tracking biased and time-varying sensory evidence

Decision-making is a fundamental human activity requiring explanation at the neurocognitive level. Current theoretical frameworks assume that, during sensory-based decision-making, the stimulus is sampled sequentially. The resulting evidence is accumulated over time as a decision variable until a threshold is reached and a response is initiated. Several neural signals, including the centroparietal positivity (CPP) measured from the human electroencephalogram (EEG), appear to display the accumulation-to-bound profile associated with the decision variable. Here, we evaluate the putative computational role of the CPP as a model-derived accumulation-to-bound signal, focussing on point-by-point correspondence between model predictions and data in order to go beyond simple summary measures like average slope. In two experiments, we explored the CPP under two manipulations (namely non-stationary evidence and probabilistic decision biases) that complement one another by targeting the shape and amplitude of accumulation respectively. We fit sequential sampling models to the behavioural data, and used the resulting parameters to simulate the decision variable, before directly comparing the simulated profile to the CPP waveform. In both experiments, model predictions deviated from our naïve expectations, yet showed similarities with the neurodynamic data, illustrating the importance of a formal modelling approach. The CPP appears to arise from brain processes that implement a decision variable (as formalised in sequential-sampling models) and may therefore inform our understanding of decision-making at both the representational and implementational levels of analysis, but at this point it is uncertain whether a single model can explain how the CPP varies across different kinds of task manipulation.

The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Our decisions often balance what we observe and what we desire. A prime candidate for implementing this complex balancing act is the basal ganglia pathway, but its roles have not yet been examined experimentally in detail. Here, we show that a major input station of the basal ganglia, the caudate nucleus, plays a causal role in integrating uncertain visual evidence and reward context to guide adaptive decision-making. In monkeys making saccadic decisions based on motion cues and asymmetric reward-choice associations, single caudate neurons encoded both sources of information. Electrical microstimulation at caudate sites during motion viewing affected the monkeys’ decisions. These microstimulation effects included coordinated changes in multiple computational components of the decision process that mimicked the monkeys’ similarly coordinated voluntary strategies for balancing visual and reward information. These results imply that the caudate nucleus plays causal roles in coordinating decision processes that balance external evidence and internal preferences.

Reward-Related Suppression of Neural Activity in Macaque Visual Area V4

In order for organisms to survive, they need to detect rewarding stimuli, for example, food or a mate, in a complex environment with many competing stimuli. These rewarding stimuli should be detected even if they are nonsalient or irrelevant to the current goal. The value-driven theory of attentional selection proposes that this detection takes place through reward-associated stimuli automatically engaging attentional mechanisms. But how this is achieved in the brain is not very well understood. Here, we investigate the effect of differential reward on the multiunit activity in visual area V4 of monkeys performing a perceptual judgment task. Surprisingly, instead of finding reward-related increases in neural responses to the perceptual target, we observed a large suppression at the onset of the reward indicating cues. Therefore, while previous research showed that reward increases neural activity, here we report a decrease. More suppression was caused by cues associated with higher reward than with lower reward, although neither cue was informative about the perceptually correct choice. This finding of reward-associated neural suppression further highlights normalization as a general cortical mechanism and is consistent with predictions of the value-driven attention theory.

Influence of Expected Reward on Temporal Order Judgment

Perceptual decision-making has been shown to be influenced by reward expected from alternative options or actions, but the underlying neural mechanisms are currently unknown. More specifically, it is debated whether reward effects are mediated through changes in sensory processing, later stages of decision-making, or both. To address this question, we conducted two experiments in which human participants made saccades to what they perceived to be either the first or second of two visually identical but asynchronously presented targets while we manipulated expected reward from correct and incorrect responses on each trial. By comparing reward-induced bias in target selection (i.e., reward bias) during the two experiments, we determined whether reward caused changes in sensory or decision-making processes. We found similar reward biases in the two experiments indicating that reward information mainly influenced later stages of decision-making. Moreover, the observed reward biases were independent of the individual's sensitivity to sensory signals. This suggests that reward effects were determined heuristically via modulation of decision-making processes instead of sensory processing. To further explain our findings and uncover plausible neural mechanisms, we simulated our experiments with a cortical network model and tested alternative mechanisms for how reward could exert its influence. We found that our experimental observations are more compatible with reward-dependent input to the output layer of the decision circuit. Together, our results suggest that, during a temporal judgment task, reward exerts its influence via changing later stages of decision-making (i.e., response bias) rather than early sensory processing (i.e., perceptual bias).

Choice history biases subsequent evidence accumulation

Perceptual choices depend not only on the current sensory input but also on the behavioral context, such as the history of one's own choices. Yet, it remains unknown how such history signals shape the dynamics of later decision formation. In models of decision formation, it is commonly assumed that choice history shifts the starting point of accumulation toward the bound reflecting the previous choice. We here present results that challenge this idea. We fit bounded-accumulation decision models to human perceptual choice data, and estimated bias parameters that depended on observers' previous choices. Across multiple task protocols and sensory modalities, individual history biases in overt behavior were consistently explained by a history-dependent change in the evidence accumulation, rather than in its starting point. Choice history signals thus seem to bias the interpretation of current sensory input, akin to shifting endogenous attention toward (or away from) the previously selected interpretation.

Enhanced Theta-Band Coherence Between Midfrontal and Posterior Parietal Areas Reflects Post-feedback Adjustments in the State of Outcome Uncertainty

Medial frontal cortex is currently viewed as the main hub of the performance monitoring system; upon detection of an error committed, it establishes functional connections with brain regions involved in task performance, thus leading to neural adjustments in them. Previous research has identified targets of such adjustments in the dorsolateral prefrontal cortex, posterior cortical regions, motor cortical areas, and subthalamic nucleus. Yet most of such studies involved visual tasks with relatively moderate cognitive load and strong dependence on motor inhibition - thus highlighting sensory, executive and motor effects while underestimating sensorimotor transformation and related aspects of decision making. Currently there is ample evidence that posterior parietal cortical areas are involved in task-specific neural processes of decision making (including evidence accumulation, sensorimotor transformation, attention, etc.) - yet, to our knowledge, no EEG studies have demonstrated post-error increase in functional connectivity in the theta-band between midfrontal and posterior parietal areas during performance on non-visual tasks. In the present study, we recorded EEG while subjects were performing an auditory version of the cognitively demanding attentional condensation task; this task involves rather non-straightforward stimulus-to-response mapping rules, thus, creating increased load on sensorimotor transformation. We observed strong pre-response alpha-band suppression in the left parietal area, which presumably reflected involvement of the posterior parietal cortex in task-specific decision-making processes. Negative feedback was followed by increased midfrontal theta-band power and increased functional coupling in the theta band between midfrontal and left parietal regions. This could be interpreted as activation of the performance monitoring system and top-down influence of this system on the posterior parietal regions involved in decision making, respectively. This inter-site coupling related to negative feedback was stronger for subjects who tended to commit errors with slower response times. Generally, current findings support the idea that slower errors are related to the state of outcome uncertainty caused by failures of task-specific processes, associated with posterior parietal regions.

Residual Information of Previous Decision Affects Evidence Accumulation in Current Decision

Bias in perceptual decisions can be generally defined as an effect which is controlled by factors other than the decision-relevant information (e.g., perceptual information in a perceptual task, when trials are independent). The literature on decision-making suggests two main hypotheses to account for this kind of bias: internal bias signals are derived from (a) the residual of motor signals generated to report a decision in the past, and (b) the residual of sensory information extracted from the stimulus in the past. Beside these hypotheses, this study suggests that making a decision in the past per se may bias the next decision. We demonstrate the validity of this assumption, first, by performing behavioral experiments based on the two-alternative forced-choice (TAFC) discrimination of motion direction paradigms and, then, we modified the pure drift-diffusion model (DDM) based on the accumulation-to-bound mechanism to account for the sequential effect. In both cases, the trace of the previous trial influences the current decision. Results indicate that the probability of being correct in the current decision increases if it is in line with the previously made decision even in the presence of feedback. Moreover, a modified model that keeps the previous decision information in the starting point of evidence accumulation provides a better fit to the behavioral data. Our findings suggest that the accumulated evidence in the decision-making process after crossing the bound in the previous decision can affect the parameters of information accumulation for the current decision in consecutive trials.

Context-Sensitive Computational Mechanisms of Decision Making

Real-world information is primarily sensory in nature, and understandably people attach value to the sensory information to prepare for appropriate behavioral responses. This review presents research from value-based, perceptual, and social decision-making domains, so far studied using isolated paradigms and their corresponding computational models. For example, in perceptual decision making, the sensory evidence accumulation rather than value computation becomes central to choice behavior. Furthermore, we identify cross-linkages between the perceptual and value-based domains to help us better understand generic processes pertaining to individual decision making. The purpose of this review is 2-fold. First, we identify the need for integrated study of different domains of decision making. Second, given that both our perception and valuation are influenced by the surrounding context, we suggest the integration of different types of information in decision making could be done by studying contextual influences in decision making. Future research needs to attempt toward a system-level understanding of various subprocesses involved in decision making.

A Biased Bayesian Inference for Decision-Making and Cognitive Control

Although classical decision-making studies have assumed that subjects behave in a Bayes-optimal way, the sub-optimality that causes biases in decision-making is currently under debate. Here, we propose a synthesis based on exponentially-biased Bayesian inference, including various decision-making and probability judgments with different bias levels. We arrange three major parameter estimation methods in a two-dimensional bias parameter space (prior and likelihood), of the biased Bayesian inference. Then, we discuss a neural implementation of the biased Bayesian inference on the basis of changes in weights in neural connections, which we regarded as a combination of leaky/unstable neural integrator and probabilistic population coding. Finally, we discuss mechanisms of cognitive control which may regulate the bias levels.

Ongoing, rational calibration of reward-driven perceptual biases

Decision-making is often interpreted in terms of normative computations that maximize a particular reward function for stable, average behaviors. Aberrations from the reward-maximizing solutions, either across subjects or across different sessions for the same subject, are often interpreted as reflecting poor learning or physical limitations. Here we show that such aberrations may instead reflect the involvement of additional satisficing and heuristic principles. For an asymmetric-reward perceptual decision-making task, three monkeys produced adaptive biases in response to changes in reward asymmetries and perceptual sensitivity. Their choices and response times were consistent with a normative accumulate-to-bound process. However, their context-dependent adjustments to this process deviated slightly but systematically from the reward-maximizing solutions. These adjustments were instead consistent with a rational process to find satisficing solutions based on the gradient of each monkey's reward-rate function. These results suggest new dimensions for assessing the rational and idiosyncratic aspects of flexible decision-making.

Sensory and decision-making processes underlying perceptual adaptation

Perceptual systems adapt to their inputs. As a result, prolonged exposure to particular stimuli alters judgments about subsequent stimuli. This phenomenon is commonly assumed to be sensory in origin. Changes in the decision-making process, however, may also be a component of adaptation. Here, we quantify sensory and decision-making contributions to adaptation in a facial expression paradigm. As expected, exposure to happy or sad expressions shifts the psychometric function toward the adaptor. More surprisingly, response times show both an overall decline and an asymmetry, with faster responses opposite the adapting category, implicating a substantial change in the decision-making process. Specifically, we infer that sensory changes from adaptation are accompanied by changes in how much sensory information is accumulated for the two choices. We speculate that adaptation influences implicit expectations about the stimuli one will encounter, causing modifications in the decision-making process as part of a normative response to a change in context.

Counterfactual Reasoning Underlies the Learning of Priors in Decision Making

Accurate decisions require knowledge of prior probabilities (e.g., prevalence or base rate), but it is unclear how prior probabilities are learned in the absence of a teacher. We hypothesized that humans could learn base rates from experience making decisions, even without feedback. Participants made difficult decisions about the direction of dynamic random dot motion. Across blocks of 15–42 trials, the base rate favoring left or right varied. Participants were not informed of the base rate or choice accuracy, yet they gradually biased their choices and thereby increased accuracy and confidence in their decisions. They achieved this by updating knowledge of base rate after each decision, using a counterfactual representation of confidence that simulates a neutral prior. The strategy is consistent with Bayesian updating of belief and suggests that humans represent both true confidence, which incorporates the evolving belief of the prior, and counterfactual confidence, which discounts the prior.

•

People can learn base rates without feedback and apply them to make better decisions

•

The estimate of base rate is updated based on the confidence in each decision

•

The form of confidence used is counterfactual, as if the base rate were uninformative

•

The study extends the Bayesian framework from choice to prior probability estimation

Multiple timescales of normalized value coding underlie adaptive choice behavior

Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.

Differential Bilateral Primary Motor Cortex tDCS Fails to Modulate Choice Bias and Readiness in Perceptual Decision Making

One of the critical factors that guide choice behavior is the prior bias of the decision-maker with respect to different options, namely, the relative readiness by which the decision-maker opts for a specific choice. Although previous neuroimaging work has shown decision bias related activity in the orbitofrontal cortex, intraparietal sulcus (IPS) and dorsolateral prefrontal cortex, in a recent work by Javadi et al. (2015), primary motor cortex was also implicated. By applying transcranial direct current stimulation (tDCS), they have revealed a causal role of the primary motor cortex excitability in the induction of response time (RT) differences and decision bias in the form of choice probability. The current study aimed to replicate these recent findings with an experimental design that contained a sham group to increase experimental control and an additional testing phase to investigate the possible after-effects of tDCS. The conventional decision outputs such as choice proportion and RT were analyzed along with the theory-driven estimates of choice bias and non-decision related components of RTs (e.g., motor implementation speed of choices made). None of the statistical comparisons favored the alternative hypotheses over the null hypotheses. Consequently, previous findings regarding the effect of primary motor cortex excitability on choice bias and response times could not be replicated with a more controlled experimental design that is recommended for tDCS studies (Horvath et al., 2015). This empirical discrepancy between the two studies adds to the evidence demonstrating inconsistent effects of tDCS in establishing causal relationships between cortical excitability and motor behavior.

Lateral intraparietal area (LIP) is largely effector-specific in free-choice decisions

Despite many years of intense research, there is no strong consensus about the role of the lateral intraparietal area (LIP) in decision making. One view of LIP function is that it guides spatial attention, providing a “saliency map” of the external world. If this were the case, it would contribute to target selection regardless of which action would be performed to implement the choice. On the other hand, LIP inactivation has been shown to influence spatial selection and oculomotor metrics in free-choice decisions, which are made using eye movements, arguing that it contributes to saccade decisions. To dissociate between a more general attention role and a more effector specific saccade role, we reversibly inactivated LIP while non-human primates freely selected between two targets, presented in the two hemifields, with either saccades or reaches. Unilateral LIP inactivation induced a strong choice bias to ipsilesional targets when decisions were made with saccades. Interestingly, the inactivation also caused a reduction of contralesional choices when decisions were made with reaches, albeit the effect was less pronounced. These findings suggest that LIP is part of a network for making oculomotor decisions and is largely effector-specific in free-choice decisions.

Translational Assessments of Reward and Anhedonia: A Tribute to Athina Markou

Loss of pleasure (clinically referred to as anhedonia), impairments in other reward-related processes such as reward learning, motivation, and reward valuation, and blunted affect characterize several mood and other psychiatric disorders. Despite the availability of many therapeutic options for these disorders, reward-related impairments remain challenging to treat and often persist despite alleviation of other symptoms. Lack of animal models of reward-related impairments and affect that have high construct and predictive validity is a key obstacle to developing novel treatments. This review highlights 1) guidelines to consider when developing translatable animal models; and 2) recent efforts to develop new reward-related assessments in humans and nonhuman animals that have been translated or back-translated from one species to another. The procedures described in this review are used to assess aspects of reward learning, motivated behavior, reward valuation, and affect. In several cases, researchers have attempted to implement task parameters that are as identical as possible to the parallel parameters used in existing cross-species tasks, with the goal of improving the translation of preclinical drug discovery findings to the clinic. In this regard, Dr. Athina Markou, who worked tirelessly throughout her career to understand and treat reward-related impairments across several psychiatric disorders, had great influence on conceptualizing the development and use of translational animal models of reward-related processes.

Emphasizing the "positive" in positive reinforcement: using nonbinary rewarding for training monkeys on cognitive tasks

Nonhuman primates constitute an indispensable model system for studying higher brain functions at the neurophysiological level. Studies involving these animals elucidated the neuronal mechanisms of various cognitive and executive functions, such as visual attention, working memory, and decision-making. Positive reinforcement training (PRT) constitutes the gold standard for training animals on the cognitive tasks employed in these studies. In the laboratory, PRT is usually based on application of a liquid reward as the reinforcer to strengthen the desired behavior and absence of the reward if the animal's response is wrong. By trial and error, the monkey may adapt its behavior and successfully reduce the number of error trials, and eventually learn even very sophisticated tasks. However, progress and success of the training strongly depend on reasonable error rates. If errors get too frequent, they may cause a drop in the animal's motivation to cooperate or its adaptation to high error rates and poor overall performance. We introduce in this report an alternative training regime to minimize errors and base the critical information for learning on graded rewarding. For every new task rule, the feedback to the animal is provided by different amounts of reward to distinguish the desired, optimal behavior from less optimal behavior. We applied this regime in different situations during training of visual attention tasks and analyzed behavioral performance and reaction times to evaluate its effectiveness. For both simple and complex behaviors, graded rewarding was found to constitute a powerful technique allowing for effective training without trade-off in accessible task difficulty or task performance. NEW & NOTEWORTHY Laboratory training of monkeys usually builds on providing a fixed amount of reward for the desired behavior, and no reward otherwise. We present a nonbinary, graded reward schedule to emphasize the positive, desired behavior and to keep errors on a moderate level. Using data from typical training situations, we demonstrate that graded rewards help to effectively guide the animal by success rather than errors and provide a powerful new tool for positive reinforcement training.

Dynamic Interplay of Value and Sensory Information in High-Speed Decision Making

In dynamic environments, split-second sensorimotor decisions must be prioritized according to potential payoffs to maximize overall rewards. The impact of relative value on deliberative perceptual judgments has been examined extensively [1-6], but relatively little is known about value-biasing mechanisms in the common situation where physical evidence is strong but the time to act is severely limited. In prominent decision models, a noisy but statistically stationary representation of sensory evidence is integrated over time to an action-triggering bound, and value-biases are affected by starting the integrator closer to the more valuable bound. Here, we show significant departures from this account for humans making rapid sensory-instructed action choices. Behavior was best explained by a simple model in which the evidence representation-and hence, rate of accumulation-is itself biased by value and is non-stationary, increasing over the short decision time frame. Because the value bias initially dominates, the model uniquely predicts a dynamic "turn-around" effect on low-value cues, where the accumulator first launches toward the incorrect action but is then re-routed to the correct one. This was clearly exhibited in electrophysiological signals reflecting motor preparation and evidence accumulation. Finally, we construct an extended model that implements this dynamic effect through plausible sensory neural response modulations and demonstrate the correspondence between decision signal dynamics simulated from a behavioral fit of that model and the empirical decision signals. Our findings suggest that value and sensory information can exert simultaneous and dynamically countervailing influences on the trajectory of the accumulation-to-bound process, driving rapid, sensory-guided actions.

The Role of the Lateral Intraparietal Area in (the Study of) Decision Making

Over the past two decades, neurophysiological responses in the lateral intraparietal area (LIP) have received extensive study for insight into decision making. In a parallel manner, inferred cognitive processes have enriched interpretations of LIP activity. Because of this bidirectional interplay between physiology and cognition, LIP has served as fertile ground for developing quantitative models that link neural activity with decision making. These models stand as some of the most important frameworks for linking brain and mind, and they are now mature enough to be evaluated in finer detail and integrated with other lines of investigation of LIP function. Here, we focus on the relationship between LIP responses and known sensory and motor events in perceptual decision-making tasks, as assessed by correlative and causal methods. The resulting sensorimotor-focused approach offers an account of LIP activity as a multiplexed amalgam of sensory, cognitive, and motor-related activity, with a complex and often indirect relationship to decision processes. Our data-driven focus on multiplexing (and de-multiplexing) of various response components can complement decision-focused models and provides more detailed insight into how neural signals might relate to cognitive processes such as decision making.

Developing a Nonstationary Computational Framework With Application to Modeling Dynamic Modulations in Neural Spiking Responses

OBJECTIVE

This paper aims to develop a computational model that incorporates the functional effects of modulatory covariates (such as context, task, or behavior), which dynamically alter the relationship between the stimulus and the neural response.

METHODS

We develop a general computational approach along with an efficient estimation procedure in the widely used generalized linear model (GLM) framework to characterize such nonstationary dynamics in spiking response and spatiotemporal characteristics of a neuron at the level of individual trials. The model employs a set of modulatory components, which nonlinearly interact with other stimulus-related signals to reproduce such nonstationary effects.

RESULTS

The model is tested for its ability to predict the responses of neurons in the middle temporal cortex of macaque monkeys during an eye movement task. The fitted model proves successful in capturing the fast temporal modulations in the response, reproducing the spike response temporal statistics, and accurately accounting for the neurons' dynamic spatiotemporal sensitivities, during eye movements.

CONCLUSION

The nonstationary GLM framework developed in this study can be used in cases where a time-varying behavioral or cognitive component makes GLM-based models insufficient to describe the dependencies of neural responses on the stimulus-related covariates.

SIGNIFICANCE

In addition to being quite powerful in encoding time-varying response modulations, this general framework also enables a readout of the neural code while dissociating the influence of other nonstimulus covariates. This framework will advance our ability to understand sensory processing in higher brain areas when modulated by several behavioral or cognitive variables.

Motor actions influence subsequent sensorimotor decisions

Sensorimotor decisions are influenced by factors beyond the current sensory input, but little is known about the effect of preceding motor actions. Here, we show that choice-unrelated motor actions influence subsequent sensorimotor decisions. By instructing participants to perform choice-unrelated motor responses before visuomotor decisions, we could manipulate upcoming decisions in a directed fashion. Subjects tended not to repeat the instructed motor response. Our results show that simple motor behaviors can influence subsequent sensorimotor decision.

Neural Signature of Value-Based Sensorimotor Prioritization in Humans

In situations in which impending sensory events demand fast action choices, we must be ready to prioritize higher-value courses of action to avoid missed opportunities. When such a situation first presents itself, stimulus-action contingencies and their relative value must be encoded to establish a value-biased state of preparation for an impending sensorimotor decision. Here, we sought to identify neurophysiological signatures of such processes in the human brain (both female and male). We devised a task requiring fast action choices based on the discrimination of a simple visual cue in which the differently valued sensory alternatives were presented 750-800 ms before as peripheral "targets" that specified the stimulus-action mapping for the upcoming decision. In response to the targets, we identified a discrete, transient, spatially selective signal in the event-related potential (ERP), which scaled with relative value and strongly predicted the degree of behavioral bias in the upcoming decision both across and within subjects. This signal is not compatible with any hitherto known ERP signature of spatial selection and also bears novel distinctions with respect to characterizations of value-sensitive, spatially selective activity found in sensorimotor areas of nonhuman primates. Specifically, a series of follow-up experiments revealed that the signal was reliably invoked regardless of response laterality, response modality, sensory feature, and reward valence. It was absent, however, when the response deadline was relaxed and the strategic need for biasing removed. Therefore, more than passively representing value or salience, the signal appears to play a versatile and active role in adaptive sensorimotor prioritization.SIGNIFICANCE STATEMENT In many situations such as fast-moving sports, we must be ready to act fast in response to sensory events and, in our preparation, prioritize courses of action that lead to greater rewards. Although behavioral effects of value biases in sensorimotor decision making have been widely studied, little is known about the neural processes that set these biases in place beforehand. Here, we report the discovery of a transient, spatially selective neural signal in humans that encodes the relative value of competing decision alternatives and strongly predicts behavioral value biases in decisions made ∼500 ms later. Follow-up manipulations of value differential, reward valence, response modality, sensory features, and time constraints establish that the signal reflects an active, feature- and effector-general preparatory mechanism for value-based prioritization.

Learning Contextual Reward Expectations for Value Adaptation

Substantial evidence indicates that subjective value is adapted to the statistics of reward expected within a given temporal context. However, how these contextual expectations are learned is poorly understood. To examine such learning, we exploited a recent observation that participants performing a gambling task adjust their preferences as a function of context. We show that, in the absence of contextual cues providing reward information, an average reward expectation was learned from recent past experience. Learning dependent on contextual cues emerged when two contexts alternated at a fast rate, whereas both cue-independent and cue-dependent forms of learning were apparent when two contexts alternated at a slower rate. Motivated by these behavioral findings, we reanalyzed a previous fMRI data set to probe the neural substrates of learning contextual reward expectations. We observed a form of reward prediction error related to average reward such that, at option presentation, activity in ventral tegmental area/substantia nigra and ventral striatum correlated positively and negatively, respectively, with the actual and predicted value of options. Moreover, an inverse correlation between activity in ventral tegmental area/substantia nigra (but not striatum) and predicted option value was greater in participants showing enhanced choice adaptation to context. The findings help understanding the mechanisms underlying learning of contextual reward expectation.

Decision Making and Sequential Sampling from Memory

Decisions take time, and as a rule more difficult decisions take more time. But this only raises the question of what consumes the time. For decisions informed by a sequence of samples of evidence, the answer is straightforward: more samples are available with more time. Indeed, the speed and accuracy of such decisions are explained by the accumulation of evidence to a threshold or bound. However, the same framework seems to apply to decisions that are not obviously informed by sequences of evidence samples. Here, we proffer the hypothesis that the sequential character of such tasks involves retrieval of evidence from memory. We explore this hypothesis by focusing on value-based decisions and argue that mnemonic processes can account for regularities in choice and decision time. We speculate on the neural mechanisms that link sampling of evidence from memory to circuits that represent the accumulated evidence bearing on a choice. We propose that memory processes may contribute to a wider class of decisions that conform to the regularities of choice-reaction time predicted by the sequential sampling framework.

Target-distractor synchrony affects performance in a novel motor task for studying action selection

The study of action selection in humans can present challenges of task design since our actions are usually defined by many degrees of freedom and therefore occupy a large action-space. While saccadic eye-movement offers a more constrained paradigm for investigating action selection, the study of reach-and-grasp in upper limbs has often been defined by more complex scenarios, not easily interpretable in terms of such selection. Here we present a novel motor behaviour task which addresses this by limiting the action space to a single degree of freedom in which subjects have to track (using a stylus) a vertical coloured target line displayed on a tablet computer, whilst ignoring a similarly oriented distractor line in a different colour. We ran this task with 55 subjects and showed that, in agreement with previous studies, the presence of the distractor generally increases the movement latency and directional error rate. Further, we used two distractor conditions according to whether the distractor’s location changes asynchronously or synchronously with the location of the target. We found that the asynchronous distractor yielded poorer performance than its synchronous counterpart, with significantly higher movement latencies and higher error rates. We interpret these results in an action selection framework with two actions (move left or right) and competing ‘action requests’ offered by the target and distractor. As such, the results provide insights into action selection performance in humans and supply data for directly constraining future computational models therein.