Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats

A cross-species framework for investigating perceptual evidence accumulation

Cross-species studies are important for a comprehensive understanding of brain functions. However, direct quantitative comparison of behaviors across species presents a significant challenge. To enable such comparisons in perceptual decision-making, we developed a synchronized evidence accumulation task for human and non-human animals, by aligning mechanics, stimuli, and training. The task was readily learned by rats, mice and humans, with each species exhibiting qualitatively similar performance. Quantitative model comparison revealed that all three species employed an evidence accumulation strategy, but differed in speed, accuracy, and key decision parameters. Human performance prioritized accuracy, whereas rodent performance was limited by internal time-pressure. Rats optimized reward rate, while mice appeared to switch between evidence accumulation and other strategies trial-to-trial. Together, these results reveal striking similarities and species-specific priorities in decision-making. Furthermore, the synchronized behavioral framework we present may facilitate future studies involving cross-species comparisons, such as evaluating the face validity of animal models of neuropsychiatric disorders.

Highlights

Development of an evidence accumulation task for rats and miceSynchronized video game allows direct comparisons with humansRat, mouse and human behavior are well fit by the same decision modelsModel parameters reveal species-specific priorities in accumulation strategy.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting ongoing decision processes occur before commitment to a choice. Here we combine two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free-response pulse-based evidence accumulation task, in which stimuli continue until choice is reported, and the second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find that the mDDM provides a better fit to rats' decisions in the free-response accumulation task than traditional drift diffusion models. Interestingly, on each trial we observed a period, before choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds, and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.NEW & NOTEWORTHY In this study we combine a novel pulse-based evidence accumulation task with a newly developed motion-based drift diffusion model (mDDM). In this model, we incorporate movement parameters derived from high-resolution video data to estimate parameters of the model on a trial-by-trial basis. We find that this new model is an improved description of animal choice behavior.

Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, previous work has suggested that they covary in their prevalence and that their proposed neural substrates overlap. Here we demonstrate that during decision-making, history biases and apparent lapses can both arise from a common cognitive process that is optimal under mistaken beliefs that the world is changing i.e. nonstationary. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct decision-making datasets of male rats, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Corrective feedback guides human perceptual decision-making by informing about the world state rather than rewarding its choice

Corrective feedback received on perceptual decisions is crucial for adjusting decision-making strategies to improve future choices. However, its complex interaction with other decision components, such as previous stimuli and choices, challenges a principled account of how it shapes subsequent decisions. One popular approach, based on animal behavior and extended to human perceptual decision-making, employs "reinforcement learning," a principle proven successful in reward-based decision-making. The core idea behind this approach is that decision-makers, although engaged in a perceptual task, treat corrective feedback as rewards from which they learn choice values. Here, we explore an alternative idea, which is that humans consider corrective feedback on perceptual decisions as evidence of the actual state of the world rather than as rewards for their choices. By implementing these "feedback-as-reward" and "feedback-as-evidence" hypotheses on a shared learning platform, we show that the latter outperforms the former in explaining how corrective feedback adjusts the decision-making strategy along with past stimuli and choices. Our work suggests that humans learn about what has happened in their environment rather than the values of their own choices through corrective feedback during perceptual decision-making.

Advances in cellular resolution microscopy for brain imaging in rats

Rats are used in neuroscience research because of their physiological similarities with humans and accessibility as model organisms, trainability, and behavioral repertoire. In particular, rats perform a wide range of sophisticated social, cognitive, motor, and learning behaviors within the contexts of both naturalistic and laboratory environments. Further progress in neuroscience can be facilitated by using advanced imaging methods to measure the complex neural and physiological processes during behavior in rats. However, compared with the mouse, the rat nervous system offers a set of challenges, such as larger brain size, decreased neuron density, and difficulty with head restraint. Here, we review recent advances in in vivo imaging techniques in rats with a special focus on open-source solutions for calcium imaging. Finally, we provide suggestions for both users and developers of in vivo imaging systems for rats.

Strategically managing learning during perceptual decision making

Making optimal decisions in the face of noise requires balancing short-term speed and accuracy. But a theory of optimality should account for the fact that short-term speed can influence long-term accuracy through learning. Here, we demonstrate that long-term learning is an important dynamical dimension of the speed-accuracy trade-off. We study learning trajectories in rats and formally characterize these dynamics in a theory expressed as both a recurrent neural network and an analytical extension of the drift-diffusion model that learns over time. The model reveals that choosing suboptimal response times to learn faster sacrifices immediate reward, but can lead to greater total reward. We empirically verify predictions of the theory, including a relationship between stimulus exposure and learning speed, and a modulation of reaction time by future learning prospects. We find that rats' strategies approximately maximize total reward over the full learning epoch, suggesting cognitive control over the learning process.

Trial-history biases in evidence accumulation can give rise to apparent lapses

Trial history biases and lapses are two of the most common suboptimalities observed during perceptual decision-making. These suboptimalities are routinely assumed to arise from distinct processes. However, several hints in the literature suggest that they covary in their prevalence and that their proposed neural substrates overlap - what could underlie these links? Here we demonstrate that history biases and apparent lapses can both arise from a common cognitive process that is normative under misbeliefs about non-stationarity in the world. This corresponds to an accumulation-to-bound model with history-dependent updates to the initial state of the accumulator. We test our model's predictions about the relative prevalence of history biases and lapses, and show that they are robustly borne out in two distinct rat decision-making datasets, including data from a novel reaction time task. Our model improves the ability to precisely predict decision-making dynamics within and across trials, by positing a process through which agents can generate quasi-stochastic choices.

Assessing evidence accumulation and rule learning in humans with an online game

Evidence accumulation, an essential component of perception and decision making, is frequently studied with psychophysical tasks involving noisy or ambiguous stimuli. In these tasks, participants typically receive verbal or written instructions that describe the strategy that should be used to guide decisions. Although convenient and effective, explicit instructions can influence learning and decision making strategies and can limit comparisons with animal models, in which behaviors are reinforced through feedback. Here, we developed an online video game and nonverbal training pipeline, inspired by pulse-based tasks for rodents, as an alternative to traditional psychophysical tasks used to study evidence accumulation. Using this game, we collected behavioral data from hundreds of participants trained with an explicit description of the decision rule or with experiential feedback. Participants trained with feedback alone learned the game rules rapidly and used strategies and displayed biases similar to those who received explicit instructions. Finally, by leveraging data across hundreds of participants, we show that perceptual judgments were well described by an accumulation process in which noise scaled nonlinearly with evidence, consistent with previous animal studies but inconsistent with diffusion models widely used to describe perceptual decisions in humans. These results challenge the conventional description of the accumulation process and suggest that online games provide a valuable platform to examine perceptual decision making and learning in humans. In addition, the feedback-based training pipeline developed for this game may be useful for evaluating perceptual decision making in human populations with difficulty following verbal instructions.NEW & NOTEWORTHY Perceptual uncertainty sets critical constraints on our ability to accumulate evidence and make decisions; however, its sources remain unclear. We developed a video game, and feedback-based training pipeline, to study uncertainty during decision making. Leveraging choices from hundreds of subjects, we demonstrate that human choices are inconsistent with popular diffusion models of human decision making and instead are best fit by models in which perceptual uncertainty scales nonlinearly with the strength of sensory evidence.

Response Time Distributions and the Accumulation of Visual Evidence in Freely Moving Mice

Response time (RT) distributions are histograms of the observed RTs for discriminative choices, comprising a rich source of empirical information to study perceptual processes. The drift-diffusion model (DDM), a mathematical formulation predicting decision tasks, reproduces the RT distributions, contributing to our understanding of these processes from a theoretical perspective. Notably, although the mouse is a popular model system for studying brain function and behavior, little is known about mouse perceptual RT distributions, and their description from an information-accumulation perspective. We combined an automated visual discrimination task with pharmacological micro-infusions of targeted brain regions to acquire thousands of responses from freely-moving adult mice. Both choices and escape latencies showed a strong dependency on stimulus discriminability. By applying a DDM fit to our experimental data, we found that the rate of incoming evidence (drift rate) increased with stimulus contrast but was reversibly impaired when inactivating the primary visual cortex (V1). Other brain regions involved in the decision-making process, the posterior parietal cortex (PPC) and the frontal orienting fields (FOF), also influenced relevant parameters from the DDM. The large number of empirical observations that we collected for this study allowed us to achieve accurate convergence for the model fit. Therefore, changes in the experimental conditions were mirrored by changes in model parameters, suggesting the participation of relevant brain areas in the decision-making process. This approach could help interpret future studies involving attention, discrimination, and learning in adult mice.

Humans actively sample evidence to support prior beliefs

No one likes to be wrong. Previous research has shown that participants may underweight information incompatible with previous choices, a phenomenon called confirmation bias. In this paper, we argue that a similar bias exists in the way information is actively sought. We investigate how choice influences information gathering using a perceptual choice task and find that participants sample more information from a previously chosen alternative. Furthermore, the higher the confidence in the initial choice, the more biased information sampling becomes. As a consequence, when faced with the possibility of revising an earlier decision, participants are more likely to stick with their original choice, even when incorrect. Critically, we show that agency controls this phenomenon. The effect disappears in a fixed sampling condition where presentation of evidence is controlled by the experimenter, suggesting that the way in which confirmatory evidence is acquired critically impacts the decision process. These results suggest active information acquisition plays a critical role in the propagation of strongly held beliefs over time.

Different Forms of Variability Could Explain a Difference Between Human and Rat Decision Making

When observers make rapid, difficult perceptual decisions, their response time is highly variable from trial to trial. In a visual motion discrimination task, it has been reported that human accuracy declines with increasing response time, whereas rat accuracy increases with response time. This is of interest because different mathematical theories of decision-making differ in their predictions regarding the correlation of accuracy with response time. On the premise that perceptual decision-making mechanisms are likely to be conserved among mammals, we seek to unify the rodent and primate results in a common theoretical framework. We show that a bounded drift diffusion model (DDM) can explain both effects with variable parameters: trial-to-trial variability in the starting point of the diffusion process produces the pattern typically observed in rats, whereas variability in the drift rate produces the pattern typically observed in humans. We further show that the same effects can be produced by deterministic biases, even in the absence of parameter stochasticity or parameter change within a trial.

Neurons as will and representation

Memory recollections and voluntary actions are often perceived as spontaneously generated irrespective of external stimuli. Although products of our neurons, they are only rarely accessible in humans at the neuronal level. Here I review insights gleaned from unique neurosurgical opportunities to record and stimulate single-neuron activity in people who can declare their thoughts, memories and wishes. I discuss evidence that the subjective experience of human recollection and that of voluntary action arise from the activity of two internal neuronal generators, the former from medial temporal lobe reactivation and the latter from frontoparietal preactivation. I characterize properties of these generators and their interaction, enabling flexible recruitment of memory-based choices for action as well as recruitment of action-based plans for the representation of conceptual knowledge in memories. Both internal generators operate on surprisingly explicit but different neuronal codes, which appear to arise with distinct single-neuron activity, often observed before participants' reports of conscious awareness. I discuss prediction of behaviour based on these codes, and the potential for their modulation. The prospects of editing human memories and volitions by enhancement, inception or deletion of specific, selected content raise therapeutic possibilities and ethical concerns.

MesoNet allows automated scaling and segmentation of mouse mesoscale cortical maps using machine learning

Understanding the basis of brain function requires knowledge of cortical operations over wide spatial scales and the quantitative analysis of brain activity in well-defined brain regions. Matching an anatomical atlas to brain functional data requires substantial labor and expertise. Here, we developed an automated machine learning-based registration and segmentation approach for quantitative analysis of mouse mesoscale cortical images. A deep learning model identifies nine cortical landmarks using only a single raw fluorescent image. Another fully convolutional network was adapted to delimit brain boundaries. This anatomical alignment approach was extended by adding three functional alignment approaches that use sensory maps or spatial-temporal activity motifs. We present this methodology as MesoNet, a robust and user-friendly analysis pipeline using pre-trained models to segment brain regions as defined in the Allen Mouse Brain Atlas. This Python-based toolbox can also be combined with existing methods to facilitate high-throughput data analysis.

Probabilistic discrimination of relative stimulus features in mice

During perceptual decision-making, the brain encodes the upcoming decision and the stimulus information in a mixed representation. Paradigms suitable for studying decision computations in isolation rely on stimulus comparisons, with choices depending on relative rather than absolute properties of the stimuli. The adoption of tasks requiring relative perceptual judgments in mice would be advantageous in view of the powerful tools available for the dissection of brain circuits. However, whether and how mice can perform a relative visual discrimination task has not yet been fully established. Here, we show that mice can solve a complex orientation discrimination task in which the choices are decoupled from the orientation of individual stimuli. Moreover, we demonstrate a typical discrimination acuity of 9°, challenging the common belief that mice are poor visual discriminators. We reached these conclusions by introducing a probabilistic choice model that explained behavioral strategies in 40 mice and demonstrated that the circularity of the stimulus space is an additional source of choice variability for trials with fixed difficulty. Furthermore, history biases in the model changed with task engagement, demonstrating behavioral sensitivity to the availability of cognitive resources. In conclusion, our results reveal that mice adopt a diverse set of strategies in a task that decouples decision-relevant information from stimulus-specific information, thus demonstrating their usefulness as an animal model for studying neural representations of relative categories in perceptual decision-making research.

Standardized and reproducible measurement of decision-making in mice

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.

Standardized and reproducible measurement of decision-making in mice

Progress in science requires standardized assays whose results can be readily shared, compared, and reproduced across laboratories. Reproducibility, however, has been a concern in neuroscience, particularly for measurements of mouse behavior. Here, we show that a standardized task to probe decision-making in mice produces reproducible results across multiple laboratories. We adopted a task for head-fixed mice that assays perceptual and value-based decision making, and we standardized training protocol and experimental hardware, software, and procedures. We trained 140 mice across seven laboratories in three countries, and we collected 5 million mouse choices into a publicly available database. Learning speed was variable across mice and laboratories, but once training was complete there were no significant differences in behavior across laboratories. Mice in different laboratories adopted similar reliance on visual stimuli, on past successes and failures, and on estimates of stimulus prior probability to guide their choices. These results reveal that a complex mouse behavior can be reproduced across multiple laboratories. They establish a standard for reproducible rodent behavior, and provide an unprecedented dataset and open-access tools to study decision-making in mice. More generally, they indicate a path toward achieving reproducibility in neuroscience through collaborative open-science approaches.

Lapses in perceptual decisions reflect exploration

Perceptual decision-makers often display a constant rate of errors independent of evidence strength. These ‘lapses’ are treated as a nuisance arising from noise tangential to the decision, e.g. inattention or motor errors. Here, we use a multisensory decision task in rats to demonstrate that these explanations cannot account for lapses’ stimulus dependence. We propose a novel explanation: lapses reflect a strategic trade-off between exploiting known rewarding actions and exploring uncertain ones. We tested this model’s predictions by selectively manipulating one action’s reward magnitude or probability. As uniquely predicted by this model, changes were restricted to lapses associated with that action. Finally, we show that lapses are a powerful tool for assigning decision-related computations to neural structures based on disruption experiments (here, posterior striatum and secondary motor cortex). These results suggest that lapses reflect an integral component of decision-making and are informative about action values in normal and disrupted brain states.

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Perceptual decisions entail the accumulation of evidence until a decision criterion is reached. The amount of noise in this process is inversely related to the behavioral performance of the decision-maker. Hence, reducing the amount of perceived noise could improve performance in perceptual decisions. In this study, we investigated whether providing monetary reward for correct responses in a perceptual decision-making task would enhance performance based on prior research linking noise reduction to the administration of reward. To this end, thirty-one healthy young adults carried out an incentivized dot tracking task (iDT) during recording of functional magnetic resonance imaging (fMRI). Behavioral responses were fitted to a Bayesian version of the drift-diffusion model that, among other parameters, also includes an estimate of sensory noise. Fifty percent of the trials were incentivized to compare rewarded with unrewarded trials regarding behavior, brain responses and estimates of model parameters. In order to establish a link between the noise parameter and fMRI activity, we correlated percent signal change (PSC) values from nucleus accumbens and caudate nucleus with noise levels in rewarded and unrewarded trials respectively. Although reward did not affect behavioral performance and model parameters, the fMRI analyses showed notable differences in nucleus accumbens, caudate nucleus and rostral anterior cingulate cortex in rewarded relative to unrewarded trials. Furthermore, higher PSC within nucleus accumbens was significantly associated with lower sensory noise levels, which was specific to rewarded trials. This work is consistent with previous findings on reward modulation of brain responses and marks a first step towards elucidating the effects of reward-induced noise suppression during perceptual decision-making.

Parietal Cortex Is Required for the Integration of Acoustic Evidence

Sensory-driven decisions are formed by accumulating information over time. Although parietal cortex activity is thought to represent accumulated evidence for sensory-based decisions, recent perturbation studies in rodents and non-human primates have challenged the hypothesis that these representations actually influence behavior. Here, we asked whether the parietal cortex integrates acoustic features from auditory cortical inputs during a perceptual decision-making task. If so, we predicted that selective inactivation of this projection should impair subjects' ability to accumulate sensory evidence. We trained gerbils to perform an auditory discrimination task and obtained measures of integration time as a readout of evidence accumulation capability. Minimum integration time was calculated behaviorally as the shortest stimulus duration for which subjects could discriminate the acoustic signals. Direct pharmacological inactivation of parietal cortex increased minimum integration times, suggesting its role in the behavior. To determine the specific impact of sensory evidence, we chemogenetically inactivated the excitatory projections from auditory cortex to parietal cortex and found this was sufficient to increase minimum behavioral integration times. Our signal-detection-theory-based model accurately replicated behavioral outcomes and indicated that the deficits in task performance were plausibly explained by elevated sensory noise. Together, our findings provide causal evidence that parietal cortex plays a role in the network that integrates auditory features for perceptual judgments.

Aging impairs perceptual decision-making in mice: integrating computational and neurobiological approaches

Decision-making is one of the cognitive domains which has been under-investigated in animal models of cognitive aging along with its neurobiological correlates. This study investigated the latent variables of the decision process using the hierarchical drift-diffusion model (HDDM). Neurobiological correlates of these processes were examined via immunohistochemistry. Young (n = 11, 4 months old), adult (n = 10, 10 months old), and old (n = 10, 18 months old) mice were tested in a perceptual decision-making task (i.e. two-alternative forced-choice; 2AFC). Observed data showed that there was an age-dependent decrease in the accuracy rate of old mice while response times were comparable between age groups. HDDM results revealed that age-dependent accuracy difference was a result of a decrease in the quality of evidence integration during decision-making. Significant positive correlations observed between evidence integration rate and the number of tyrosine hydroxylase positive (TH+) neurons in the ventral tegmental area (VTA) and axon terminals in dorsomedial striatum (DMS) suggest that decrease in the quality of evidence integration in aging is related to decreased function of mesocortical and nigrostriatal dopamine.

The impact of learning on perceptual decisions and its implication for speed-accuracy tradeoffs

In standard models of perceptual decision-making, noisy sensory evidence is considered to be the primary source of choice errors and the accumulation of evidence needed to overcome this noise gives rise to speed-accuracy tradeoffs. Here, we investigated how the history of recent choices and their outcomes interact with these processes using a combination of theory and experiment. We found that the speed and accuracy of performance of rats on olfactory decision tasks could be best explained by a Bayesian model that combines reinforcement-based learning with accumulation of uncertain sensory evidence. This model predicted the specific pattern of trial history effects that were found in the data. The results suggest that learning is a critical factor contributing to speed-accuracy tradeoffs in decision-making, and that task history effects are not simply biases but rather the signatures of an optimal learning strategy.

Here, the authors show that rats’ performance on olfactory decision tasks is best explained by a Bayesian model that combines reinforcement-based learning with accumulation of uncertain sensory evidence. The results suggest that learning is a critical factor contributing to speed-accuracy tradeoffs.

ChaRTr: An R toolbox for modeling choices and response times in decision-making tasks

BACKGROUND

Decision-making is the process of choosing and performing actions in response to sensory cues to achieve behavioral goals. Many mathematical models have been developed to describe the choice behavior and response time (RT) distributions of observers performing decision-making tasks. However, relatively few researchers use these models because it demands expertise in various numerical, statistical, and software techniques.

NEW METHOD

We present a toolbox - Choices and Response Times in R, or ChaRTr - that provides the user the ability to implement and test a wide variety of decision-making models ranging from classic through to modern versions of the diffusion decision model, to models with urgency signals, or collapsing boundaries.

RESULTS

In three different case studies, we demonstrate how ChaRTr can be used to effortlessly discriminate between multiple models of decision-making behavior. We also provide guidance on how to extend the toolbox to incorporate future developments in decision-making models.

COMPARISON WITH EXISTING METHOD(S)

Existing software packages surmounted some of the numerical issues but have often focused on the classical decision-making model, the diffusion decision model. Recent models that posit roles for urgency, time-varying decision thresholds, noise in various aspects of the decision-formation process or low pass filtering of sensory evidence have proven to be challenging to incorporate in a coherent software framework that permits quantitative evaluation among these competing classes of decision-making models.

CONCLUSION

ChaRTr can be used to make insightful statements about the cognitive processes underlying observed decision-making behavior and ultimately for deeper insights into decision mechanisms.

Lateral orbitofrontal cortex promotes trial-by-trial learning of risky, but not spatial, biases

Individual choices are not made in isolation but are embedded in a series of past experiences, decisions, and outcomes. The effects of past experiences on choices, often called sequential biases, are ubiquitous in perceptual and value-based decision-making, but their neural substrates are unclear. We trained rats to choose between cued guaranteed and probabilistic rewards in a task in which outcomes on each trial were independent. Behavioral variability often reflected sequential effects, including increased willingness to take risks following risky wins, and spatial 'win-stay/lose-shift' biases. Recordings from lateral orbitofrontal cortex (lOFC) revealed encoding of reward history and receipt, and optogenetic inhibition of lOFC eliminated rats' increased preference for risk following risky wins, but spared other sequential effects. Our data show that different sequential biases are neurally dissociable, and the lOFC's role in adaptive behavior promotes learning of more abstract biases (here, biases for the risky option), but not spatial ones.

The mechanistic foundation of Weber's law

Although Weber's law is the most firmly established regularity in sensation, no principled way has been identified to choose between its many proposed explanations. We investigated Weber's law by training rats to discriminate the relative intensity of sounds at the two ears at various absolute levels. These experiments revealed the existence of a psychophysical regularity, which we term time-intensity equivalence in discrimination (TIED), describing how reaction times change as a function of absolute level. The TIED enables the mathematical specification of the computational basis of Weber's law, placing strict requirements on how stimulus intensity is encoded in the stochastic activity of sensory neurons and revealing that discriminative choices must be based on bounded exact accumulation of evidence. We further demonstrate that this mechanism is not only necessary for the TIED to hold but is also sufficient to provide a virtually complete quantitative description of the behavior of the rats.

An Analysis of Decision under Risk in Rats

In 1979, Daniel Kahneman and Amos Tversky published a ground-breaking paper titled "Prospect Theory: An Analysis of Decision under Risk," which presented a behavioral economic theory that accounted for the ways in which humans deviate from economists' normative workhorse model, Expected Utility Theory [1, 2]. For example, people exhibit probability distortion (they overweight low probabilities), loss aversion (losses loom larger than gains), and reference dependence (outcomes are evaluated as gains or losses relative to an internal reference point). We found that rats exhibited many of these same biases, using a task in which rats chose between guaranteed and probabilistic rewards. However, prospect theory assumes stable preferences in the absence of learning, an assumption at odds with alternative frameworks such as animal learning theory and reinforcement learning [3-7]. Rats also exhibited trial history effects, consistent with ongoing learning. A reinforcement learning model in which state-action values were updated by the subjective value of outcomes according to prospect theory reproduced rats' nonlinear utility and probability weighting functions and also captured trial-by-trial learning dynamics.

Adaptive stimulus selection for multi-alternative psychometric functions with lapses

Psychometric functions (PFs) quantify how external stimuli affect behavior, and they play an important role in building models of sensory and cognitive processes. Adaptive stimulus-selection methods seek to select stimuli that are maximally informative about the PF given data observed so far in an experiment and thereby reduce the number of trials required to estimate the PF. Here we develop new adaptive stimulus-selection methods for flexible PF models in tasks with two or more alternatives. We model the PF with a multinomial logistic regression mixture model that incorporates realistic aspects of psychophysical behavior, including lapses and multiple alternatives for the response. We propose an information-theoretic criterion for stimulus selection and develop computationally efficient methods for inference and stimulus selection based on adaptive Markov-chain Monte Carlo sampling. We apply these methods to data from macaque monkeys performing a multi-alternative motion-discrimination task and show in simulated experiments that our method can achieve a substantial speed-up over random designs. These advances will reduce the amount of data needed to build accurate models of multi-alternative PFs and can be extended to high-dimensional PFs that would be infeasible to characterize with standard methods.

Short-Term Influence of Recent Trial History on Perceptual Choice Changes with Stimulus Strength

Perceptual decisions, especially for difficult stimuli, can be influenced by choices and outcomes in previous trials. However, it is not well understood how stimulus strength modulates the temporal characteristics as well as the magnitude of trial history influence. We addressed this question using a contrast detection task in freely moving mice. We found that, at lower as compared to higher stimulus contrast, the current choice of the mice was more influenced by choices and outcomes in the past trials and the influence emerged from a longer history. To examine the neural basis of stimulus strength-dependent history influence, we recorded from the secondary motor cortex (M2), a prefrontal region that plays an important role in cue-guided actions and memory-guided behaviors. We found that more M2 neurons conveyed information about choices on the past two trials at lower than at higher contrast. Furthermore, history-trial activity in M2 was important for decoding upcoming choice at low contrast. Thus, trial history influence of perceptual choice is adaptive to the strength of sensory evidence, which may be important for action selection in a dynamic environment.

Habits without values

Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Efficient inference for time-varying behavior during learning

The process of learning new behaviors over time is a problem of great interest in both neuroscience and artificial intelligence. However, most standard analyses of animal training data either treat behavior as fixed or track only coarse performance statistics (e.g., accuracy, bias), providing limited insight into the evolution of the policies governing behavior. To overcome these limitations, we propose a dynamic psychophysical model that efficiently tracks trial-to-trial changes in behavior over the course of training. Our model consists of a dynamic logistic regression model, parametrized by a set of time-varying weights that express dependence on sensory stimuli as well as task-irrelevant covariates, such as stimulus, choice, and answer history. Our implementation scales to large behavioral datasets, allowing us to infer 500K parameters (e.g., 10 weights over 50K trials) in minutes on a desktop computer. We optimize hyperparameters governing how rapidly each weight evolves over time using the decoupled Laplace approximation, an efficient method for maximizing marginal likelihood in non-conjugate models. To illustrate performance, we apply our method to psychophysical data from both rats and human subjects learning a delayed sensory discrimination task. The model successfully tracks the psychophysical weights of rats over the course of training, capturing day-to-day and trial-to-trial fluctuations that underlie changes in performance, choice bias, and dependencies on task history. Finally, we investigate why rats frequently make mistakes on easy trials, and suggest that apparent lapses can be explained by sub-optimal weighting of known task covariates.

Decision Making through Integration of Sensory Evidence at Prolonged Timescales

When multiple pieces of information bear on a decision, the best approach is to combine the evidence provided by each one. Evidence integration models formalize the computations underlying this process [1-3], explain human perceptual discrimination behavior [4-9], and correspond to neuronal responses elicited by discrimination tasks [10-14]. These findings suggest that evidence integration is key to understanding the neural basis of decision making [15-18]. But while evidence integration has most often been studied with simple tasks that limit deliberation to relatively brief periods, many natural decisions unfold over much longer durations. Neural network models imply acute limitations on the timescale of evidence integration [19-23], and it is currently unknown whether existing computational insights can generalize beyond rapid judgments. Here, we introduce a new psychophysical task and report model-based analyses of human behavior that demonstrate evidence integration at long timescales. Our task requires probabilistic inference using brief samples of visual evidence that are separated in time by long and unpredictable gaps. We show through several quantitative assays how decision making can approximate a normative integration process that extends over tens of seconds without accruing significant memory leak or noise. These results support the generalization of evidence integration models to a broader class of behaviors while posing new challenges for models of how these computations are implemented in biological networks.

Imaging Cortical Dynamics in GCaMP Transgenic Rats with a Head-Mounted Widefield Macroscope

Widefield imaging of calcium dynamics is an emerging method for mapping regional neural activity but is currently limited to restrained animals. Here we describe cScope, a head-mounted widefield macroscope developed to image large-scale cortical dynamics in rats during natural behavior. cScope provides a 7.8 × 4 mm field of view and dual illumination paths for both fluorescence and hemodynamic correction and can be fabricated at low cost using readily attainable components. We also report the development of Thy-1 transgenic rat strains with widespread neuronal expression of the calcium indicator GCaMP6f. We combined these two technologies to image large-scale calcium dynamics in the dorsal neocortex during a visual evidence accumulation task. Quantitative analysis of task-related dynamics revealed multiple regions having neural signals that encode behavioral choice and sensory evidence. Our results provide a new transgenic resource for calcium imaging in rats and extend the domain of head-mounted microscopes to larger-scale cortical dynamics. VIDEO ABSTRACT.

Strategic and Dynamic Temporal Weighting for Perceptual Decisions in Humans and Macaques

Perceptual decision-making is often modeled as the accumulation of sensory evidence over time. Recent studies using psychophysical reverse correlation have shown that even though the sensory evidence is stationary over time, subjects may exhibit a time-varying weighting strategy, weighting some stimulus epochs more heavily than others. While previous work has explained time-varying weighting as a consequence of static decision mechanisms (e.g., decision bound or leak), here we show that time-varying weighting can reflect strategic adaptation to stimulus statistics, and thus can readily take a number of forms. We characterized the temporal weighting strategies of humans and macaques performing a motion discrimination task in which the amount of information carried by the motion stimulus was manipulated over time. Both species could adapt their temporal weighting strategy to match the time-varying statistics of the sensory stimulus. When early stimulus epochs had higher mean motion strength than late, subjects adopted a pronounced early weighting strategy, where early information was weighted more heavily in guiding perceptual decisions. When the mean motion strength was greater in later stimulus epochs, in contrast, subjects shifted to a marked late weighting strategy. These results demonstrate that perceptual decisions involve a temporally flexible weighting process in both humans and monkeys, and introduce a paradigm with which to manipulate sensory weighting in decision-making tasks.

Visual Evidence Accumulation Guides Decision-Making in Unrestrained Mice

The ability to manipulate neural activity with precision is an asset in uncovering neural circuits for decision-making. Diverse tools for manipulating neurons are available for mice, but their feasibility remains unclear, especially when decisions require accumulating visual evidence. For example, whether mice' decisions reflect leaky accumulation is unknown, as are the relevant/irrelevant factors that influence decisions. Further, causal circuits for visual evidence accumulation are poorly understood. To address this, we measured decisions in mice judging the fluctuating rate of a flash sequence. An initial analysis (>500,000 trials, 29 male and female mice) demonstrated that information throughout the 1000 ms trial influenced choice, with early information most influential. This suggests that information persists in neural circuits for ∼1000 ms with minimal accumulation leak. Next, in a subset of animals, we probed strategy more extensively and found that although animals were influenced by stimulus rate, they were unable to entirely suppress the influence of stimulus brightness. Finally, we identified anteromedial (AM) visual area via retinotopic mapping and optogenetically inhibited it using JAWS. Light activation biased choices in both injected and uninjected animals, demonstrating that light alone influences behavior. By varying stimulus-response contingency while holding stimulated hemisphere constant, we surmounted this obstacle to demonstrate that AM suppression biases decisions. By leveraging a large dataset to quantitatively characterize decision-making behavior, we establish mice as suitable for neural circuit manipulation studies. Further, by demonstrating that mice accumulate visual evidence, we demonstrate that this strategy for reducing uncertainty in decision-making is used by animals with diverse visual systems.SIGNIFICANCE STATEMENT To connect behaviors to their underlying neural mechanism, a deep understanding of behavioral strategy is needed. This understanding is incomplete for mice. To surmount this, we measured the outcome of >500,000 decisions made by 29 mice trained to judge visual stimuli and performed behavioral/optogenetic manipulations in smaller subsets. Our analyses offer new insights into mice' decision-making strategies and compares them with those of other species. We then disrupted neural activity in a candidate neural structure and examined the effect on decisions. Our findings establish mice as suitable for visual accumulation of evidence decisions. Further, the results highlight similarities in decision-making strategies across very different species.

Rats adopt the optimal timescale for evidence integration in a dynamic environment

Decision making in dynamic environments requires discounting old evidence that may no longer inform the current state of the world. Previous work found that humans discount old evidence in a dynamic environment, but do not discount at the optimal rate. Here we investigated whether rats can optimally discount evidence in a dynamic environment by adapting the timescale over which they accumulate evidence. Using discrete evidence pulses, we exactly compute the optimal inference process. We show that the optimal timescale for evidence discounting depends on both the stimulus statistics and noise in sensory processing. When both of these components are taken into account, rats accumulate and discount evidence with the optimal timescale. Finally, by changing the volatility of the environment, we demonstrate experimental control over the rats' accumulation timescale. The mechanisms supporting integration are a subject of extensive study, and experimental control over these timescales may open new avenues of investigation.

Single-trial dynamics explain magnitude sensitive decision making

BACKGROUND

Previous research has reported or predicted, on the basis of theoretical and computational work, magnitude sensitive reaction times. Magnitude sensitivity can arise (1) as a function of single-trial dynamics and/or (2) as recent computational work has suggested, while single-trial dynamics may be magnitude insensitive, magnitude sensitivity could arise as a function of overall reward received which in turn affects the speed at which decision boundaries collapse, allowing faster responses as the overall reward received increases.

RESULTS

Here, we review previous theoretical and empirical results and we present new evidence for magnitude sensitivity arising as a function of single-trial dynamics.

CONCLUSIONS

The result of magnitude sensitive reaction times reported is not compatible with single-trial magnitude insensitive models, such as the statistically optimal drift diffusion model.

Cerebellar involvement in an evidence-accumulation decision-making task

To make successful evidence-based decisions, the brain must rapidly and accurately transform sensory inputs into specific goal-directed behaviors. Most experimental work on this subject has focused on forebrain mechanisms. Using a novel evidence-accumulation task for mice, we performed recording and perturbation studies of crus I of the lateral posterior cerebellum, which communicates bidirectionally with numerous forebrain regions. Cerebellar inactivation led to a reduction in the fraction of correct trials. Using two-photon fluorescence imaging of calcium, we found that Purkinje cell somatic activity contained choice/evidence-related information. Decision errors were represented by dendritic calcium spikes, which in other contexts are known to drive cerebellar plasticity. We propose that cerebellar circuitry may contribute to computations that support accurate performance in this perceptual decision-making task.

The State of the NIH BRAIN Initiative

The BRAIN Initiative arose from a grand challenge to "accelerate the development and application of new technologies that will enable researchers to produce dynamic pictures of the brain that show how individual brain cells and complex neural circuits interact at the speed of thought." The BRAIN Initiative is a public-private effort focused on the development and use of powerful tools for acquiring fundamental insights about how information processing occurs in the central nervous system (CNS). As the Initiative enters its fifth year, NIH has supported >500 principal investigators, who have answered the Initiative's challenge via hundreds of publications describing novel tools, methods, and discoveries that address the Initiative's seven scientific priorities. We describe scientific advances produced by individual laboratories, multi-investigator teams, and entire consortia that, over the coming decades, will produce more comprehensive and dynamic maps of the brain, deepen our understanding of how circuit activity can produce a rich tapestry of behaviors, and lay the foundation for understanding how its circuitry is disrupted in brain disorders. Much more work remains to bring this vision to fruition, and the National Institutes of Health continues to look to the diverse scientific community, from mathematics, to physics, chemistry, engineering, neuroethics, and neuroscience, to ensure that the greatest scientific benefit arises from this unique research Initiative.

An Accumulation-of-Evidence Task Using Visual Pulses for Mice Navigating in Virtual Reality

The gradual accumulation of sensory evidence is a crucial component of perceptual decision making, but its neural mechanisms are still poorly understood. Given the wide availability of genetic and optical tools for mice, they can be useful model organisms for the study of these phenomena; however, behavioral tools are largely lacking. Here, we describe a new evidence-accumulation task for head-fixed mice navigating in a virtual reality (VR) environment. As they navigate down the stem of a virtual T-maze, they see brief pulses of visual evidence on either side, and retrieve a reward on the arm with the highest number of pulses. The pulses occur randomly with Poisson statistics, yielding a diverse yet well-controlled stimulus set, making the data conducive to a variety of computational approaches. A large number of mice of different genotypes were able to learn and consistently perform the task, at levels similar to rats in analogous tasks. They are sensitive to side differences of a single pulse, and their memory of the cues is stable over time. Moreover, using non-parametric as well as modeling approaches, we show that the mice indeed accumulate evidence: they use multiple pulses of evidence from throughout the cue region of the maze to make their decision, albeit with a small overweighting of earlier cues, and their performance is affected by the magnitude but not the duration of evidence. Additionally, analysis of the mice's running patterns revealed that trajectories are fairly stereotyped yet modulated by the amount of sensory evidence, suggesting that the navigational component of this task may provide a continuous readout correlated to the underlying cognitive variables. Our task, which can be readily integrated with state-of-the-art techniques, is thus a valuable tool to study the circuit mechanisms and dynamics underlying perceptual decision making, particularly under more complex behavioral contexts.

Rat Prefrontal Cortex Inactivations during Decision Making Are Explained by Bistable Attractor Dynamics

Two-node attractor networks are flexible models for neural activity during decision making. Depending on the network configuration, these networks can model distinct aspects of decisions including evidence integration, evidence categorization, and decision memory. Here, we use attractor networks to model recent causal perturbations of the frontal orienting fields (FOF) in rat cortex during a perceptual decision-making task (Erlich, Brunton, Duan, Hanks, & Brody, 2015 ). We focus on a striking feature of the perturbation results. Pharmacological silencing of the FOF resulted in a stimulus-independent bias. We fit several models to test whether integration, categorization, or decision memory could account for this bias and found that only the memory configuration successfully accounts for it. This memory model naturally accounts for optogenetic perturbations of FOF in the same task and correctly predicts a memory-duration-dependent deficit caused by silencing FOF in a different task. Our results provide mechanistic support for a "postcategorization" memory role of the FOF in upcoming choices.

Functional dissection of signal and noise in MT and LIP during decision-making

During perceptual decision making, responses in the middle temporal (MT) and lateral intraparietal (LIP) areas appear to map onto theoretically defined quantities, with MT representing instantaneous motion evidence and LIP reflecting the accumulated evidence. However, several aspects of the transformation between the two areas have not been empirically tested. We therefore performed multi-stage systems identification analyses of the simultaneous activity of MT and LIP during individual decisions. We found that monkeys based their choices on evidence presented in early epochs of the motion stimulus, and that substantial early weighting of motion was present in MT responses. LIP’s responses recapitulated MT’s early weighting and contained a choice-dependent buildup that was distinguishable from motion integration. Furthermore, trial-by-trial variability in LIP did not depend on MT activity. These results identify important deviations from the idealizations of MT and LIP and motivate inquiry into sensorimotor computations that may intervene between MT and LIP.

Selective overweighting of larger magnitudes during noisy numerical comparison

Humans are often required to compare average magnitudes in numerical data; for example, when comparing product prices on two rival consumer websites. However, the neural and computational mechanisms by which numbers are weighted, integrated and compared during categorical decisions are largely unknown^1,2,3,4,5. Here, we show a systematic deviation from 'optimality' in both visual and auditory tasks requiring averaging of symbolic numbers. Participants comparing numbers drawn from two categories selectively overweighted larger numbers when making a decision, and larger numbers evoked disproportionately stronger decision-related neural signals over the parietal cortex. A representational similarity analysis⁶ showed that neural (dis)similarity in patterns of electroencephalogram activity reflected numerical distance, but that encoding of number in neural data was systematically distorted in a way predicted by the behavioural weighting profiles, with greater neural distance between adjacent larger numbers. Finally, using a simple computational model, we show that although it is suboptimal for a lossless observer, this selective overweighting policy paradoxically maximizes expected accuracy by making decisions more robust to noise arising during approximate numerical integration². In other words, although selective overweighting discards decision information, it can be beneficial for limited-capacity agents engaging in rapid numerical averaging.

Fronto-parietal Cortical Circuits Encode Accumulated Evidence with a Diversity of Timescales

Decision-making in dynamic environments often involves accumulation of evidence, in which new information is used to update beliefs and select future actions. Using in vivo cellular resolution imaging in voluntarily head-restrained rats, we examined the responses of neurons in frontal and parietal cortices during a pulse-based accumulation of evidence task. Neurons exhibited activity that predicted the animal's upcoming choice, previous choice, and graded responses that reflected the strength of the accumulated evidence. The pulsatile nature of the stimuli enabled characterization of the responses of neurons to a single quantum (pulse) of evidence. Across the population, individual neurons displayed extensive heterogeneity in the dynamics of responses to pulses. The diversity of responses was sufficiently rich to form a temporal basis for accumulated evidence estimated from a latent variable model. These results suggest that heterogeneous, often transient sensory responses distributed across the fronto-parietal cortex may support working memory on behavioral timescales. VIDEO ABSTRACT.

Segregated fronto-cortical and midbrain connections in the mouse and their relation to approach and avoidance orienting behaviors

The orchestration of orienting behaviors requires the interaction of many cortical and subcortical areas, for example the superior colliculus (SC), as well as prefrontal areas responsible for top–down control. Orienting involves different behaviors, such as approach and avoidance. In the rat, these behaviors are at least partially mapped onto different SC subdomains, the lateral (SCl) and medial (SCm), respectively. To delineate the circuitry involved in the two types of orienting behavior in mice, we injected retrograde tracer into the intermediate and deep layers of the SCm and SCl, and thereby determined the main input structures to these subdomains. Overall the SCm receives larger numbers of afferents compared to the SCl. The prefrontal cingulate area (Cg), visual, oculomotor, and auditory areas provide strong input to the SCm, while prefrontal motor area 2 (M2), and somatosensory areas provide strong input to the SCl. The prefrontal areas Cg and M2 in turn connect to different cortical and subcortical areas, as determined by anterograde tract tracing. Even though connectivity pattern often overlap, our labeling approaches identified segregated neural circuits involving SCm, Cg, secondary visual cortices, auditory areas, and the dysgranular retrospenial cortex likely to be involved in avoidance behaviors. Conversely, SCl, M2, somatosensory cortex, and the granular retrospenial cortex comprise a network likely involved in approach/appetitive behaviors.

Time to wake up: Studying neurovascular coupling and brain-wide circuit function in the un-anesthetized animal

Functional magnetic resonance imaging (fMRI) has allowed the noninvasive study of task-based and resting-state brain dynamics in humans by inferring neural activity from blood-oxygenation-level dependent (BOLD) signal changes. An accurate interpretation of the hemodynamic changes that underlie fMRI signals depends on the understanding of the quantitative relationship between changes in neural activity and changes in cerebral blood flow, oxygenation and volume. While there has been extensive study of neurovascular coupling in anesthetized animal models, anesthesia causes large disruptions of brain metabolism, neural responsiveness and cardiovascular function. Here, we review work showing that neurovascular coupling and brain circuit function in the awake animal are profoundly different from those in the anesthetized state. We argue that the time is right to study neurovascular coupling and brain circuit function in the awake animal to bridge the physiological mechanisms that underlie animal and human neuroimaging signals, and to interpret them in light of underlying neural mechanisms. Lastly, we discuss recent experimental innovations that have enabled the study of neurovascular coupling and brain-wide circuit function in un-anesthetized and behaving animal models.

Three challenges for connecting model to mechanism in decision-making

Recent years have seen a growing interest in understanding the neural mechanisms that support decision-making. The advent of new tools for measuring and manipulating neurons, alongside the inclusion of multiple new animal models and sensory systems has led to the generation of many novel datasets. The potential for these new approaches to constrain decision-making models is unprecedented. Here, we argue that to fully leverage these new approaches, three challenges must be met. First, experimenters must design well-controlled behavioral experiments that make it possible to distinguish competing behavioral strategies. Second, analyses of neural responses should think beyond single neurons, taking into account tradeoffs of single-trial versus trial-averaged approaches. Finally, quantitative model comparisons should be used, but must consider common obstacles.

Distinct roles of visual, parietal, and frontal motor cortices in memory-guided sensorimotor decisions

Mapping specific sensory features to future motor actions is a crucial capability of mammalian nervous systems. We investigated the role of visual (V1), posterior parietal (PPC), and frontal motor (fMC) cortices for sensorimotor mapping in mice during performance of a memory-guided visual discrimination task. Large-scale calcium imaging revealed that V1, PPC, and fMC neurons exhibited heterogeneous responses spanning all task epochs (stimulus, delay, response). Population analyses demonstrated unique encoding of stimulus identity and behavioral choice information across regions, with V1 encoding stimulus, fMC encoding choice even early in the trial, and PPC multiplexing the two variables. Optogenetic inhibition during behavior revealed that all regions were necessary during the stimulus epoch, but only fMC was required during the delay and response epochs. Stimulus identity can thus be rapidly transformed into behavioral choice, requiring V1, PPC, and fMC during the transformation period, but only fMC for maintaining the choice in memory prior to execution.

DOI:http://dx.doi.org/10.7554/eLife.13764.001