Discriminating Among Probability Weighting Functions Using Adaptive Design Optimization

Reliable population code for subjective economic value from heterogeneous neuronal signals in primate orbitofrontal cortex

Behavior-related neuronal signals often vary between neurons, which might reflect the unreliability of individual neurons or a truly heterogeneous code. This notion may also apply to economic ("value-based") choices and the underlying reward signals. Reward value is subjective and can be described by a nonlinearly weighted magnitude (utility) and probability. Defining subjective values relies on the continuity axiom, whose testing involves structured variations of a wide range of reward magnitudes and probabilities. Axiom compliance demonstrates understanding of the stimuli and the meaningful character of choices. Using these tests, we investigated the encoding of subjective economic value by neurons in a key economic-decision structure of the monkey brain, the orbitofrontal cortex (OFC). We found that individual neurons carry heterogeneous neuronal value signals that largely fail to match the animal's choices. However, neuronal population signals matched the animal's choices well, suggesting accurate subjective economic value encoding by a heterogeneous population of unreliable neurons.

Adaptive Design Optimization as a Promising Tool for Reliable and Efficient Computational Fingerprinting

A key challenge in understanding mental (dys)functions is their etiological and functional heterogeneity, and several multidimensional assessments have been proposed for their comprehensive characterization. However, such assessments require lengthy testing, which may hinder reliable and efficient characterization of individual differences due to increased fatigue and distraction, especially in clinical populations. Computational modeling may address this challenge as it often provides more reliable measures of latent neurocognitive processes underlying observed behaviors and captures individual differences better than traditional assessments. However, even with a state-of-the-art hierarchical modeling approach, reliable estimation of model parameters still requires a large number of trials. Recent work suggests that Bayesian adaptive design optimization (ADO) is a promising way to address these challenges. With ADO, experimental design is optimized adaptively from trial to trial to extract the maximum amount of information about an individual's characteristics. In this review, we first describe the ADO methodology and then summarize recent work demonstrating that ADO increases the reliability and efficiency of latent neurocognitive measures. We conclude by discussing the challenges and future directions of ADO and proposing development of ADO-based computational fingerprints to reliably and efficiently characterize the heterogeneous profiles of psychiatric disorders.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

Attribute attention and option attention in risky choice

Probability weighting is one of the most powerful theoretical constructs in descriptive models of risky choice and constitutes a central component of cumulative prospect theory (CPT). Probability weighting has been shown to be related to two facets of attention allocation: one analysis showed that differences in the shape of CPT's probability-weighting function are linked to differences in how attention is allocated across attributes (i.e., probabilities vs. outcomes); another analysis (that used a different measure of attention) showed a link between probability weighting and differences in how attention is allocated across options. However, the relationship between these two links is unclear. We investigate to what extent attribute attention and option attention independently contribute to probability weighting. Reanalyzing data from a process-tracing study, we first demonstrate links between probability weighting and both attribute attention and option attention within the same data set and the same measure of attention. We then find that attribute attention and option attention are at best weakly related and have independent and distinct effects on probability weighting. Moreover, deviations from linear weighting mainly emerged when attribute attention or option attention were imbalanced. Our analyses enrich the understanding of the cognitive underpinnings of preferences and illustrate that similar probability-weighting patterns can be associated with very different attentional policies. This complicates an unambiguous psychological interpretation of psycho-economic functions. Our findings indicate that cognitive process models of decision making should aim to concurrently account for the effects of different facets of attention allocation on preference. In addition, we argue that the origins of biases in attribute attention and option attention need to be better understood.

The Legacy of Peter Fishburn: Foundational Work and Lasting Impact

Peter Fishburn has had a tremendous impact on the field of decision analysis, developing ideas that would come to be foundational across decision analysis and that would impact the literature on decision making in economics, psychology, finance, engineering, and mathematics. This paper provides an overview of his legacy. We summarize 11 of his influential papers. We then trace his impact on recent research in topics including preference representation and elicitation, risk attitudes, time preferences, health preferences, behavioral decision making, social choice and voting, and geometric analyses.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/deca.2022.0461 .

Dorsolateral prefrontal cortex plays causal role in probability weighting during risky choice

In this study, we provide causal evidence that the dorsolateral prefrontal cortex (DLPFC) supports the computation of subjective value in choices under risk via its involvement in probability weighting. Following offline continuous theta-burst transcranial magnetic stimulation (cTBS) of the DLPFC subjects (N = 30, mean age 23.6, 56% females) completed a computerized task consisting of 96 binary lottery choice questions presented in random order. Using the hierarchical Bayesian modeling approach, we then estimated the structural parameters of risk preferences (the degree of risk aversion and the curvature of the probability weighting function) and analyzed the obtained posterior distributions to determine the effect of stimulation on model parameters. On a behavioral level, temporary downregulation of the left DLPFC excitability through cTBS decreased the likelihood of choosing an option with higher expected reward while the probability of choosing a riskier lottery did not significantly change. Modeling the stimulation effects on risk preference parameters showed anecdotal evidence as assessed by Bayes factors that probability weighting parameter increased after the left DLPFC TMS compared to sham.

Risky choice: Probability weighting explains independence axiom violations in monkeys

Expected Utility Theory (EUT) provides axioms for maximizing utility in risky choice. The Independence Axiom (IA) is its most demanding axiom: preferences between two options should not change when altering both options equally by mixing them with a common gamble. We tested common consequence (CC) and common ratio (CR) violations of the IA over several months in thousands of stochastic choices using a large variety of binary option sets. Three monkeys showed consistently few outright Preference Reversals (8%) but substantial graded Preference Changes (46%) between the initial preferred gamble and the corresponding altered gamble. Linear Discriminant Analysis (LDA) indicated that gamble probabilities predicted most Preference Changes in CC (72%) and CR (88%) tests. The Akaike Information Criterion indicated that probability weighting within Cumulative Prospect Theory (CPT) explained choices better than models using Expected Value (EV) or EUT. Fitting by utility and probability weighting functions of CPT resulted in nonlinear and non-parallel indifference curves (IC) in the Marschak-Machina triangle and suggested IA non-compliance of models using EV or EUT. Indeed, CPT models predicted Preference Changes better than EV and EUT models. Indifference points in out-of-sample tests were closer to CPT-estimated ICs than EV and EUT ICs. Finally, while the few outright Preference Reversals may reflect the long experience of our monkeys, their more graded Preference Changes corresponded to those reported for humans. In benefitting from the wide testing possibilities in monkeys, our stringent axiomatic tests contribute critical information about risky decision-making and serves as basis for investigating neuronal decision mechanisms.

Supplementary information

The online version contains supplementary material available at 10.1007/s11166-022-09388-7.

Gaze-dependent evidence accumulation predicts multi-alternative risky choice behaviour

Choices are influenced by gaze allocation during deliberation, so that fixating an alternative longer leads to increased probability of choosing it. Gaze-dependent evidence accumulation provides a parsimonious account of choices, response times and gaze-behaviour in many simple decision scenarios. Here, we test whether this framework can also predict more complex context-dependent patterns of choice in a three-alternative risky choice task, where choices and eye movements were subject to attraction and compromise effects. Choices were best described by a gaze-dependent evidence accumulation model, where subjective values of alternatives are discounted while not fixated. Finally, we performed a systematic search over a large model space, allowing us to evaluate the relative contribution of different forms of gaze-dependence and additional mechanisms previously not considered by gaze-dependent accumulation models. Gaze-dependence remained the most important mechanism, but participants with strong attraction effects employed an additional similarity-dependent inhibition mechanism found in other models of multi-alternative multi-attribute choice.

Faced with different choice alternatives, such as food options or risky prospects, our decisions and allocation of gaze (that is where we look) are closely linked, such that items that are looked at longer are often more likely to be chosen. In simple decisions (e.g., choosing between two chocolate bars), these decisions and their associations with gaze allocation are well described by computational models that assume accumulation of evidence in favour of each alternative over time and discounting of momentarily unattended information. However, an important question is whether this class of models can also describe choice behaviour in more complex settings. Specifically, so-called context effects, where preferences between two alternatives can vary with the addition of a third alternative, challenge many models of simple decision making. Our study addresses this question by evaluating gaze-dependent evidence accumulation models in a setting where choices between two risky lotteries are systematically influenced by a third alternative. We find gaze-dependent models to be able to describe context effects because decision-makers‘ gaze allocation also varies with different sets of alternatives.

Adaptive Design Optimization for a Mnemonic Similarity Task

The Mnemonic Similarity Task (MST: Stark et al., 2019) is a modified recognition memory task designed to place strong demand on pattern separation. The sensitivity and reliability of the MST make it an extremely valuable tool in clinical settings, where it has been used to identify hippocampal dysfunction associated with healthy aging, dementia, schizophrenia, depression, and other disorders. As with any test used in a clinical setting, it is especially important for the MST to be administered as efficiently as possible. We apply adaptive design optimization methods (Lesmes et al., 2015; Myung et al., 2013) to optimize the presentation of test stimuli in accordance with previous responses. This optimization is based on a signal-detection model of an individual's memory capabilities and decision-making processes. We demonstrate that the adaptive design optimization approach generally reduces the number of test stimuli needed to provide these measures.

Modeling Ethical and Operational Preferences in Automated Driving Systems

Whereas automated driving technology has made tremendous gains in the last decade, significant questions remain regarding its integration into society. Given its revolutionary nature, the use of automated driving systems (ADSs) is accompanied by myriad novel quandaries relating to both operational and ethical concerns that are relevant to numerous stakeholders (e.g., governments, manufacturers, and passengers). When considering any such problem, the ADS’s decision-making calculus is always a central component. This is true for concerns about public perception and trust to others regarding explainability and legal certainty. Therefore, in this manuscript, we set forth a general decision-analytic framework tailorable to multitudinous stakeholders. More specifically, we develop and validate a generic tree of ADS management objectives, explore potential attributes for their measurement, and provide multiattribute utility functions for implementation. Given the contention surrounding numerous ethical concerns in ADS operations, we explore how each of the aforementioned components can be tailored in accordance with the stakeholder’s desired ethical perspective. A simulation environment is developed upon which our framework is tested. Within this environment we illustrate how our approach can be leveraged by stakeholders to make strategic trade-offs regarding ADS behavior and to inform policymaking efforts. In so doing, our framework is demonstrated as a practical, tractable, and transparent means of modeling ADS decision making.

Personalized brain stimulation for effective neurointervention across participants

Accumulating evidence from human-based research has highlighted that the prevalent one-size-fits-all approach for neural and behavioral interventions is inefficient. This approach can benefit one individual, but be ineffective or even detrimental for another. Studying the efficacy of the large range of different parameters for different individuals is costly, time-consuming and requires a large sample size that makes such research impractical and hinders effective interventions. Here an active machine learning technique is presented across participants-personalized Bayesian optimization (pBO)-that searches available parameter combinations to optimize an intervention as a function of an individual's ability. This novel technique was utilized to identify transcranial alternating current stimulation (tACS) frequency and current strength combinations most likely to improve arithmetic performance, based on a subject's baseline arithmetic abilities. The pBO was performed across all subjects tested, building a model of subject performance, capable of recommending parameters for future subjects based on their baseline arithmetic ability. pBO successfully searches, learns, and recommends parameters for an effective neurointervention as supported by behavioral, simulation, and neural data. The application of pBO in human-based research opens up new avenues for personalized and more effective interventions, as well as discoveries of protocols for treatment and translation to other clinical and non-clinical domains.

Data-driven experimental design and model development using Gaussian process with active learning

Interest in computational modeling of cognition and behavior continues to grow. To be most productive, modelers should be equipped with tools that ensure optimal efficiency in data collection and in the integrity of inference about the phenomenon of interest. Traditionally, models in cognitive science have been parametric, which are particularly susceptible to model misspecification because their strong assumptions (e.g. parameterization, functional form) may introduce unjustified biases in data collection and inference. To address this issue, we propose a data-driven nonparametric framework for model development, one that also includes optimal experimental design as a goal. It combines Gaussian Processes, a stochastic process often used for regression and classification, with active learning, from machine learning, to iteratively fit the model and use it to optimize the design selection throughout the experiment. The approach, dubbed Gaussian process with active learning (GPAL), is an extension of the parametric, adaptive design optimization (ADO) framework (Cavagnaro, Myung, Pitt, & Kujala, 2010). We demonstrate the application and features of GPAL in a delay discounting task and compare its performance to ADO in two experiments. The results show that GPAL is a viable modeling framework that is noteworthy for its high sensitivity to individual differences, identifying novel patterns in the data that were missed by the model-constrained ADO. This investigation represents a first step towards the development of a data-driven cognitive modeling framework that serves as a middle ground between raw data, which can be difficult to interpret, and parametric models, which rely on strong assumptions.

ADOpy: a python package for adaptive design optimization

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Rapid, precise, and reliable measurement of delay discounting using a Bayesian learning algorithm

Machine learning has the potential to facilitate the development of computational methods that improve the measurement of cognitive and mental functioning. In three populations (college students, patients with a substance use disorder, and Amazon Mechanical Turk workers), we evaluated one such method, Bayesian adaptive design optimization (ADO), in the area of delay discounting by comparing its test-retest reliability, precision, and efficiency with that of a conventional staircase method. In all three populations tested, the results showed that ADO led to 0.95 or higher test-retest reliability of the discounting rate within 10-20 trials (under 1-2 min of testing), captured approximately 10% more variance in test-retest reliability, was 3-5 times more precise, and was 3-8 times more efficient than the staircase method. The ADO methodology provides efficient and precise protocols for measuring individual differences in delay discounting.

Probability range and probability distortion in a gambling task

In decision making under risk, adults tend to overestimate small and underestimate large probabilities (Tversky & Kahneman, 1992). This inverse S-shaped distortion pattern is similar to that observed in a wide variety of proportion judgment tasks (see Hollands & Dyre, 2000, for review). In proportion judgment tasks, distortion patterns tend not to be fixed but rather to depend on the reference points to which the targets are compared. Here, we tested the novel hypothesis that probability distortion in decision making under risk might also be influenced by reference points-in this case, references implied by the probability range. Adult participants were assigned to either a full-range (probabilities from 0-100%), upper-range (50-100%), or lower-range (0-50%) condition, where they indicated certainty equivalents for 176 hypothetical monetary gambles (e.g., "a 50% chance of $100, otherwise $0"). Using a modified cumulative prospect theory model, we found only minimal differences in probability distortion as a function of condition, suggesting no differences in use of reference points by condition, and broadly demonstrating the robustness of distortion pattern across contexts. However, we also observed deviations from the curve across all conditions that warrant further research.

A model-based analysis of decision making under risk in obsessive-compulsive and hoarding disorders

Attitudes towards risk are highly consequential in clinical disorders thought to be prone to "risky behavior", such as substance dependence, as well as those commonly associated with excessive risk aversion, such as obsessive-compulsive disorder (OCD) and hoarding disorder (HD). Moreover, it has recently been suggested that attitudes towards risk may serve as a behavioral biomarker for OCD. We investigated the risk preferences of participants with OCD and HD using a novel adaptive task and a quantitative model from behavioral economics that decomposes risk preferences into outcome sensitivity and probability sensitivity. Contrary to expectation, compared to healthy controls, participants with OCD and HD exhibited less outcome sensitivity, implying less risk aversion in the standard economic framework. In addition, risk attitudes were strongly correlated with depression, hoarding, and compulsion scores, while compulsion (hoarding) scores were associated with more (less) "rational" risk preferences. These results demonstrate how fundamental attitudes towards risk relate to specific psychopathology and thereby contribute to our understanding of the cognitive manifestations of mental disorders. In addition, our findings indicate that the conclusion made in recent work that decision making under risk is unaltered in OCD is premature.

On the Functional Form of Temporal Discounting: An Optimized Adaptive Test

The tendency to discount the value of future rewards has become one of the best-studied constructs in the behavioral sciences. Although hyperbolic discounting remains the dominant quantitative characterization of this phenomenon, a variety of models have been proposed and consensus around the one that most accurately describes behavior has been elusive. To help bring some clarity to this issue, we propose an Adaptive Design Optimization (ADO) method for fitting and comparing models of temporal discounting. We then conduct an ADO experiment aimed at discriminating among six popular models of temporal discounting. Rather than supporting a single underlying model, our results show that each model is inadequate in some way to describe the full range of behavior exhibited across subjects. The precision of results provided by ADO further identify specific properties of models, such as accommodating both increasing and decreasing impatience, that are mandatory to describe temporal discounting broadly.

Probabilistic Inference: Task Dependency and Individual Differences of Probability Weighting Revealed by Hierarchical Bayesian Modeling

Cognitive determinants of probabilistic inference were examined using hierarchical Bayesian modeling techniques. A classic urn-ball paradigm served as experimental strategy, involving a factorial two (prior probabilities) by two (likelihoods) design. Five computational models of cognitive processes were compared with the observed behavior. Parameter-free Bayesian posterior probabilities and parameter-free base rate neglect provided inadequate models of probabilistic inference. The introduction of distorted subjective probabilities yielded more robust and generalizable results. A general class of (inverted) S-shaped probability weighting functions had been proposed; however, the possibility of large differences in probability distortions not only across experimental conditions, but also across individuals, seems critical for the model's success. It also seems advantageous to consider individual differences in parameters of probability weighting as being sampled from weakly informative prior distributions of individual parameter values. Thus, the results from hierarchical Bayesian modeling converge with previous results in revealing that probability weighting parameters show considerable task dependency and individual differences. Methodologically, this work exemplifies the usefulness of hierarchical Bayesian modeling techniques for cognitive psychology. Theoretically, human probabilistic inference might be best described as the application of individualized strategic policies for Bayesian belief revision.

A computational analysis of the neural bases of Bayesian inference

Empirical support for the Bayesian brain hypothesis, although of major theoretical importance for cognitive neuroscience, is surprisingly scarce. This hypothesis posits simply that neural activities code and compute Bayesian probabilities. Here, we introduce an urn-ball paradigm to relate event-related potentials (ERPs) such as the P300 wave to Bayesian inference. Bayesian model comparison is conducted to compare various models in terms of their ability to explain trial-by-trial variation in ERP responses at different points in time and over different regions of the scalp. Specifically, we are interested in dissociating specific ERP responses in terms of Bayesian updating and predictive surprise. Bayesian updating refers to changes in probability distributions given new observations, while predictive surprise equals the surprise about observations under current probability distributions. Components of the late positive complex (P3a, P3b, Slow Wave) provide dissociable measures of Bayesian updating and predictive surprise. Specifically, the updating of beliefs about hidden states yields the best fit for the anteriorly distributed P3a, whereas the updating of predictions of observations accounts best for the posteriorly distributed Slow Wave. In addition, parietally distributed P3b responses are best fit by predictive surprise. These results indicate that the three components of the late positive complex reflect distinct neural computations. As such they are consistent with the Bayesian brain hypothesis, but these neural computations seem to be subject to nonlinear probability weighting. We integrate these findings with the free-energy principle that instantiates the Bayesian brain hypothesis.

A hierarchical adaptive approach to optimal experimental design

Experimentation is at the core of research in the behavioral and neural sciences, yet observations can be expensive and time-consuming to acquire (e.g., MRI scans, responses from infant participants). A major interest of researchers is designing experiments that lead to maximal accumulation of information about the phenomenon under study with the fewest possible number of observations. In addressing this challenge, statisticians have developed adaptive design optimization methods. This letter introduces a hierarchical Bayes extension of adaptive design optimization that provides a judicious way to exploit two complementary schemes of inference (with past and future data) to achieve even greater accuracy and efficiency in information gain. We demonstrate the method in a simulation experiment in the field of visual perception.

A Tutorial on Adaptive Design Optimization

Experimentation is ubiquitous in the field of psychology and fundamental to the advancement of its science, and one of the biggest challenges for researchers is designing experiments that can conclusively discriminate the theoretical hypotheses or models under investigation. The recognition of this challenge has led to the development of sophisticated statistical methods that aid in the design of experiments and that are within the reach of everyday experimental scientists. This tutorial paper introduces the reader to an implementable experimentation methodology, dubbed Adaptive Design Optimization, that can help scientists to conduct "smart" experiments that are maximally informative and highly efficient, which in turn should accelerate scientific discovery in psychology and beyond.