Orbitofrontal signals for two-component choice options comply with indifference curves of Revealed Preference Theory

Dynamic coding and sequential integration of multiple reward attributes by primate amygdala neurons

The value of visual stimuli guides learning, decision-making, and motivation. Although stimulus values often depend on multiple attributes, how neurons extract and integrate distinct value components from separate cues remains unclear. Here we recorded the activity of amygdala neurons while two male monkeys viewed sequential cues indicating the probability and magnitude of expected rewards. Amygdala neurons frequently signaled reward probability in an abstract, stimulus-independent code that generalized across cue formats. While some probability-coding neurons were insensitive to magnitude information, signaling 'pure' probability rather than value, many neurons showed biphasic responses that signaled probability and magnitude in a dynamic (temporally-patterned) and flexible (reversible) value code. Specific amygdala neurons integrated these reward attributes into risk signals that quantified the variance of expected rewards, distinct from value. Population codes were accurate, mutually transferable between value components, and expressed differently across amygdala nuclei. Our findings identify amygdala neurons as a substrate for the sequential integration of multiple reward attributes into value and risk.

Prefrontal default-mode network interactions with posterior hippocampus during exploration

Hippocampal maps and ventral prefrontal cortex (vPFC) value and goal representations support foraging in continuous spaces. How might hippocampal-vPFC interactions control the balance between behavioral exploration and exploitation? Using fMRI and reinforcement learning modeling, we investigated vPFC and hippocampal responses as humans explored and exploited a continuous one-dimensional space, with out-of-session and out-of-sample replication. The spatial distribution of rewards, or value landscape, modulated activity in the hippocampus and default network vPFC subregions, but not in ventrolateral prefrontal control subregions or medial orbitofrontal limbic subregions. While prefrontal default network and hippocampus displayed higher activity in less complex, easy-to-exploit value landscapes, vPFC-hippocampal connectivity increased in uncertain landscapes requiring exploration. Further, synchronization between prefrontal default network and posterior hippocampus scaled with behavioral exploration. Considered alongside electrophysiological studies, our findings suggest that locations to be explored are identified through coordinated activity binding prefrontal default network value representations to posterior hippocampal maps.

Laminar Architecture of a Decision Circuit in Orbitofrontal Cortex

During economic choice, different neurons in orbitofrontal cortex (OFC) encode individual offer values, the binary choice outcome, and the chosen value. Previous work suggests that these cell groups form a decision circuit, but the anatomical organization of this circuit is poorly understood. Using calcium imaging, we recorded from layer 2/3 (L2/3) and layer 5 (L5) of mice choosing between juice flavors. Decision variables were differentially represented across layers: juice-specific offer values and their spatial configuration were predominant in L2/3, while spatial offer values, chosen side, and chosen value were predominant in L5. Within each layer, functional cell groups were organized in clusters. The temporal dynamics of neural signals in the two layers indicated a combination of feed-forward and feed-back processes, and pointed to L5 as the locus for winner-take-all value comparison. These results reveal that economic decisions rely on a complex architecture distributed across layers of OFC.

Attention-dependent attribute comparisons underlie multi-attribute decision-making in orbitofrontal cortex

Economic decisions often require weighing multiple dimensions, or attributes. The orbitofrontal cortex FC) is thought to be important for computing the integrated value of an option from its attributes and comparing lues to make a choice. Although OFC neurons are known to encode integrated values, evidence for value mparison has been limited. Here, we used a multi-attribute choice task for monkeys to investigate how OFC eurons integrate and compare multi-attribute options. Attributes were represented separately and eye tracking as used to measure attention. We found that OFC neurons encode the value of attended attributes, dependent of other attributes in the same option. Encoding was negatively weighted by the value of the same tribute in the other option, consistent with a comparison between the two like attributes. These results indicate at OFC computes comparisons among attributes rather than integrated values, and does so dynamically, ifting with the focus of attention.

The representation of decision variables in orbitofrontal cortex is longitudinally stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the choice input and the choice output, suggesting that the cell groups found in the OFC constitute the building blocks of a decision circuit. Here, we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we record from the OFC of mice engaged in a juice-choice task. Imaging of individual cells continues for up to 40 weeks. For each cell and each session pair, we compare activity profiles using cosine similarity, and we assess whether the neuron encodes the same variable in both sessions. We find a high degree of stability and a modest representational drift. Quantitative estimates indicate that this drift would not randomize the circuit within the animal's lifetime.

Dopamine neurons encode trial-by-trial subjective reward value in an auction-like task

The dopamine reward prediction error signal is known to be subjective but has so far only been assessed in aggregate choices. However, personal choices fluctuate across trials and thus reflect the instantaneous subjective reward value. In the well-established Becker-DeGroot-Marschak (BDM) auction-like mechanism, participants are encouraged to place bids that accurately reveal their instantaneous subjective reward value; inaccurate bidding results in suboptimal reward ("incentive compatibility"). In our experiment, male rhesus monkeys became experienced over several years to place accurate BDM bids for juice rewards without specific external constraints. Their bids for physically identical rewards varied trial by trial and increased overall for larger rewards. In these highly experienced animals, responses of midbrain dopamine neurons followed the trial-by-trial variations of bids despite constant, explicitly predicted reward amounts. Inversely, dopamine responses were similar with similar bids for different physical reward amounts. Support Vector Regression demonstrated accurate prediction of the animals' bids by as few as twenty dopamine neurons. Thus, the phasic dopamine reward signal reflects instantaneous subjective reward value.

Multiattribute Decision-making in Macaques Relies on Direct Attribute Comparisons

In value-based decisions, there are frequently multiple attributes, such as cost, quality, or quantity, that contribute to the overall goodness of an option. Because one option may not be better in all attributes at once, the decision process should include a means of weighing relevant attributes. Most decision-making models solve this problem by computing an integrated value, or utility, for each option from a weighted combination of attributes. However, behavioral anomalies in decision-making, such as context effects, indicate that other attribute-specific computations might be taking place. Here, we tested whether rhesus macaques show evidence of attribute-specific processing in a value-based decision-making task. Monkeys made a series of decisions involving choice options comprising a sweetness and probability attribute. Each attribute was represented by a separate bar with one of two mappings between bar size and the magnitude of the attribute (i.e., bigger = better or bigger = worse). We found that translating across different mappings produced selective impairments in decision-making. Choices were less accurate and preferences were more variable when like attributes differed in mapping, suggesting that preventing monkeys from easily making direct attribute comparisons resulted in less accurate choice behavior. This was not the case when mappings of unalike attributes within the same option were different. Likewise, gaze patterns favored transitions between like attributes over transitions between unalike attributes of the same option, so that like attributes were sampled sequentially to support within-attribute comparisons. Together, these data demonstrate that value-based decisions rely, at least in part, on directly comparing like attributes of multiattribute options.

Neuronal Activity in the Gustatory Cortex during Economic Choice

An economic choice entails computing and comparing the values of individual offers. Offer values are represented in the orbitofrontal cortex (OFC)-an area that participates in value comparison-but it is unknown where offer values are computed in the first place. One possibility is that this computation takes place in OFC. Alternatively, offer values might be computed upstream of OFC. For choices between edible goods, a primary candidate is the gustatory region of the anterior insula (gustatory cortex, GC). Here we recorded from the GC of male rhesus monkeys choosing between different juice types. As a population, neurons in GC represented the flavor, the quantity, and the subjective value of the juice chosen by the animal. These variables were represented by distinct groups of cells and with different time courses. Specifically, chosen value signals emerged shortly after offer presentation, while neurons encoding the chosen juice and the chosen quantity peaked after juice delivery. Surprisingly, neurons in GC did not represent individual offer values in a systematic way. In a computational sense, the variables encoded in GC follow the process of value comparison. Thus our results argue against the hypothesis that offer values are computed in GC. At the same time, signals representing the subjective value of the expected reward indicate that responses in GC are not purely sensory. Thus neuronal responses in GC appear consummatory in nature.

Preferences reveal dissociable encoding across prefrontal-limbic circuits

Individual preferences for the flavor of different foods and fluids exert a strong influence on behavior. Most current theories posit that preferences are integrated with other state variables in the orbitofrontal cortex (OFC), which is thought to derive the relative subjective value of available options to guide choice behavior. Here, we report that instead of a single integrated valuation system in the OFC, another complementary one is centered in the ventrolateral prefrontal cortex (vlPFC) in macaques. Specifically, we found that the OFC and vlPFC preferentially represent outcome flavor and outcome probability, respectively, and that preferences are separately integrated into value representations in these areas. In addition, the vlPFC, but not the OFC, represented the probability of receiving the available outcome flavors separately, with the difference between these representations reflecting the degree of preference for each flavor. Thus, both the vlPFC and OFC exhibit dissociable but complementary representations of subjective value, both of which are necessary for decision-making.

Dorsal raphe neurons integrate the values of reward amount, delay, and uncertainty in multi-attribute decision-making

The dorsal raphe nucleus (DRN) is implicated in psychiatric disorders that feature impaired sensitivity to reward amount, impulsivity when facing reward delays, and risk-seeking when confronting reward uncertainty. However, it has been unclear whether and how DRN neurons signal reward amount, reward delay, and reward uncertainty during multi-attribute value-based decision-making, where subjects consider these attributes to make a choice. We recorded DRN neurons as monkeys chose between offers whose attributes, namely expected reward amount, reward delay, and reward uncertainty, varied independently. Many DRN neurons signaled offer attributes, and this population tended to integrate the attributes in a manner that reflected monkeys' preferences for amount, delay, and uncertainty. After decision-making, in response to post-decision feedback, these same neurons signaled signed reward prediction errors, suggesting a broader role in tracking value across task epochs and behavioral contexts. Our data illustrate how the DRN participates in value computations, guiding theories about the role of the DRN in decision-making and psychiatric disease.

Dopamine neurons encode trial-by-trial subjective reward value in an auction-like task

The dopamine reward prediction error signal is known to be subjective but has so far only been assessed in aggregate choices. However, personal choices fluctuate across trials and thus reflect the instantaneous subjective reward value. In the well-established Becker-DeGroot-Marschak (BDM) auction-like mechanism, participants are encouraged to place bids that accurately reveal their instantaneous subjective reward value; inaccurate bidding results in suboptimal reward ('incentive compatibility'). In our experiment, male rhesus monkeys became experienced over several years to place accurate BDM bids for juice rewards without specific external constraints. Their bids for physically identical rewards varied trial by trial and increased overall for larger rewards. In these highly experienced animals, responses of midbrain dopamine neurons followed the trial-by-trial variations of bids despite constant, explicitly predicted reward amounts. Inversely, dopamine responses were similar with similar bids for different physical reward amounts. Support Vector Regression demonstrated accurate prediction of the animals' bids by as few as twenty dopamine neurons. Thus, the phasic dopamine reward signal reflects instantaneous subjective reward value.

Dissociable representations of decision variables within subdivisions of macaque orbitofrontal and ventrolateral frontal cortex

Ventral frontal cortex (VFC) in macaques is involved in many affective and cognitive processes and has a key role in flexibly guiding reward-based decision-making. VFC is composed of a set of anatomically distinct subdivisions that are within the orbitofrontal cortex, ventrolateral prefrontal cortex, and anterior insula. In part, because prior studies have lacked the resolution to test for differences, it is unclear if neural representations related to decision-making are dissociable across these subdivisions. Here we recorded the activity of thousands of neurons within eight anatomically defined subregions of VFC in macaque monkeys performing a two-choice probabilistic task for different fruit juices outcomes. We found substantial variation in the encoding of decision variables across these eight subdivisions. Notably, ventrolateral subdivision 12l was unique relative to the other areas that we recorded from as the activity of single neurons integrated multiple attributes when monkeys evaluated the different choice options. Activity within 12o, by contrast, more closely represented reward probability and whether reward was received on a given trial. Orbitofrontal area 11m/l contained more specific representations of the quality of the outcome that could be earned later on. We also found that reward delivery encoding was highly distributed across all VFC subregions, while the properties of the reward, such as its flavor, were more strongly represented in areas 11m/l and 13m. Taken together, our work reveals the diversity of encoding within the various anatomically distinct subdivisions of VFC in primates.

The Representation of Decision Variables in Orbitofrontal Cortex is Longitudinally Stable

The computation and comparison of subjective values underlying economic choices rely on the orbitofrontal cortex (OFC). In this area, distinct groups of neurons encode the value of individual options, the binary choice outcome, and the chosen value. These variables capture both the input and the output of the choice process, suggesting that the cell groups found in OFC constitute the building blocks of a decision circuit. Here we show that this neural circuit is longitudinally stable. Using two-photon calcium imaging, we recorded from mice choosing between different juice flavors. Recordings of individual cells continued for up to 20 weeks. For each cell and each pair of sessions, we compared the activity profiles using cosine similarity, and we assessed whether the cell encoded the same variable in both sessions. These analyses revealed a high degree of stability and a modest representational drift. A quantitative estimate indicated this drift would not randomize the circuit within the animal's lifetime.

Multi-attribute decision-making in macaques relies on direct attribute comparisons

In value-based decisions, there are frequently multiple attributes, such as cost, quality, or quantity, that contribute to the overall goodness of an option. Since one option may not be better in all attributes at once, the decision process should include a means of weighing relevant attributes. Most decision-making models solve this problem by computing an integrated value, or utility, for each option from a weighted combination of attributes. However, behavioral anomalies in decision-making, such as context effects, indicate that other attribute-specific computations might be taking place. Here, we tested whether rhesus macaques show evidence of attribute-specific processing in a value-based decision-making task. Monkeys made a series of decisions involving choice options comprising a sweetness and probability attribute. Each attribute was represented by a separate bar with one of two mappings between bar size and the magnitude of the attribute (i.e., bigger=better or bigger=worse). We found that translating across different mappings produced selective impairments in decision-making. When like attributes differed, monkeys were prevented from easily making direct attribute comparisons, and choices were less accurate and preferences were more variable. This was not the case when mappings of unalike attributes within the same option were different. Likewise, gaze patterns favored transitions between like attributes over transitions between unalike attributes of the same option, so that like attributes were sampled sequentially to support within-attribute comparisons. Together, these data demonstrate that value-based decisions rely, at least in part, on directly comparing like attributes of multi-attribute options.

Significance Statement

Value-based decision-making is a cognitive function impacted by a number of clinical conditions, including substance use disorder and mood disorders. Understanding the neural mechanisms, including online processing steps involved in decision formation, will provide critical insights into decision-making deficits characteristic of human psychiatric disorders. Using rhesus monkeys as a model species capable of complex decision-making, this study shows that decisions involve a process of comparing like features, or attributes, of multi-attribute options. This is contrary to popular models of decision-making in which attributes are first combined into an overall value, or utility, to make a choice. Therefore, these results serve as an important foundation for establishing a more complete understanding of the neural mechanisms involved in forming complex decisions.

Studying neural responses for multi-component economic choices in human and non-human primates using concept-based behavioral choice experiments

Realistic, everyday rewards contain multiple components, such as taste and size. However, our reward valuations and the associated neural reward signals are single dimensional (vector to scalar transformation). Here, we present a protocol to identify these single-dimensional neural responses for multi-component choice options in humans and monkeys using concept-based behavioral choice experiments. We describe the use of stringent economic concepts to develop and implement behavioral tasks. We detail regional neuroimaging in humans and fine-grained neurophysiology in monkeys and describe approaches for data analysis. For complete details on the use and execution of this protocol, please refer to our work on humans Seak et al.1 and Pastor-Bernier et al.2 and monkeys Pastor-Bernier et al. 3, Pastor-Bernier et al.4, and Pastor-Bernier et al.5.

Dynamic prospect theory: Two core decision theories coexist in the gambling behavior of monkeys and humans

Research in the multidisciplinary field of neuroeconomics has mainly been driven by two influential theories regarding human economic choice: prospect theory, which describes decision-making under risk, and reinforcement learning theory, which describes learning for decision-making. We hypothesized that these two distinct theories guide decision-making in a comprehensive manner. Here, we propose and test a decision-making theory under uncertainty that combines these highly influential theories. Collecting many gambling decisions from laboratory monkeys allowed for reliable testing of our model and revealed a systematic violation of prospect theory's assumption that probability weighting is static. Using the same experimental paradigm in humans, substantial similarities between these species were uncovered by various econometric analyses of our dynamic prospect theory model, which incorporates decision-by-decision learning dynamics of prediction errors into static prospect theory. Our model provides a unified theoretical framework for exploring a neurobiological model of economic choice in human and nonhuman primates.

Preferences reveal separable valuation systems in prefrontal-limbic circuits

Individual preferences for the flavor of different foods and fluids exert a strong influence on behavior. Most current theories posit that preferences are integrated with other state variables in orbitofrontal cortex (OFC), which is thought to derive the relative subjective value of available options to drive choice behavior. Here we report that instead of a single integrated valuation system in OFC, another separate one is centered in ventrolateral prefrontal cortex (vlPFC) in macaque monkeys. Specifically, we found that OFC and vlPFC preferentially represent outcome flavor and outcome probability, respectively, and that preferences are separately integrated into these two aspects of subjective valuation. In addition, vlPFC, but not OFC, represented the outcome probability for the two options separately, with the difference between these representations reflecting the degree of preference. Thus, there are at least two separable valuation systems that work in concert to guide choices and that both are biased by preferences.

Systematic comparison of risky choices in humans and monkeys

The past decades have seen tremendous progress in fundamental studies on economic choice in humans. However, elucidation of the underlying neuronal processes requires invasive neurophysiological studies that are met with difficulties in humans. Monkeys as evolutionary closest relatives offer a solution. The animals display sophisticated and well-controllable behavior that allows to implement key constructs of proven economic choice theories. However, the similarity of economic choice between the two species has never been systematically investigated. We investigated compliance with the independence axiom (IA) of expected utility theory as one of the most demanding choice tests and compared IA violations between humans and monkeys. Using generalized linear modeling and cumulative prospect theory (CPT), we found that humans and monkeys made comparable risky choices, although their subjective values (utilities) differed. These results suggest similar fundamental choice mechanism across these primate species and encourage to study their underlying neurophysiological mechanisms.

Orbitofrontal cortex contributes to the comparison of values underlying economic choices

Economic choices between goods entail the computation and comparison of subjective values. Previous studies examined neuronal activity in the orbitofrontal cortex (OFC) of monkeys choosing between different types of juices. Three groups of neurons were identified: offer value cells encoding the value of individual offers, chosen juice cells encoding the identity of the chosen juice, and chosen value cells encoding the value of the chosen offer. The encoded variables capture both the input (offer value) and the output (chosen juice, chosen value) of the decision process, suggesting that values are compared within OFC. Recent work demonstrates that choices are causally linked to the activity of offer value cells. Conversely, the hypothesis that OFC contributes to value comparison has not been confirmed. Here we show that weak electrical stimulation of OFC specifically disrupts value comparison without altering offer values. This result implies that neuronal populations in OFC participate in value comparison.

Risky choice: Probability weighting explains independence axiom violations in monkeys

Expected Utility Theory (EUT) provides axioms for maximizing utility in risky choice. The Independence Axiom (IA) is its most demanding axiom: preferences between two options should not change when altering both options equally by mixing them with a common gamble. We tested common consequence (CC) and common ratio (CR) violations of the IA over several months in thousands of stochastic choices using a large variety of binary option sets. Three monkeys showed consistently few outright Preference Reversals (8%) but substantial graded Preference Changes (46%) between the initial preferred gamble and the corresponding altered gamble. Linear Discriminant Analysis (LDA) indicated that gamble probabilities predicted most Preference Changes in CC (72%) and CR (88%) tests. The Akaike Information Criterion indicated that probability weighting within Cumulative Prospect Theory (CPT) explained choices better than models using Expected Value (EV) or EUT. Fitting by utility and probability weighting functions of CPT resulted in nonlinear and non-parallel indifference curves (IC) in the Marschak-Machina triangle and suggested IA non-compliance of models using EV or EUT. Indeed, CPT models predicted Preference Changes better than EV and EUT models. Indifference points in out-of-sample tests were closer to CPT-estimated ICs than EV and EUT ICs. Finally, while the few outright Preference Reversals may reflect the long experience of our monkeys, their more graded Preference Changes corresponded to those reported for humans. In benefitting from the wide testing possibilities in monkeys, our stringent axiomatic tests contribute critical information about risky decision-making and serves as basis for investigating neuronal decision mechanisms.

Supplementary information

The online version contains supplementary material available at 10.1007/s11166-022-09388-7.

Logistic analysis of choice data: A primer

Logistic regressions were developed in economics to model individual choice behavior. In recent years, they have become an important tool in decision neuroscience. Here, I describe and discuss different logistic models, emphasizing the underlying assumptions and possible interpretations. Logistic models may be used to quantify a variety of behavioral traits, including the relative subjective value of different goods, the choice accuracy, risk attitudes, and choice biases. More complex logistic models can be used for choices between good bundles, in cases of nonlinear value functions, and for choices between multiple options. Finally, logistic models can quantify the explanatory power of neuronal activity on choices, thus providing a valid alternative to receiver operating characteristic (ROC) analyses.

Neuronal origins of reduced accuracy and biases in economic choices under sequential offers

Economic choices are characterized by a variety of biases. Understanding their origins is a long-term goal for neuroeconomics, but progress on this front has been limited. Here, we examined choice biases observed when two goods are offered sequentially. In the experiments, rhesus monkeys chose between different juices offered simultaneously or in sequence. Choices under sequential offers were less accurate (higher variability). They were also biased in favor of the second offer (order bias) and in favor of the preferred juice (preference bias). Analysis of neuronal activity recorded in the orbitofrontal cortex revealed that these phenomena emerged at different computational stages. Lower choice accuracy reflected weaker offer value signals (valuation stage), the order bias emerged during value comparison (decision stage), and the preference bias emerged late in the trial (post-comparison). By neuronal measures, each phenomenon reduced the value obtained on average in each trial and was thus costly to the monkey.

Economic Choices under Simultaneous or Sequential Offers Rely on the Same Neural Circuit

A series of studies in which monkeys chose between two juices offered in variable amounts identified in the orbitofrontal cortex (OFC) different groups of neurons encoding the value of individual options (offer value), the binary choice outcome (chosen juice), and the chosen value. These variables capture both the input and the output of the choice process, suggesting that the cell groups identified in OFC constitute the building blocks of a decision circuit. Several lines of evidence support this hypothesis. However, in previous experiments offers were presented simultaneously, raising the question of whether current notions generalize to when goods are presented or are examined in sequence. Recently, Ballesta and Padoa-Schioppa (2019) examined OFC activity under sequential offers. An analysis of neuronal responses across time windows revealed that a small number of cell groups encoded specific sequences of variables. These sequences appeared analogous to the variables identified under simultaneous offers, but the correspondence remained tentative. Thus, in the present study, we examined the relation between cell groups found under sequential versus simultaneous offers. We recorded from the OFC while monkeys chose between different juices. Trials with simultaneous and sequential offers were randomly interleaved in each session. We classified cells in each choice modality, and we examined the relation between the two classifications. We found a strong correspondence; in other words, the cell groups measured under simultaneous offers and under sequential offers were one and the same. This result indicates that economic choices under simultaneous or sequential offers rely on the same neural circuit.SIGNIFICANCE STATEMENT Research in the past 20 years has shed light on the neuronal underpinnings of economic choices. A large number of results indicates that decisions between goods are formed in a neural circuit within the orbitofrontal cortex. In most previous studies, subjects chose between two goods offered simultaneously. Yet, in daily situations, goods available for choice are often presented or examined in sequence. Here we recorded neuronal activity in the primate orbitofrontal cortex alternating trials under simultaneous and under sequential offers. Our analyses demonstrate that the same neural circuit supports choices in the two modalities. Hence, current notions on the neuronal mechanisms underlying economic decisions generalize to choices under sequential offers.

The Neural Instantiation of an Abstract Cognitive Map for Economic Choice

Since the discovery of cognitive maps in rodent hippocampus (HC), the cognitive map has evolved from originally referring to spatial representations encoding locations and objects in Euclidean spaces to a general low-dimensional organization of information along selected feature dimensions. A cognitive map includes hypothetical constructs that bridge between environmental stimuli and the final overt behavior. To neuroeconomists, utility and utility functions are such constructs with neurobiological basis that drive choice behavior. Emergence of distinct functional neuron groups in the primate orbitofrontal cortex (OFC) during simple economic choice indicates the formation of an abstract cognitive map for organizing information of goods for value computation. Experimental evidence suggests that organization of neuronal activity in such cognitive map reflects the abstraction of core task features. Thus, such map can be adapted to accommodate economic choices under various task contexts.

Reward-specific satiety affects subjective value signals in orbitofrontal cortex during multicomponent economic choice

Sensitivity to satiety constitutes a basic requirement for neuronal coding of subjective reward value. Satiety from natural ongoing consumption affects reward functions in learning and approach behavior. More specifically, satiety reduces the subjective economic value of individual rewards during choice between options that typically contain multiple reward components. The unconfounded assessment of economic reward value requires tests at choice indifference between two options, which is difficult to achieve with sated rewards. By conceptualizing choices between options with multiple reward components ("bundles"), Revealed Preference Theory may offer a solution. Despite satiety, choices against an unaltered reference bundle may remain indifferent when the reduced value of a sated bundle reward is compensated by larger amounts of an unsated reward of the same bundle, and then the value loss of the sated reward is indicated by the amount of the added unsated reward. Here, we show psychophysically titrated choice indifference in monkeys between bundles of differently sated rewards. Neuronal chosen value signals in the orbitofrontal cortex (OFC) followed closely the subjective value change within recording periods of individual neurons. A neuronal classifier distinguishing the bundles and predicting choice substantiated the subjective value change. The choice between conventional single rewards confirmed the neuronal changes seen with two-reward bundles. Thus, reward-specific satiety reduces subjective reward value signals in OFC. With satiety being an important factor of subjective reward value, these results extend the notion of subjective economic reward value coding in OFC neurons.

Preferences for nutrients and sensory food qualities identify biological sources of economic values in monkeys

Value is a foundational concept in reinforcement learning and economic choice theory. In these frameworks, individuals choose by assigning values to objects and learn by updating values with experience. These theories have been instrumental for revealing influences of probability, risk, and delay on choices. However, they do not explain how values are shaped by intrinsic properties of the choice objects themselves. Here, we investigated how economic value derives from the biologically critical components of foods: their nutrients and sensory qualities. When monkeys chose nutrient-defined liquids, they consistently preferred fat and sugar to low-nutrient alternatives. Rather than maximizing energy indiscriminately, they seemed to assign subjective values to specific nutrients, flexibly trading them against offered reward amounts. Nutrient-value functions accurately modeled these preferences, predicted choices across contexts, and accounted for individual differences. The monkeys' preferences shifted their daily nutrient balance away from dietary reference points, contrary to ecological foraging models but resembling human suboptimal eating in free-choice situations. To identify the sensory basis of nutrient values, we developed engineering tools that measured food textures on biological surfaces, mimicking oral conditions. Subjective valuations of two key texture parameters-viscosity and sliding friction-explained the monkeys' fat preferences, suggesting a texture-sensing mechanism for nutrient values. Extended reinforcement learning and choice models identified candidate neuronal mechanisms for nutrient-sensitive decision-making. These findings indicate that nutrients and food textures constitute critical reward components that shape economic values. Our nutrient-choice paradigm represents a promising tool for studying food-reward mechanisms in primates to better understand human-like eating behavior and obesity.

Single-Dimensional Human Brain Signals for Two-Dimensional Economic Choice Options

Rewarding choice options typically contain multiple components, but neural signals in single brain voxels are scalar and primarily vary up or down. In a previous study, we had designed reward bundles that contained the same two milkshakes with independently set amounts; we had used psychophysics and rigorous economic concepts to estimate two-dimensional choice indifference curves (ICs) that represented revealed stochastic preferences for these bundles in a systematic, integrated manner. All bundles on the same ICs were equally revealed preferred (and thus had same utility, as inferred from choice indifference); bundles on higher ICs (higher utility) were preferred to bundles on lower ICs (lower utility). In the current study, we used the established behavior for testing with functional magnetic resonance imaging (fMRI). We now demonstrate neural responses in reward-related brain structures of human female and male participants, including striatum, midbrain, and medial orbitofrontal cortex (mid-OFC) that followed the characteristic pattern of ICs: similar responses along ICs (same utility despite different bundle composition), but monotonic change across ICs (different utility). Thus, these brain structures integrated multiple reward components into a scalar signal, well beyond the known subjective value coding of single-component rewards.SIGNIFICANCE STATEMENT Rewards have several components, like the taste and size of an apple, but it is unclear how each component contributes to the overall value of the reward. While choice indifference curves (ICs) of economic theory provide behavioral approaches to this question, it is unclear whether brain responses capture the preference and utility integrated from multiple components. We report activations in striatum, midbrain, and orbitofrontal cortex (OFC) that follow choice ICs representing behavioral preferences over and above variations of individual reward components. In addition, the concept-driven approach encourages future studies on natural, multicomponent rewards that are prone to irrational choice of normal and brain-damaged individuals.

Excitatory-inhibitory tone shapes decision strategies in a hierarchical neural network model of multi-attribute choice

We are constantly faced with decisions between alternatives defined by multiple attributes, necessitating an evaluation and integration of different information sources. Time-varying signals in multiple brain areas are implicated in decision-making; but we lack a rigorous biophysical description of how basic circuit properties, such as excitatory-inhibitory (E/I) tone and cascading nonlinearities, shape attribute processing and choice behavior. Furthermore, how such properties govern choice performance under varying levels of environmental uncertainty is unknown. We investigated two-attribute, two-alternative decision-making in a dynamical, cascading nonlinear neural network with three layers: an input layer encoding choice alternative attribute values; an intermediate layer of modules processing separate attributes; and a final layer producing the decision. Depending on intermediate layer E/I tone, the network displays distinct regimes characterized by linear (I), convex (II) or concave (III) choice indifference curves. In regimes I and II, each option's attribute information is additively integrated. In regime III, time-varying nonlinear operations amplify the separation between offer distributions by selectively attending to the attribute with the larger differences in input values. At low environmental uncertainty, a linear combination most consistently selects higher valued alternatives. However, at high environmental uncertainty, regime III is more likely than a linear operation to select alternatives with higher value. Furthermore, there are conditions where readout from the intermediate layer could be experimentally indistinguishable from the final layer. Finally, these principles are used to examine multi-attribute decisions in systems with reduced inhibitory tone, leading to predictions of different choice patterns and overall performance between those with restrictions on inhibitory tone and neurotypicals.

Conditional valuation for combinations of goods in primates

Valuing goods and selecting the one with the highest value forms the basis of adaptive behaviour across species. While it is obvious that the valuation of a given type of goods depends on ownership and availability of that type of goods, the effects of other goods on valuation of the original good are sometimes underappreciated. Yet, goods interact with each other, indicating that the valuation of a given good is conditional on the other goods it is combined with, both in the wild and the laboratory. Here, we introduce conditional valuation in the context of valuing multiple goods and briefly review how human and animal experimentalists can leverage economic tools for the study of interactions among goods. We then review evidence for conditional valuation for combined foods in both human and non-human primates. In the laboratory, non-human primates show increased valuation of certain combinations of foods but decreased valuation of other types of combinations. Thus, similarly to humans, monkeys appear to value combinations of goods in a conditional fashion. Additionally, both humans and monkeys appear to employ similar neural substrates for the valuation of single goods, such as the orbitofrontal cortex. Together, investigations of our evolutionary precursors may provide insights on how we value interacting goods. This article is part of the theme issue 'Existence and prevalence of economic behaviours among non-human primates'.

Values encoded in orbitofrontal cortex are causally related to economic choices

In the eighteenth century, Daniel Bernoulli, Adam Smith and Jeremy Bentham proposed that economic choices rely on the computation and comparison of subjective values1. This hypothesis continues to inform modern economic theory2 and research in behavioural economics3, but behavioural measures are ultimately not sufficient to verify the proposal4. Consistent with the hypothesis, when agents make choices, neurons in the orbitofrontal cortex (OFC) encode the subjective value of offered and chosen goods5. Value-encoding cells integrate multiple dimensions6-9, variability in the activity of each cell group correlates with variability in choices10,11 and the population dynamics suggests the formation of a decision12. However, it is unclear whether these neural processes are causally related to choices. More generally, the evidence linking economic choices to value signals in the brain13-15 remains correlational16. Here we show that neuronal activity in the OFC is causal to economic choices. We conducted two experiments using electrical stimulation in rhesus monkeys (Macaca mulatta). Low-current stimulation increased the subjective value of individual offers and thus predictably biased choices. Conversely, high-current stimulation disrupted both the computation and the comparison of subjective values, and thus increased choice variability. These results demonstrate a causal chain linking subjective values encoded in OFC to valuation and choice.

Experimentally revealed stochastic preferences for multicomponent choice options

Realistic, everyday rewards contain multiple components. An apple has taste and size. However, we choose in single dimensions, simply preferring some apples to others. How can such single-dimensional preference relationships refer to multicomponent choice options? Here, we measured how stochastic choices revealed preferences for 2-component milkshakes. The preferences were intuitively graphed as indifference curves that represented the orderly integration of the 2 components as trade-off: parts of 1 component were given up for obtaining 1 additional unit of the other component without a change in preference. The well-ordered, nonoverlapping curves satisfied leave-one-out tests, followed predictions by machine learning decoders and correlated with single-dimensional Becker-DeGroot-Marschak (BDM) auction-like bids for the 2-component rewards. This accuracy suggests a decision process that integrates multiple reward components into single-dimensional estimates in a systematic fashion. In interspecies comparisons, human performance matched that of highly experienced laboratory monkeys, as measured by accuracy of the critical trade-off between bundle components. These data describe the nature of choices of multicomponent choice options and attest to the validity of the rigorous economic concepts and their convenient graphic schemes for explaining choices of human and nonhuman primates. The results encourage formal behavioral and neural investigations of normal, irrational, and pathological economic choices. (PsycInfo Database Record (c) 2020 APA, all rights reserved).