Primate anterior insular cortex represents economic decision variables proposed by prospect theory

The Isle of Craig: Neuroanatomical and Functional Evidence for a Role of the Insular Cortex in Subjective Feelings

At the start of the twenty-first century, Arthur D. (Bud) Craig brought back to the fore the Island of Reil (insular cortex or insula). He did so by following, step by step, with rigor and tenacity, the afferent sensory pathway that informs the forebrain about the ongoing physiological status of the organs and tissues of the body. Along with his demonstration of the existence of a primate-specific ascending interoceptive pathway and his subsequent re-interpretation of Sherrington's concept of interoception, Bud Craig's seminal experiments and profound interpretations led him to make the groundbreaking proposals that the dorsal posterior insular cortex provides an ideal substrate for James's concept of emotional embodiment, that the insular cortex contextualizes interoception across a posterior-to-mid-to-anterior integration with multimodal activities, and that the anterior insular cortex has a crucial role in the evolutionary emergence of the awareness of subjective feelings in humans, for the purpose of optimizing metabolic energy usage. Bud Craig's unique work paves the path for further elucidation of the role of the insula and other brain regions in subjective feelings. His discoveries and proposals rest on implacable attention to neuroanatomical and neurophysiological details and a serendipitous quest for the fundamental evolutionary Logic of Life. This chapter provides a detailed description of the ascending interoceptive pathway and the functional and comparative neuroanatomy of the insular cortex in primates. Building on Bud Craig's work, our recent findings suggest that the primary interoceptive cortex serves as a representation of the spino-solitary-parabrachial neuraxis, merging with posterior-to-mid-anterior and dorsal-to-ventral processing streams that form a latticework integration pattern. At the ventral anterior tip of this integration, the von Economo neuron area closes the corticofugal interoceptive-autonomic loop of the sensory-motor homeostatic system through projections to all brainstem nuclei integrating interoceptive afferences.

Dynamic coding and sequential integration of multiple reward attributes by primate amygdala neurons

The value of visual stimuli guides learning, decision-making, and motivation. Although stimulus values often depend on multiple attributes, how neurons extract and integrate distinct value components from separate cues remains unclear. Here we recorded the activity of amygdala neurons while two male monkeys viewed sequential cues indicating the probability and magnitude of expected rewards. Amygdala neurons frequently signaled reward probability in an abstract, stimulus-independent code that generalized across cue formats. While some probability-coding neurons were insensitive to magnitude information, signaling 'pure' probability rather than value, many neurons showed biphasic responses that signaled probability and magnitude in a dynamic (temporally-patterned) and flexible (reversible) value code. Specific amygdala neurons integrated these reward attributes into risk signals that quantified the variance of expected rewards, distinct from value. Population codes were accurate, mutually transferable between value components, and expressed differently across amygdala nuclei. Our findings identify amygdala neurons as a substrate for the sequential integration of multiple reward attributes into value and risk.

The amygdala and the pursuit of future rewards

The successful pursuit of future rewards requires forming an internal goal, followed by planning, decision-making, and progress-tracking over multiple steps. The initial step-forming goals and the plans for obtaining them-involves the subjective valuation of an anticipated reward, considering both the reward's properties and associated delay and physical-effort costs. Recent findings indicate individuals similarly evaluate cognitive effort over time (Johnson and Most, 2023). Success and failure in these processes have been linked to differential life outcomes and psychiatric conditions. Here we review evidence from single-neuron recordings and neuroimaging studies that implicate the amygdala-a brain structure long associated with cue-reactivity and emotion-in decision-making and the planned pursuit of future rewards (Grabenhorst et al., 2012, 2016, 2019, 2023;Hernadi et al., 2015;Zangemeister et al., 2016). The main findings are that, in behavioral tasks in which future rewards can be pursued through planning and stepwise decision-making, amygdala neurons prospectively encode the value of anticipated rewards and related behavioral plans. Moreover, amygdala neurons predict the stepwise choices to pursue these rewards, signal progress toward goals, and distinguish internally generated (i.e., self-determined) choices from externally imposed actions. Importantly, amygdala neurons integrate the subjective value of a future reward with delay and effort costs inherent in pursuing it. This neural evidence identifies three key computations of the primate amygdala that underlie the pursuit of future rewards: (1) forming a self-determined internal goal based on subjective reward-cost valuations, (2) defining a behavioral plan for obtaining the goal, (3) executing this plan through stepwise decision-making and progress-tracking. Based on this framework, we suggest that amygdala neurons constitute vulnerabilities for dysfunction that contribute to maladaptive reward pursuit in psychiatric and behavioral conditions. Consequently, amygdala neurons may also represent potential targets for behavioral-change interventions that aim to improve individual decision-making.

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions, with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, the OFC leads the processing of symbolic RL in the ventral frontostriatal circuitry.

Direct stimulation of anterior insula and ventromedial prefrontal cortex disrupts economic choices

Neural activity within the ventromedial prefrontal cortex (vmPFC) and anterior insula (aIns) is often associated with economic choices and confidence. However, it remains unclear whether these brain regions are causally related to these processes. To address this issue, we leveraged intracranial electrical stimulation (iES) data obtained from patients with epilepsy performing an economic choice task. Our results reveal opposite effects of stimulation on decision-making depending on its location along a dorso-ventral axis within each region. Specifically, stimulation of the ventral subregion within aIns reduces risk-taking by increasing participants' sensitivity to potential losses, whereas stimulation of the dorsal subregion of aIns and the ventral portion of the vmPFC increases risk-taking by reducing participants' sensitivity to losses. Moreover, stimulation of the aIns consistently decreases participants' confidence, regardless of its location within the aIns. These findings suggest the existence of functionally dissociated neural subregions and circuits causally involved in accepting or avoiding challenges.

Neuronal Activity in the Gustatory Cortex during Economic Choice

An economic choice entails computing and comparing the values of individual offers. Offer values are represented in the orbitofrontal cortex (OFC)-an area that participates in value comparison-but it is unknown where offer values are computed in the first place. One possibility is that this computation takes place in OFC. Alternatively, offer values might be computed upstream of OFC. For choices between edible goods, a primary candidate is the gustatory region of the anterior insula (gustatory cortex, GC). Here we recorded from the GC of male rhesus monkeys choosing between different juice types. As a population, neurons in GC represented the flavor, the quantity, and the subjective value of the juice chosen by the animal. These variables were represented by distinct groups of cells and with different time courses. Specifically, chosen value signals emerged shortly after offer presentation, while neurons encoding the chosen juice and the chosen quantity peaked after juice delivery. Surprisingly, neurons in GC did not represent individual offer values in a systematic way. In a computational sense, the variables encoded in GC follow the process of value comparison. Thus our results argue against the hypothesis that offer values are computed in GC. At the same time, signals representing the subjective value of the expected reward indicate that responses in GC are not purely sensory. Thus neuronal responses in GC appear consummatory in nature.

Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

Reinforcement learning is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g., money, points) that are later exchanged for primary reinforcers (e.g., food, drink). Although symbolic reinforcers are ubiquitous in our daily lives, widely used in laboratory tasks because they can be motivating, mechanisms by which they become motivating are less understood. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g., the current number of accumulated tokens, choice options, task epoch, trials since the last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to then correlate with the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n = 5 monkeys, three males and two females). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in the state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reinforcement learning (RL), particularly in primates, is often driven by symbolic outcomes. However, it is usually studied with primary reinforcers. To examine the neural mechanisms underlying learning from symbolic outcomes, we trained monkeys on a task in which they learned to choose options that led to gains of tokens and avoid choosing options that led to losses of tokens. We then recorded simultaneously from the orbitofrontal cortex (OFC), ventral striatum (VS), amygdala (AMY), and the mediodorsal thalamus (MDt). We found that the OFC played a dominant role in coding token outcomes and token prediction errors. The other areas contributed complementary functions with the VS coding appetitive outcomes and the AMY coding the salience of outcomes. The MDt coded actions and relayed information about tokens between the OFC and VS. Thus, OFC leads the process of symbolic reinforcement learning in the ventral frontostriatal circuitry.

Intrinsic functional clustering of the macaque insular cortex

The functional organization of the primate insula has been studied using a variety of techniques focussing on regional differences in either architecture, connectivity, or function. These complementary methods offered insights into the complex organization of the insula and proposed distinct parcellation schemes at varying levels of detail and complexity. The advent of imaging techniques that allow non-invasive assessment of structural and functional connectivity, has popularized data-driven connectivity-based parcellation methods to investigate the organization of the human insula. Yet, it remains unclear if the subdivisions derived from these data-driven clustering methods reflect meaningful descriptions of the functional specialization of the insula. In this study, we employed hierarchical clustering to examine the cluster parcellations of the macaque insula. As our aim was exploratory, we examined parcellations consisting of two up to ten clusters. Three different cluster validation methods (fingerprinting, silhouette, elbow) converged on a four-cluster solution as the most optimal representation of our data. Examining functional response properties of these clusters, in addition to their brain-wide functional connectivity suggested a functional specialization related to processing gustatory, somato-motor, vestibular and social visual cues. However, a more detailed functional differentiation aligning with previous functional investigations of insula subfields became evident at higher cluster numbers beyond the proposed optimal four clusters. Overall, our findings demonstrate that resting-state-based hierarchical clustering can provide a meaningful description of the insula's functional organization at some level of detail. Nonetheless, cluster parcellations derived from this method are best combined with data obtained through other modalities, to provide a more comprehensive and detailed account of the insula's complex functional organization.

A longitudinal study of perceived social position and health-related quality of life

BACKGROUND

A large body of evidence suggests that there is a social gradient in the association between perceived social position and various health outcomes. Yet only a fraction of this research uses longitudinal data, and these studies usually rely on two data points in time, consider a single health outcome measure, overlook non-linear effects of perceived social position, and come almost exclusively from the Western welfare democracies.

METHODS

Using data for 1921 individuals from three waves (2008, 2013, 2018) of the Polish Panel Survey (POLPAN), we fit between- and within-individuals hybrid-effects models with cluster-robust standard errors to investigate the association between one's perceived social position (self-placement on a socioeconomic hierarchy scale varying 1 to 10) and subsequent health-related quality of life (HRQoL) measured using the Nottingham Health Profile (NHP) and its six components.

RESULTS

We find that the association between perceived social position and health-related quality of life is larger when estimated between individuals than within individuals, yet in fixed- and hybrid-effects models perceived social position remains significantly and negatively linked with both the aggregated NHP measure as well as with its components such as emotional reaction, physical abilities, sleep, and social isolation. We also identify that starting to perceive oneself at the lower end of the social hierarchy is associated with a deteriorating health-related quality of life but a change at the top of the perceived social hierarchy is not linked with an improvement in NHP scores.

CONCLUSIONS

We provide new evidence on the significant and non-linear links between perceived social position and health-related quality of life and highlight possible pathways linking these two aspects of individuals' lives.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

A view-based decision mechanism for rewards in the primate amygdala

Primates make decisions visually by shifting their view from one object to the next, comparing values between objects, and choosing the best reward, even before acting. Here, we show that when monkeys make value-guided choices, amygdala neurons encode their decisions in an abstract, purely internal representation defined by the monkey's current view but not by specific object or reward properties. Across amygdala subdivisions, recorded activity patterns evolved gradually from an object-specific value code to a transient, object-independent code in which currently viewed and last-viewed objects competed to reflect the emerging view-based choice. Using neural-network modeling, we identified a sequence of computations by which amygdala neurons implemented view-based decision making and eventually recovered the chosen object's identity when the monkeys acted on their choice. These findings reveal a neural mechanism in the amygdala that derives object choices from abstract, view-based computations, suggesting an efficient solution for decision problems with many objects.

Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex

Evidence from monkeys and humans suggests that the orbitofrontal cortex (OFC) encodes the subjective value of options under consideration during choice. Data from non-human primates suggests that these value signals are context-dependent, representing subjective value in a way influenced by the decision makers' recent experience. Using electrodes distributed throughout cortical and subcortical structures, human epilepsy patients performed an auction task where they repeatedly reported the subjective values they placed on snack food items. High-gamma activity in many cortical and subcortical sites including the OFC positively correlated with subjective value. Other OFC sites showed signals contextually modulated by the subjective value of previously offered goods-a context dependency predicted by theory but not previously observed in humans. These results suggest that value and value-context signals are simultaneously present but separately represented in human frontal cortical activity.

Computational mechanisms underlying motivation to earn symbolic reinforcers

Reinforcement learning (RL) is a theoretical framework that describes how agents learn to select options that maximize rewards and minimize punishments over time. We often make choices, however, to obtain symbolic reinforcers (e.g. money, points) that can later be exchanged for primary reinforcers (e.g. food, drink). Although symbolic reinforcers are motivating, little is understood about the neural or computational mechanisms underlying the motivation to earn them. In the present study, we examined how monkeys learn to make choices that maximize fluid rewards through reinforcement with tokens. The question addressed here is how the value of a state, which is a function of multiple task features (e.g. current number of accumulated tokens, choice options, task epoch, trials since last delivery of primary reinforcer, etc.), drives value and affects motivation. We constructed a Markov decision process model that computes the value of task states given task features to capture the motivational state of the animal. Fixation times, choice reaction times, and abort frequency were all significantly related to values of task states during the tokens task (n=5 monkeys). Furthermore, the model makes predictions for how neural responses could change on a moment-by-moment basis relative to changes in state value. Together, this task and model allow us to capture learning and behavior related to symbolic reinforcement.

Functional characterization of macaque insula using task-based and resting-state fMRI

Neurophysiological investigations over the past decades have demonstrated the involvement of the primate insula in a wide array of sensory, cognitive, affective and regulatory functions, yet the complex functional organization of the insula remains unclear. Here we examined to what extent non-invasive task-based and resting-state fMRI provides support for functional specialization and integration of sensory and motor information in the macaque insula. Task-based fMRI experiments suggested a functional specialization related to processing of ingestive/taste/distaste information in anterior insula, grasping-related sensorimotor responses in middle insula and vestibular information in posterior insula. Visual stimuli depicting social information involving conspecific`s lip-smacking gestures yielded responses in middle and anterior portions of dorsal and ventral insula, overlapping partially with the sensorimotor and ingestive/taste/distaste fields. Functional specialization/integration of the insula was further corroborated by seed-based whole brain resting-state analyses, showing distinct functional connectivity gradients across the anterio-posterior extent of both dorsal and ventral insula. Posterior insula showed functional correlations in particular with vestibular/optic flow network regions, mid-dorsal insula with vestibular/optic flow as well as parieto-frontal regions of the sensorimotor grasping network, mid-ventral insula with social/affiliative network regions in temporal, cingulate and prefrontal cortices and anterior insula with taste and mouth motor networks including premotor and frontal opercular regions.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Systematic comparison of risky choices in humans and monkeys

The past decades have seen tremendous progress in fundamental studies on economic choice in humans. However, elucidation of the underlying neuronal processes requires invasive neurophysiological studies that are met with difficulties in humans. Monkeys as evolutionary closest relatives offer a solution. The animals display sophisticated and well-controllable behavior that allows to implement key constructs of proven economic choice theories. However, the similarity of economic choice between the two species has never been systematically investigated. We investigated compliance with the independence axiom (IA) of expected utility theory as one of the most demanding choice tests and compared IA violations between humans and monkeys. Using generalized linear modeling and cumulative prospect theory (CPT), we found that humans and monkeys made comparable risky choices, although their subjective values (utilities) differed. These results suggest similar fundamental choice mechanism across these primate species and encourage to study their underlying neurophysiological mechanisms.

A neuronal prospect theory model in the brain reward circuitry

Prospect theory, arguably the most prominent theory of choice, is an obvious candidate for neural valuation models. How the activity of individual neurons, a possible computational unit, obeys prospect theory remains unknown. Here, we show, with theoretical accuracy equivalent to that of human neuroimaging studies, that single-neuron activity in four core reward-related cortical and subcortical regions represents the subjective valuation of risky gambles in monkeys. The activity of individual neurons in monkeys passively viewing a lottery reflects the desirability of probabilistic rewards parameterized as a multiplicative combination of utility and probability weighting functions, as in the prospect theory framework. The diverse patterns of valuation signals were not localized but distributed throughout most parts of the reward circuitry. A network model aggregating these signals reconstructed the risk preferences and subjective probability weighting revealed by the animals’ choices. Thus, distributed neural coding explains the computation of subjective valuations under risk.

It is unclear how the activity of individual neurons conform to prospect theory. Here, the authors demonstrate that the activity of single neurons in various reward-related regions in the monkey brain can be described as encoding a multiplicative combination of utility and probability weighting, and that this subjective valuation process is achieved via a distributed coding scheme.