1
|
Marshall TR, Ruesseler M, Hunt LT, O’Reilly JX. The representation of priors and decisions in the human parietal cortex. PLoS Biol 2024; 22:e3002383. [PMID: 38285671 PMCID: PMC10824454 DOI: 10.1371/journal.pbio.3002383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Accepted: 12/11/2023] [Indexed: 01/31/2024] Open
Abstract
Animals actively sample their environment through orienting actions such as saccadic eye movements. Saccadic targets are selected based both on sensory evidence immediately preceding the saccade, and a "salience map" or prior built-up over multiple saccades. In the primate cortex, the selection of each individual saccade depends on competition between target-selective cells that ramp up their firing rate to saccade release. However, it is less clear how a cross-saccade prior might be implemented, either in neural firing or through an activity-silent mechanism such as modification of synaptic weights on sensory inputs. Here, we present evidence from magnetoencephalography for 2 distinct processes underlying the selection of the current saccade, and the representation of the prior, in human parietal cortex. While the classic ramping decision process for each saccade was reflected in neural firing rates (measured in the event-related field), a prior built-up over multiple saccades was implemented via modulation of the gain on sensory inputs from the preferred target, as evidenced by rapid frequency tagging. A cascade of computations over time (initial representation of the prior, followed by evidence accumulation and then an integration of prior and evidence) provides a mechanism by which a salience map may be built up across saccades in parietal cortex. It also provides insight into the apparent contradiction that inactivation of parietal cortex has been shown not to affect performance on single-trials, despite the presence of clear evidence accumulation signals in this region.
Collapse
Affiliation(s)
- Tom R. Marshall
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| | - Maria Ruesseler
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | - Laurence T. Hunt
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | - Jill X. O’Reilly
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, Nuffield Department for Clinical Neurosciences, Oxford University, Oxford, United Kingdom
| |
Collapse
|
2
|
Levitas DJ, Folco KL, James TW. Impact of aversive affect on neural mechanisms of categorization decisions. Brain Behav 2023; 13:e3312. [PMID: 37969052 PMCID: PMC10726818 DOI: 10.1002/brb3.3312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2023] [Revised: 09/13/2023] [Accepted: 10/24/2023] [Indexed: 11/17/2023] Open
Abstract
INTRODUCTION Many theories contend that evidence accumulation is a critical component of decision-making. Cognitive accumulation models typically interpret two main parameters: a drift rate and decision threshold. The former is the rate of accumulation, based on the quality of evidence, and the latter is the amount of evidence required for a decision. Some studies have found neural signals that mimic evidence accumulators and can be described by the two parameters. However, few studies have related these neural parameters to experimental manipulations of sensory data or memory representations. Here, we investigated the influence of affective salience on neural accumulation parameters. High affective salience has been repeatedly shown to influence decision-making, yet its effect on neural evidence accumulation has been unexamined. METHODS The current study used a two-choice object categorization task of body images (feet or hands). Half the images in each category were high in affective salience because they contained highly aversive features (gore and mutilation). To study such quick categorization decisions with a relatively slow technique like functional magnetic resonance imaging, we used a gradual reveal paradigm to lengthen cognitive processing time through the gradual "unmasking" of stimuli. RESULTS Because the aversive features were task-irrelevant, high affective salience produced a distractor effect, slowing decision time. In visual accumulation regions of interest, high affective salience produced a longer time to peak activation. Unexpectedly, the later peak appeared to be the product of changes to both drift rate and decision threshold. The drift rate for high affective salience was shallower, and the decision threshold was greater. To our knowledge, this is the first demonstration of an experimental manipulation of sensory data or memory representations that changed the neural decision threshold. CONCLUSION These findings advance our knowledge of the neural mechanisms underlying affective responses in general and the influence of high affective salience on object representations and categorization decisions.
Collapse
Affiliation(s)
- Daniel J. Levitas
- Department of Psychological and Brain SciencesIndiana UniversityBloomingtonIndianaUSA
| | - Kess L. Folco
- Department of Psychological and Brain SciencesIndiana UniversityBloomingtonIndianaUSA
| | - Thomas W. James
- Department of Psychological and Brain SciencesIndiana UniversityBloomingtonIndianaUSA
| |
Collapse
|
3
|
Perkins AQ, Gillis ZS, Rich EL. Multi-attribute decision-making in macaques relies on direct attribute comparisons. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.22.563329. [PMID: 37961522 PMCID: PMC10634707 DOI: 10.1101/2023.10.22.563329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
In value-based decisions, there are frequently multiple attributes, such as cost, quality, or quantity, that contribute to the overall goodness of an option. Since one option may not be better in all attributes at once, the decision process should include a means of weighing relevant attributes. Most decision-making models solve this problem by computing an integrated value, or utility, for each option from a weighted combination of attributes. However, behavioral anomalies in decision-making, such as context effects, indicate that other attribute-specific computations might be taking place. Here, we tested whether rhesus macaques show evidence of attribute-specific processing in a value-based decision-making task. Monkeys made a series of decisions involving choice options comprising a sweetness and probability attribute. Each attribute was represented by a separate bar with one of two mappings between bar size and the magnitude of the attribute (i.e., bigger=better or bigger=worse). We found that translating across different mappings produced selective impairments in decision-making. When like attributes differed, monkeys were prevented from easily making direct attribute comparisons, and choices were less accurate and preferences were more variable. This was not the case when mappings of unalike attributes within the same option were different. Likewise, gaze patterns favored transitions between like attributes over transitions between unalike attributes of the same option, so that like attributes were sampled sequentially to support within-attribute comparisons. Together, these data demonstrate that value-based decisions rely, at least in part, on directly comparing like attributes of multi-attribute options. Significance Statement Value-based decision-making is a cognitive function impacted by a number of clinical conditions, including substance use disorder and mood disorders. Understanding the neural mechanisms, including online processing steps involved in decision formation, will provide critical insights into decision-making deficits characteristic of human psychiatric disorders. Using rhesus monkeys as a model species capable of complex decision-making, this study shows that decisions involve a process of comparing like features, or attributes, of multi-attribute options. This is contrary to popular models of decision-making in which attributes are first combined into an overall value, or utility, to make a choice. Therefore, these results serve as an important foundation for establishing a more complete understanding of the neural mechanisms involved in forming complex decisions.
Collapse
|
4
|
Voloh B, Maisson DJN, Cervera RL, Conover I, Zambre M, Hayden B, Zimmermann J. Hierarchical action encoding in prefrontal cortex of freely moving macaques. Cell Rep 2023; 42:113091. [PMID: 37656619 PMCID: PMC10591875 DOI: 10.1016/j.celrep.2023.113091] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 06/23/2023] [Accepted: 08/18/2023] [Indexed: 09/03/2023] Open
Abstract
Our natural behavioral repertoires include coordinated actions of characteristic types. To better understand how neural activity relates to the expression of actions and action switches, we studied macaques performing a freely moving foraging task in an open environment. We developed a novel analysis pipeline that can identify meaningful units of behavior, corresponding to recognizable actions such as sitting, walking, jumping, and climbing. On the basis of transition probabilities between these actions, we found that behavior is organized in a modular and hierarchical fashion. We found that, after regressing out many potential confounders, actions are associated with specific patterns of firing in each of six prefrontal brain regions and that, overall, encoding of action category is progressively stronger in more dorsal and more caudal prefrontal regions. Together, these results establish a link between selection of units of primate behavior on one hand and neuronal activity in prefrontal regions on the other.
Collapse
Affiliation(s)
- Benjamin Voloh
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - David J-N Maisson
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | | | - Indirah Conover
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Mrunal Zambre
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA; Department of Neurosurgery, Baylor College of Medicine, Houston, TX 77030, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN 55455, USA; Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
5
|
Balewski ZZ, Elston TW, Knudsen EB, Wallis JD. Value dynamics affect choice preparation during decision-making. Nat Neurosci 2023; 26:1575-1583. [PMID: 37563295 PMCID: PMC10576429 DOI: 10.1038/s41593-023-01407-3] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 07/17/2023] [Indexed: 08/12/2023]
Abstract
During decision-making, neurons in the orbitofrontal cortex (OFC) sequentially represent the value of each option in turn, but it is unclear how these dynamics are translated into a choice response. One brain region that may be implicated in this process is the anterior cingulate cortex (ACC), which strongly connects with OFC and contains many neurons that encode the choice response. We investigated how OFC value signals interacted with ACC neurons encoding the choice response by performing simultaneous high-channel count recordings from the two areas in nonhuman primates. ACC neurons encoding the choice response steadily increased their firing rate throughout the decision-making process, peaking shortly before the time of the choice response. Furthermore, the value dynamics in OFC affected ACC ramping-when OFC represented the more valuable option, ACC ramping accelerated. Because OFC tended to represent the more valuable option more frequently and for a longer duration, this interaction could explain how ACC selects the more valuable response.
Collapse
Affiliation(s)
- Zuzanna Z Balewski
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA
| | - Thomas W Elston
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA
| | - Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA
| | - Joni D Wallis
- Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, USA.
- Department of Psychology, University of California at Berkeley, Berkeley, CA, USA.
| |
Collapse
|
6
|
Temporal scaling of human scalp-recorded potentials. Proc Natl Acad Sci U S A 2022; 119:e2214638119. [PMID: 36256817 PMCID: PMC9618087 DOI: 10.1073/pnas.2214638119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Neural activity is traditionally thought to occur over fixed timescales. However, recent animal work has suggested that some neural responses occur over varying timescales. We extended this animal result to humans by detecting temporally scaled signals noninvasively at the scalp in four different tasks. Our results suggest that temporal scaling is an important feature of cognitive processes known to unfold over varying timescales. Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp–recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.
Collapse
|
7
|
Abstract
People with damage to the orbitofrontal cortex (OFC) have specific problems making decisions, whereas their other cognitive functions are spared. Neurophysiological studies have shown that OFC neurons fire in proportion to the value of anticipated outcomes. Thus, a central role of the OFC is to guide optimal decision-making by signalling values associated with different choices. Until recently, this view of OFC function dominated the field. New data, however, suggest that the OFC may have a much broader role in cognition by representing cognitive maps that can be used to guide behaviour and that value is just one of many variables that are important for behavioural control. In this Review, we critically evaluate these two alternative accounts of OFC function and examine how they might be reconciled.
Collapse
Affiliation(s)
- Eric B Knudsen
- Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA
| | - Joni D Wallis
- Department of Psychology and Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, California, USA.
| |
Collapse
|
8
|
Weber LA, Tomiello S, Schöbi D, Wellstein KV, Mueller D, Iglesias S, Stephan KE. Auditory mismatch responses are differentially sensitive to changes in muscarinic acetylcholine versus dopamine receptor function. eLife 2022; 11:74835. [PMID: 35502897 PMCID: PMC9098218 DOI: 10.7554/elife.74835] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 04/26/2022] [Indexed: 11/13/2022] Open
Abstract
The auditory mismatch negativity (MMN) has been proposed as a biomarker of NMDA receptor (NMDAR) dysfunction in schizophrenia. Such dysfunction may be caused by aberrant interactions of different neuromodulators with NMDARs, which could explain clinical heterogeneity among patients. In two studies (N = 81 each), we used a double-blind placebo-controlled between-subject design to systematically test whether auditory mismatch responses under varying levels of environmental stability are sensitive to diminishing and enhancing cholinergic vs. dopaminergic function. We found a significant drug × mismatch interaction: while the muscarinic acetylcholine receptor antagonist biperiden delayed and topographically shifted mismatch responses, particularly during high stability, this effect could not be detected for amisulpride, a dopamine D2/D3 receptor antagonist. Neither galantamine nor levodopa, which elevate acetylcholine and dopamine levels, respectively, exerted significant effects on MMN. This differential MMN sensitivity to muscarinic versus dopaminergic receptor function may prove useful for developing tests that predict individual treatment responses in schizophrenia.
Collapse
Affiliation(s)
- Lilian Aline Weber
- Translational Neuroimaging Unit (TNU), Institute for Biomedical EngineeringInstitute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| | - Sara Tomiello
- Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| | - Dario Schöbi
- Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| | - Katharina V Wellstein
- Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| | - Daniel Mueller
- Institute for Clinical Chemistry, University Hospital of Zurich, Zurich, Switzerland
| | - Sandra Iglesias
- Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| | - Klaas Enno Stephan
- Translational Neuroimaging Unit (TNU), Institute for Biomedical Engineering, University of Zurich, Zurich, Switzerland
| |
Collapse
|
9
|
Ebitz RB, Hayden BY. The population doctrine in cognitive neuroscience. Neuron 2021; 109:3055-3068. [PMID: 34416170 PMCID: PMC8725976 DOI: 10.1016/j.neuron.2021.07.011] [Citation(s) in RCA: 60] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2021] [Revised: 07/02/2021] [Accepted: 07/13/2021] [Indexed: 01/08/2023]
Abstract
A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dimensions, (4) subspaces, and (5) dynamics. The work we review illustrates the progress and promise that population-level thinking holds for cognitive neuroscience-for delivering new insight into attention, working memory, decision-making, executive function, learning, and reward processing.
Collapse
Affiliation(s)
- R Becket Ebitz
- Department of Neurosciences, Faculté de médecine, Université de Montréal, Montréal, QC, Canada.
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, and Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
10
|
Maisson DJN, Cash-Padgett TV, Wang MZ, Hayden BY, Heilbronner SR, Zimmermann J. Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex. Nat Commun 2021; 12:4830. [PMID: 34376663 PMCID: PMC8355277 DOI: 10.1038/s41467-021-25219-w] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2020] [Accepted: 07/27/2021] [Indexed: 11/17/2022] Open
Abstract
Choice-relevant brain regions in prefrontal cortex may progressively transform information about options into choices. Here, we examine responses of neurons in four regions of the medial prefrontal cortex as macaques performed two-option risky choices. All four regions encode economic variables in similar proportions and show similar putative signatures of key choice-related computations. We provide evidence to support a gradient of function that proceeds from areas 14 to 25 to 32 to 24. Specifically, we show that decodability of twelve distinct task variables increases along that path, consistent with the idea that regions that are higher in the anatomical hierarchy make choice-relevant variables more separable. We also show progressively longer intrinsic timescales in the same series. Together these results highlight the importance of the medial wall in choice, endorse a specific gradient-based organization, and argue against a modular functional neuroanatomy of choice.
Collapse
Affiliation(s)
- David J-N Maisson
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA.
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA.
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA.
| | - Tyler V Cash-Padgett
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Maya Z Wang
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Sarah R Heilbronner
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
- Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
- Center for Neuroengineering, University of Minnesota, Minneapolis, MN, USA
- Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
11
|
Parr T, Rikhye RV, Halassa MM, Friston KJ. Prefrontal Computation as Active Inference. Cereb Cortex 2021; 30:682-695. [PMID: 31298270 PMCID: PMC7444741 DOI: 10.1093/cercor/bhz118] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2019] [Revised: 05/01/2019] [Accepted: 05/11/2019] [Indexed: 12/22/2022] Open
Abstract
The prefrontal cortex is vital for a range of cognitive processes, including working memory, attention, and decision-making. Notably, its absence impairs the performance of tasks requiring the maintenance of information through a delay period. In this paper, we formulate a rodent task—which requires maintenance of delay-period activity—as a Markov decision process and treat optimal task performance as an (active) inference problem. We simulate the behavior of a Bayes optimal mouse presented with 1 of 2 cues that instructs the selection of concurrent visual and auditory targets on a trial-by-trial basis. Formulating inference as message passing, we reproduce features of neuronal coupling within and between prefrontal regions engaged by this task. We focus on the micro-circuitry that underwrites delay-period activity and relate it to functional specialization within the prefrontal cortex in primates. Finally, we simulate the electrophysiological correlates of inference and demonstrate the consequences of lesions to each part of our in silico prefrontal cortex. In brief, this formulation suggests that recurrent excitatory connections—which support persistent neuronal activity—encode beliefs about transition probabilities over time. We argue that attentional modulation can be understood as the contextualization of sensory input by these persistent beliefs.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, UK
| | - Rajeev Vijay Rikhye
- Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Michael M Halassa
- Department of Brain and Cognitive Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.,Stanley Center for Psychiatric Genetics, Broad Institute, Cambridge, MA 02139, USA
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London, WC1N 3BG, UK
| |
Collapse
|
12
|
Azzalini D, Buot A, Palminteri S, Tallon-Baudry C. Responses to Heartbeats in Ventromedial Prefrontal Cortex Contribute to Subjective Preference-Based Decisions. J Neurosci 2021; 41:5102-5114. [PMID: 33926998 PMCID: PMC8197644 DOI: 10.1523/jneurosci.1932-20.2021] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Revised: 01/14/2021] [Accepted: 01/25/2021] [Indexed: 11/21/2022] Open
Abstract
Forrest Gump or The Matrix? Preference-based decisions are subjective and entail self-reflection. However, these self-related features are unaccounted for by known neural mechanisms of valuation and choice. Self-related processes have been linked to a basic interoceptive biological mechanism, the neural monitoring of heartbeats, in particular in ventromedial prefrontal cortex (vmPFC), a region also involved in value encoding. We thus hypothesized a functional coupling between the neural monitoring of heartbeats and the precision of value encoding in vmPFC. Human participants of both sexes were presented with pairs of movie titles. They indicated either which movie they preferred or performed a control objective visual discrimination that did not require self-reflection. Using magnetoencephalography, we measured heartbeat-evoked responses (HERs) before option presentation and confirmed that HERs in vmPFC were larger when preparing for the subjective, self-related task. We retrieved the expected cortical value network during choice with time-resolved statistical modeling. Crucially, we show that larger HERs before option presentation are followed by stronger value encoding during choice in vmPFC. This effect is independent of overall vmPFC baseline activity. The neural interaction between HERs and value encoding predicted preference-based choice consistency over time, accounting for both interindividual differences and trial-to-trial fluctuations within individuals. Neither cardiac activity nor arousal fluctuations could account for any of the effects. HERs did not interact with the encoding of perceptual evidence in the discrimination task. Our results show that the self-reflection underlying preference-based decisions involves HERs, and that HER integration to subjective value encoding in vmPFC contributes to preference stability.SIGNIFICANCE STATEMENT Deciding whether you prefer Forrest Gump or The Matrix is based on subjective values, which only you, the decision-maker, can estimate and compare, by asking yourself. Yet, how self-reflection is biologically implemented and its contribution to subjective valuation are not known. We show that in ventromedial prefrontal cortex, the neural response to heartbeats, an interoceptive self-related process, influences the cortical representation of subjective value. The neural interaction between the cortical monitoring of heartbeats and value encoding predicts choice consistency (i.e., whether you consistently prefer Forrest Gump over Matrix over time. Our results pave the way for the quantification of self-related processes in decision-making and may shed new light on the relationship between maladaptive decisions and impaired interoception.
Collapse
Affiliation(s)
- Damiano Azzalini
- Laboratoire de Neurosciences Cognitives et Computationnelles, Ecole Normale Supérieure, PSL University, 75005 Paris, France
- Institut National de la Santé et de la Recherche Médicale, 75005 Paris, France
| | - Anne Buot
- Laboratoire de Neurosciences Cognitives et Computationnelles, Ecole Normale Supérieure, PSL University, 75005 Paris, France
- Institut National de la Santé et de la Recherche Médicale, 75005 Paris, France
| | - Stefano Palminteri
- Laboratoire de Neurosciences Cognitives et Computationnelles, Ecole Normale Supérieure, PSL University, 75005 Paris, France
- Institut National de la Santé et de la Recherche Médicale, 75005 Paris, France
| | - Catherine Tallon-Baudry
- Laboratoire de Neurosciences Cognitives et Computationnelles, Ecole Normale Supérieure, PSL University, 75005 Paris, France
- Institut National de la Santé et de la Recherche Médicale, 75005 Paris, France
| |
Collapse
|
13
|
Zhang B, Kan JYY, Yang M, Wang X, Tu J, Dorris MC. Transforming absolute value to categorical choice in primate superior colliculus during value-based decision making. Nat Commun 2021; 12:3410. [PMID: 34099726 PMCID: PMC8184840 DOI: 10.1038/s41467-021-23747-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 04/23/2021] [Indexed: 11/09/2022] Open
Abstract
Value-based decision making involves choosing from multiple options with different values. Despite extensive studies on value representation in various brain regions, the neural mechanism for how multiple value options are converted to motor actions remains unclear. To study this, we developed a multi-value foraging task with varying menu of items in non-human primates using eye movements that dissociates value and choice, and conducted electrophysiological recording in the midbrain superior colliculus (SC). SC neurons encoded "absolute" value, independent of available options, during late fixation. In addition, SC neurons also represent value threshold, modulated by available options, different from conventional motor threshold. Electrical stimulation of SC neurons biased choices in a manner predicted by the difference between the value representation and the value threshold. These results reveal a neural mechanism directly transforming absolute values to categorical choices within SC, supporting highly efficient value-based decision making critical for real-world economic behaviors.
Collapse
Affiliation(s)
- Beizhen Zhang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China. .,University of Chinese Academy of Sciences, Beijing, China.
| | - Janis Ying Ying Kan
- Department of Biomedical and Molecular Sciences, Centre for Neuroscience Studies, Queen's University, Kingston, ON, Canada
| | - Mingpo Yang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Xiaochun Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Jiahao Tu
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China
| | - Michael Christopher Dorris
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
14
|
Hunt LT. Frontal circuit specialisations for decision making. Eur J Neurosci 2021; 53:3654-3671. [PMID: 33864305 DOI: 10.1111/ejn.15236] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2021] [Revised: 03/15/2021] [Accepted: 04/04/2021] [Indexed: 11/29/2022]
Abstract
There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.
Collapse
Affiliation(s)
- Laurence T Hunt
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
15
|
Takagi Y, Hunt LT, Woolrich MW, Behrens TEJ, Klein-Flügge MC. Adapting non-invasive human recordings along multiple task-axes shows unfolding of spontaneous and over-trained choice. eLife 2021; 10:e60988. [PMID: 33973522 PMCID: PMC8143794 DOI: 10.7554/elife.60988] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Accepted: 04/26/2021] [Indexed: 12/28/2022] Open
Abstract
Choices rely on a transformation of sensory inputs into motor responses. Using invasive single neuron recordings, the evolution of a choice process has been tracked by projecting population neural responses into state spaces. Here, we develop an approach that allows us to recover similar trajectories on a millisecond timescale in non-invasive human recordings. We selectively suppress activity related to three task-axes, relevant and irrelevant sensory inputs and response direction, in magnetoencephalography data acquired during context-dependent choices. Recordings from premotor cortex show a progression from processing sensory input to processing the response. In contrast to previous macaque recordings, information related to choice-irrelevant features is represented more weakly than choice-relevant sensory information. To test whether this mechanistic difference between species is caused by extensive over-training common in non-human primate studies, we trained humans on >20,000 trials of the task. Choice-irrelevant features were still weaker than relevant features in premotor cortex after over-training.
Collapse
Affiliation(s)
- Yu Takagi
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Department of Psychiatry, University of Oxford, Warneford HospitalOxfordUnited Kingdom
- Department of Neuropsychiatry, Graduate School of Medicine, University of TokyoTokyoJapan
| | - Laurence Tudor Hunt
- Department of Psychiatry, University of Oxford, Warneford HospitalOxfordUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, John Radcliffe HospitalOxfordUnited Kingdom
| | - Mark W Woolrich
- Department of Psychiatry, University of Oxford, Warneford HospitalOxfordUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, John Radcliffe HospitalOxfordUnited Kingdom
| | - Timothy EJ Behrens
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, John Radcliffe HospitalOxfordUnited Kingdom
- Wellcome Trust Centre for Neuroimaging, UCL Institute of Neurology, University College London (UCL)LondonUnited Kingdom
| | - Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of OxfordOxfordUnited Kingdom
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, John Radcliffe HospitalOxfordUnited Kingdom
| |
Collapse
|
16
|
Azab H, Hayden BY. Partial integration of the components of value in anterior cingulate cortex. Behav Neurosci 2021; 134:296-308. [PMID: 32658523 DOI: 10.1037/bne0000382] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
Evaluation often involves integrating multiple determinants of value, such as the different possible outcomes in risky choice. A brain region can be placed either before or after a presumed evaluation stage by measuring how responses of its neurons depend on multiple determinants of value. A brain region could also, in principle, show partial integration, which would indicate that it occupies a middle position between (preevaluative) nonintegration and (postevaluative) full integration. Existing mathematical techniques cannot distinguish full from partial integration and therefore risk misidentifying regional function. Here we use a new Bayesian regression-based approach to analyze responses of neurons in dorsal anterior cingulate cortex (dACC) to risky offers. We find that dACC neurons only partially integrate across outcome dimensions, indicating that dACC cannot be assigned to either a pre- or postevaluative position. Neurons in dACC also show putative signatures of value comparison, thereby demonstrating that comparison does not require complete evaluation before proceeding. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
- Habiba Azab
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research (CMRR), University of Minnesota, Twin Cities
| |
Collapse
|
17
|
Abstract
Comparative neuroscience is entering the era of big data. New high-throughput methods and data-sharing initiatives have resulted in the availability of large, digital data sets containing many types of data from ever more species. Here, we present a framework for exploiting the new possibilities offered. The multimodality of the data allows vertical translations, which are comparisons of different aspects of brain organization within a single species and across scales. Horizontal translations compare particular aspects of brain organization across species, often by building abstract feature spaces. Combining vertical and horizontal translations allows for more sophisticated comparisons, including relating principles of brain organization across species by contrasting horizontal translations, and for making formal predictions of unobtainable data based on observed results in a model species.
Collapse
Affiliation(s)
- Rogier B Mars
- Wellcome Centre for Integrative Neuroimaging, Centre for fMRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford OX3 9DU, United Kingdom; .,Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6525 HR Nijmegen, The Netherlands
| | - Saad Jbabdi
- Wellcome Centre for Integrative Neuroimaging, Centre for fMRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford OX3 9DU, United Kingdom;
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| |
Collapse
|
18
|
Adkins TJ, Lee TG. Reward modulates cortical representations of action. Neuroimage 2020; 228:117708. [PMID: 33385555 DOI: 10.1016/j.neuroimage.2020.117708] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2020] [Revised: 12/22/2020] [Accepted: 12/23/2020] [Indexed: 10/22/2022] Open
Abstract
People are capable of rapid improvements in performance when they are offered a reward. The neural mechanism by which this performance enhancement occurs remains unclear. We investigated this phenomenon by offering people monetary reward for successful performance in a sequence production task. We found that people performed actions more quickly and accurately when they were offered large reward. Increasing reward magnitude was associated with elevated activity throughout the brain prior to movement. Multivariate patterns of activity in these reward-responsive regions encoded information about the upcoming action. Follow-up analyses provided evidence that action decoding in pre-SMA and other motor planning areas was improved for large reward trials and successful action decoding in SMA was associated with improved performance. These results suggest that reward may enhance performance by enhancing neural representations of action used in motor planning.
Collapse
Affiliation(s)
- Tyler J Adkins
- Department of Psychology, University of Michigan, Ann Arbor, MI, 48109, USA.
| | - Taraz G Lee
- Department of Psychology, University of Michigan, Ann Arbor, MI, 48109, USA
| |
Collapse
|
19
|
Cavanagh SE, Hunt LT, Kennerley SW. A Diversity of Intrinsic Timescales Underlie Neural Computations. Front Neural Circuits 2020; 14:615626. [PMID: 33408616 PMCID: PMC7779632 DOI: 10.3389/fncir.2020.615626] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Accepted: 11/18/2020] [Indexed: 12/05/2022] Open
Abstract
Neural processing occurs across a range of temporal scales. To facilitate this, the brain uses fast-changing representations reflecting momentary sensory input alongside more temporally extended representations, which integrate across both short and long temporal windows. The temporal flexibility of these representations allows animals to behave adaptively. Short temporal windows facilitate adaptive responding in dynamic environments, while longer temporal windows promote the gradual integration of information across time. In the cognitive and motor domains, the brain sets overarching goals to be achieved within a long temporal window, which must be broken down into sequences of actions and precise movement control processed across much shorter temporal windows. Previous human neuroimaging studies and large-scale artificial network models have ascribed different processing timescales to different cortical regions, linking this to each region's position in an anatomical hierarchy determined by patterns of inter-regional connectivity. However, even within cortical regions, there is variability in responses when studied with single-neuron electrophysiology. Here, we review a series of recent electrophysiology experiments that demonstrate the heterogeneity of temporal receptive fields at the level of single neurons within a cortical region. This heterogeneity appears functionally relevant for the computations that neurons perform during decision-making and working memory. We consider anatomical and biophysical mechanisms that may give rise to a heterogeneity of timescales, including recurrent connectivity, cortical layer distribution, and neurotransmitter receptor expression. Finally, we reflect on the computational relevance of each brain region possessing a heterogeneity of neuronal timescales. We argue that this architecture is of particular importance for sensory, motor, and cognitive computations.
Collapse
Affiliation(s)
- Sean E. Cavanagh
- Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom
| | - Laurence T. Hunt
- Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom
- Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
- Max Planck-UCL Centre for Computational Psychiatry and Aging, University College London, London, United Kingdom
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, United Kingdom
| | - Steven W. Kennerley
- Department of Clinical and Movement Neurosciences, University College London, London, United Kingdom
| |
Collapse
|
20
|
Frontotemporal Regulation of Subjective Value to Suppress Impulsivity in Intertemporal Choices. J Neurosci 2020; 41:1727-1737. [PMID: 33334869 DOI: 10.1523/jneurosci.1196-20.2020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2020] [Revised: 11/06/2020] [Accepted: 11/12/2020] [Indexed: 11/21/2022] Open
Abstract
Impulsive decisions arise from preferring smaller but sooner rewards compared with larger but later rewards. How neural activity and attention to choice alternatives contribute to reward decisions during temporal discounting is not clear. Here we probed (1) attention to and (2) neural representation of delay and reward information in humans (both sexes) engaged in choices. We studied behavioral and frequency-specific dynamics supporting impulsive decisions on a fine-grained temporal scale using eye tracking and MEG recordings. In one condition, participants had to decide for themselves but pretended to decide for their best friend in a second prosocial condition, which required perspective taking. Hence, conditions varied in the value for themselves versus that pretending to choose for another person. Stronger impulsivity was reliably found across three independent groups for prosocial decisions. Eye tracking revealed a systematic shift of attention from the delay to the reward information and differences in eye tracking between conditions predicted differences in discounting. High-frequency activity (175-250 Hz) distributed over right frontotemporal sensors correlated with delay and reward information in consecutive temporal intervals for high value decisions for oneself but not the friend. Collectively, the results imply that the high-frequency activity recorded over frontotemporal MEG sensors plays a critical role in choice option integration.SIGNIFICANCE STATEMENT Humans face decisions between sooner smaller rewards and larger later rewards daily. An objective benefit of losing weight over a longer time might be devalued in face of ice cream because they prefer currently available options because of insufficiently considering long-term alternatives. The degree of contribution of neural representation and attention to choice alternatives is not clear. We investigated correlates of such decisions in participants deciding for themselves or pretending to choose for a friend. Behaviorally participants discounted less in self-choices compared with the prosocial condition. Eye movement and MEG recordings revealed how participants represent choice options most evident for options with high subjective value. These results advance our understanding of neural mechanisms underlying decision-making in humans.
Collapse
|
21
|
Yeon J, Shekhar M, Rahnev D. Overlapping and unique neural circuits are activated during perceptual decision making and confidence. Sci Rep 2020; 10:20761. [PMID: 33247212 PMCID: PMC7699640 DOI: 10.1038/s41598-020-77820-6] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Accepted: 11/16/2020] [Indexed: 12/02/2022] Open
Abstract
The period of making a perceptual decision is often followed by a period of rating confidence where one evaluates the likely accuracy of the initial decision. However, it remains unclear whether the same or different neural circuits are engaged during periods of perceptual decision making and confidence report. To address this question, we conducted two functional MRI experiments in which we dissociated the periods related to perceptual decision making and confidence report by either separating their respective regressors or asking for confidence ratings only in the second half of the experiment. We found that perceptual decision making and confidence reports gave rise to activations in large and mostly overlapping brain circuits including frontal, parietal, posterior, and cingulate regions with the results being remarkably consistent across the two experiments. Further, the confidence report period activated a number of unique regions, whereas only early sensory areas were activated for the decision period across the two experiments. We discuss the possible reasons for this overlap and explore their implications about theories of perceptual decision making and visual metacognition.
Collapse
Affiliation(s)
- Jiwon Yeon
- School of Psychology, Georgia Institute of Technology, 654 Cherry Str. NW, Atlanta, GA, 30332, USA.
| | - Medha Shekhar
- School of Psychology, Georgia Institute of Technology, 654 Cherry Str. NW, Atlanta, GA, 30332, USA
| | - Dobromir Rahnev
- School of Psychology, Georgia Institute of Technology, 654 Cherry Str. NW, Atlanta, GA, 30332, USA
| |
Collapse
|
22
|
Mennella R, Vilarem E, Grèzes J. Rapid approach-avoidance responses to emotional displays reflect value-based decisions: Neural evidence from an EEG study. Neuroimage 2020; 222:117253. [DOI: 10.1016/j.neuroimage.2020.117253] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2020] [Revised: 07/03/2020] [Accepted: 08/09/2020] [Indexed: 11/16/2022] Open
|
23
|
Iigaya K, Hauser TU, Kurth-Nelson Z, O’Doherty JP, Dayan P, Dolan RJ. The value of what's to come: Neural mechanisms coupling prediction error and the utility of anticipation. SCIENCE ADVANCES 2020; 6:eaba3828. [PMID: 32596456 PMCID: PMC7304967 DOI: 10.1126/sciadv.aba3828] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2019] [Accepted: 05/07/2020] [Indexed: 05/02/2023]
Abstract
Having something to look forward to is a keystone of well-being. Anticipation of future reward, such as an upcoming vacation, can often be more gratifying than the experience itself. Theories suggest the utility of anticipation underpins various behaviors, ranging from beneficial information-seeking to harmful addiction. However, how neural systems compute anticipatory utility remains unclear. We analyzed the brain activity of human participants as they performed a task involving choosing whether to receive information predictive of future pleasant outcomes. Using a computational model, we show three brain regions orchestrate anticipatory utility. Specifically, ventromedial prefrontal cortex tracks the value of anticipatory utility, dopaminergic midbrain correlates with information that enhances anticipation, while sustained hippocampal activity mediates a functional coupling between these regions. Our findings suggest a previously unidentified neural underpinning for anticipation's influence over decision-making and unify a range of phenomena associated with risk and time-delay preference.
Collapse
Affiliation(s)
- Kiyohito Iigaya
- Max-Planck UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London WC1B 5EH, UK
- Gatsby Computational Neuroscience Unit, University College London, 25 Howland Street, London W1T 4JG, UK
- Division of Humanities and Social Sciences, California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USA
| | - Tobias U. Hauser
- Max-Planck UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London WC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK
| | - Zeb Kurth-Nelson
- Max-Planck UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London WC1B 5EH, UK
- Deepmind, 6 Pancras Square, London N1C 4AG, UK
| | - John P. O’Doherty
- Division of Humanities and Social Sciences, California Institute of Technology, 1200 E California Blvd, Pasadena, CA 91125, USA
| | - Peter Dayan
- Max-Planck UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London WC1B 5EH, UK
- Gatsby Computational Neuroscience Unit, University College London, 25 Howland Street, London W1T 4JG, UK
- Max Planck Institute for Biological Cybernetics, 72076 Tubingen, Germany
- University of Tübingen, 72074 Tübingen, Germany
| | - Raymond J. Dolan
- Max-Planck UCL Centre for Computational Psychiatry and Ageing Research, 10-12 Russell Square, London WC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK
| |
Collapse
|
24
|
The lateral prefrontal cortex of primates encodes stimulus colors and their behavioral relevance during a match-to-sample task. Sci Rep 2020; 10:4216. [PMID: 32144331 PMCID: PMC7060344 DOI: 10.1038/s41598-020-61171-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2019] [Accepted: 02/19/2020] [Indexed: 11/09/2022] Open
Abstract
The lateral prefrontal cortex of primates (lPFC) plays a central role in complex cognitive behavior, in decision-making as well as in guiding top-down attention. However, how and where in lPFC such behaviorally relevant signals are computed is poorly understood. We analyzed neural recordings from chronic microelectrode arrays implanted in lPFC region 8Av/45 of two rhesus macaques. The animals performed a feature match-to-sample task requiring them to match both motion and color information in a test stimulus. This task allowed to separate the encoding of stimulus motion and color from their current behavioral relevance on a trial-by-trial basis. We found that upcoming motor behavior can be robustly predicted from lPFC activity. In addition, we show that 8Av/45 encodes the color of a visual stimulus, regardless of its behavioral relevance. Most notably, whether a color matches the searched-for color can be decoded independent of a trial's motor outcome and while subjects detect unique feature conjunctions of color and motion. Thus, macaque area 8Av/45 computes, among other task-relevant information, the behavioral relevance of visual color features. Such a signal is most critical for both the selection of responses as well as the deployment of top-down modulatory signals, like feature-based attention.
Collapse
|
25
|
Khalighinejad N, Bongioanni A, Verhagen L, Folloni D, Attali D, Aubry JF, Sallet J, Rushworth MFS. A Basal Forebrain-Cingulate Circuit in Macaques Decides It Is Time to Act. Neuron 2019; 105:370-384.e8. [PMID: 31813653 PMCID: PMC6975166 DOI: 10.1016/j.neuron.2019.10.030] [Citation(s) in RCA: 46] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 10/02/2019] [Accepted: 10/22/2019] [Indexed: 12/22/2022]
Abstract
The medial frontal cortex has been linked to voluntary action, but an explanation of why decisions to act emerge at particular points in time has been lacking. We show that, in macaques, decisions about whether and when to act are predicted by a set of features defining the animal’s current and past context; for example, respectively, cues indicating the current average rate of reward and recent previous voluntary action decisions. We show that activity in two brain areas—the anterior cingulate cortex and basal forebrain—tracks these contextual factors and mediates their effects on behavior in distinct ways. We use focused transcranial ultrasound to selectively and effectively stimulate deep in the brain, even as deep as the basal forebrain, and demonstrate that alteration of activity in the two areas changes decisions about when to act. Likelihood and timing of voluntary action in macaques can be partially predicted Recent experience and present context influence when voluntary action occurs A basal forebrain-cingulate circuit mediated effects of these factors on behavior Stimulation of this circuit by ultrasound changed decisions about when to act
Collapse
Affiliation(s)
- Nima Khalighinejad
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK.
| | - Alessandro Bongioanni
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK; Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen 6525 XZ, the Netherlands
| | - Davide Folloni
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK
| | - David Attali
- Physics for Medicine Paris, INSERM U1273, ESPCI Paris, CNRS FRE 2031, PSL Research University, Paris 75012, France; Pathophysiology of Psychiatric Disorders Laboratory, Inserm U1266, Institute of Psychiatry and Neuroscience of Paris, Paris Descartes University, Paris University, Paris 75014, France; Service Hospitalo-Universitaire, Sainte-Anne Hospital, UGH Paris Psychiatry and Neurosciences, Paris 75014, France
| | - Jean-Francois Aubry
- Physics for Medicine Paris, INSERM U1273, ESPCI Paris, CNRS FRE 2031, PSL Research University, Paris 75012, France
| | - Jerome Sallet
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford OX1 3SR, UK
| |
Collapse
|
26
|
Economic Decisions through Circuit Inhibition. Curr Biol 2019; 29:3814-3824.e5. [PMID: 31679936 DOI: 10.1016/j.cub.2019.09.027] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2019] [Revised: 09/04/2019] [Accepted: 09/11/2019] [Indexed: 11/21/2022]
Abstract
Economic choices between goods are thought to rely on the orbitofrontal cortex (OFC), but the decision mechanisms remain poorly understood. To shed light on this fundamental issue, we recorded from the OFC of monkeys choosing between two juices offered sequentially. An analysis of firing rates across time windows revealed the presence of different groups of neurons similar to those previously identified under simultaneous offers. This observation suggested that economic decisions in the two modalities are formed in the same neural circuit. We then examined several hypotheses on the decision mechanisms. OFC neurons encoded good identities and values in a juice-based representation (labeled lines). Contrary to previous assessments, our data argued against the idea that decisions rely on mutual inhibition at the level of offer values. In fact, we showed that previous arguments for mutual inhibition were confounded by differences in value ranges. Instead, decisions seemed to involve mechanisms of circuit inhibition, whereby each offer value indirectly inhibited neurons encoding the opposite choice outcome. Our results reconcile a variety of previous findings and provide a general account for the neuronal underpinnings of economic choices.
Collapse
|
27
|
Wallis JD. Reward. HANDBOOK OF CLINICAL NEUROLOGY 2019; 163:281-294. [PMID: 31590735 DOI: 10.1016/b978-0-12-804281-6.00015-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/28/2023]
Abstract
Neurons throughout frontal cortex show robust responses to rewards, but a challenge is determining the specific function served by these different reward signals. Most neuropsychiatric disorders involve dysfunction of circuits between frontal cortex and subcortical structures, such as the striatum. There are multiple frontostriatal loops, and different neuropsychiatric disorders involve different loops to greater or lesser extents. Understanding the role of reward in each of these different circuits is a necessary step in developing novel treatments for these disorders. This chapter summarizes the recent literature that has identified the role of reward in different subregions of the frontal cortex. Orbitofrontal cortex integrates information about multiple aspects of expected rewards in order to derive their value, which can then be used to decide between alternative potential rewards. Neurons in anterior cingulate cortex encode the difference between the expected reward and the actual outcome. This information is useful for learning, since it can ensure that behavior changes when the outcome was not anticipated. Reward also affects signals in lateral prefrontal cortex related to attention and response selection, ensuring that behaviors are optimally prioritized. Finally, the chapter discusses how reward signals contribute to social processing and autonomic control.
Collapse
Affiliation(s)
- Joni D Wallis
- Department of Psychology, University of California at Berkeley, Berkeley, CA, United States; Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, United States.
| |
Collapse
|
28
|
Grabenhorst F, Tsutsui KI, Kobayashi S, Schultz W. Primate prefrontal neurons signal economic risk derived from the statistics of recent reward experience. eLife 2019; 8:e44838. [PMID: 31343407 PMCID: PMC6658165 DOI: 10.7554/elife.44838] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2019] [Accepted: 07/12/2019] [Indexed: 01/28/2023] Open
Abstract
Risk derives from the variation of rewards and governs economic decisions, yet how the brain calculates risk from the frequency of experienced events, rather than from explicit risk-descriptive cues, remains unclear. Here, we investigated whether neurons in dorsolateral prefrontal cortex process risk derived from reward experience. Monkeys performed in a probabilistic choice task in which the statistical variance of experienced rewards evolved continually. During these choices, prefrontal neurons signaled the reward-variance associated with specific objects ('object risk') or actions ('action risk'). Crucially, risk was not derived from explicit, risk-descriptive cues but calculated internally from the variance of recently experienced rewards. Support-vector-machine decoding demonstrated accurate neuronal risk discrimination. Within trials, neuronal signals transitioned from experienced reward to risk (risk updating) and from risk to upcoming choice (choice computation). Thus, prefrontal neurons encode the statistical variance of recently experienced rewards, complying with formal decision variables of object risk and action risk.
Collapse
Affiliation(s)
- Fabian Grabenhorst
- Department of Physiology, Development and NeuroscienceUniversity of CambridgeCambridgeUnited Kingdom
| | - Ken-Ichiro Tsutsui
- Department of Physiology, Development and NeuroscienceUniversity of CambridgeCambridgeUnited Kingdom
| | - Shunsuke Kobayashi
- Department of Physiology, Development and NeuroscienceUniversity of CambridgeCambridgeUnited Kingdom
| | - Wolfram Schultz
- Department of Physiology, Development and NeuroscienceUniversity of CambridgeCambridgeUnited Kingdom
| |
Collapse
|
29
|
O'Neill M, Brown V. Special Issue: Neural basis of motivation and cognitive control. Behav Brain Res 2019; 355:1. [PMID: 30223945 DOI: 10.1016/j.bbr.2018.08.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
30
|
Computing Value from Quality and Quantity in Human Decision-Making. J Neurosci 2018; 39:163-176. [PMID: 30455186 PMCID: PMC6325261 DOI: 10.1523/jneurosci.0706-18.2018] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2018] [Revised: 09/20/2018] [Accepted: 09/26/2018] [Indexed: 12/04/2022] Open
Abstract
How organisms learn the value of single stimuli through experience is well described. In many decisions, however, value estimates are computed “on the fly” by combining multiple stimulus attributes. The neural basis of this computation is poorly understood. Here we explore a common scenario in which decision-makers must combine information about quality and quantity to determine the best option. Using fMRI, we examined the neural representation of quality, quantity, and their integration into an integrated subjective value signal in humans of both genders. We found that activity within inferior frontal gyrus (IFG) correlated with offer quality, while activity in the intraparietal sulcus (IPS) specifically correlated with offer quantity. Several brain regions, including the anterior cingulate cortex (ACC), were sensitive to an interaction of quality and quantity. However, the ACC was uniquely activated by quality, quantity, and their interaction, suggesting that this region provides a substrate for flexible computation of value from both quality and quantity. Furthermore, ACC signals across subjects correlated with the strength of quality and quantity signals in IFG and IPS, respectively. ACC tracking of subjective value also correlated with choice predictability. Finally, activity in the ACC was elevated for choice trials, suggesting that ACC provides a nexus for the computation of subjective value in multiattribute decision-making. SIGNIFICANCE STATEMENT Would you prefer three apples or two oranges? Many choices we make each day require us to weigh up the quality and quantity of different outcomes. Using fMRI, we show that option quality is selectively represented in the inferior frontal gyrus, while option quantity correlates with areas of the intraparietal sulcus that have previously been associated with numerical processing. We show that information about the two is integrated into a value signal in the anterior cingulate cortex, and the fidelity of this integration predicts choice predictability. Our results demonstrate how on-the-fly value estimates are computed from multiple attributes in human value-based decision-making.
Collapse
|
31
|
Hunt LT, Malalasekera WMN, de Berker AO, Miranda B, Farmer SF, Behrens TEJ, Kennerley SW. Triple dissociation of attention and decision computations across prefrontal cortex. Nat Neurosci 2018; 21:1471-1481. [PMID: 30258238 PMCID: PMC6331040 DOI: 10.1038/s41593-018-0239-5] [Citation(s) in RCA: 96] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2017] [Accepted: 08/11/2018] [Indexed: 01/06/2023]
Abstract
Naturalistic decision-making typically involves sequential deployment of attention to choice alternatives to gather information before a decision is made. Attention filters how information enters decision circuits, thus implying that attentional control may shape how decision computations unfold. We recorded neuronal activity from three subregions of the prefrontal cortex (PFC) while monkeys performed an attention-guided decision-making task. From the first saccade to decision-relevant information, a triple dissociation of decision- and attention-related computations emerged in parallel across PFC subregions. During subsequent saccades, orbitofrontal cortex activity reflected the value comparison between currently and previously attended information. In contrast, the anterior cingulate cortex carried several signals reflecting belief updating in light of newly attended information, the integration of evidence to a decision bound and an emerging plan for what action to choose. Our findings show how anatomically dissociable PFC representations evolve during attention-guided information search, supporting computations critical for value-guided choice.
Collapse
Affiliation(s)
- Laurence T Hunt
- Sobell Department of Motor Neuroscience, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
- Oxford Health NHS Foundation Trust, Oxford, UK.
| | | | - Archy O de Berker
- Sobell Department of Motor Neuroscience, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Bruno Miranda
- Sobell Department of Motor Neuroscience, University College London, London, UK
- International Neuroscience Doctoral Programme, Champalimaud Foundation, Lisbon, Portugal
- Instituto de Medicina Molecular, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal
| | - Simon F Farmer
- Sobell Department of Motor Neuroscience, University College London, London, UK
| | - Timothy E J Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Steven W Kennerley
- Sobell Department of Motor Neuroscience, University College London, London, UK.
| |
Collapse
|
32
|
Decoding Cognitive Processes from Neural Ensembles. Trends Cogn Sci 2018; 22:1091-1102. [PMID: 30279136 DOI: 10.1016/j.tics.2018.09.002] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2018] [Revised: 09/04/2018] [Accepted: 09/11/2018] [Indexed: 11/21/2022]
Abstract
An intrinsic difficulty in studying cognitive processes is that they are unobservable states that exist in between observable responses to the sensory environment. Cognitive states must be inferred from indirect behavioral measures. Neuroscience potentially provides the tools necessary to measure cognitive processes directly, but it is challenged on two fronts. First, neuroscientific measures often lack the spatiotemporal resolution to identify the neural computations that underlie a cognitive process. Second, the activity of a single neuron, which is the fundamental building block of neural computation, is too noisy to provide accurate measurements of a cognitive process. In this paper, I examine recent developments in neurophysiological recording and analysis methods that provide a potential solution to these problems.
Collapse
|
33
|
Saez I, Lin J, Stolk A, Chang E, Parvizi J, Schalk G, Knight RT, Hsu M. Encoding of Multiple Reward-Related Computations in Transient and Sustained High-Frequency Activity in Human OFC. Curr Biol 2018; 28:2889-2899.e3. [PMID: 30220499 DOI: 10.1016/j.cub.2018.07.045] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Revised: 06/22/2018] [Accepted: 07/16/2018] [Indexed: 11/26/2022]
Abstract
Human orbitofrontal cortex (OFC) has long been implicated in value-based decision making. In recent years, convergent evidence from human and model organisms has further elucidated its role in representing reward-related computations underlying decision making. However, a detailed description of these processes remains elusive due in part to (1) limitations in our ability to observe human OFC neural dynamics at the timescale of decision processes and (2) methodological and interspecies differences that make it challenging to connect human and animal findings or to resolve discrepancies when they arise. Here, we sought to address these challenges by conducting multi-electrode electrocorticography (ECoG) recordings in neurosurgical patients during economic decision making to elucidate the electrophysiological signature, sub-second temporal profile, and anatomical distribution of reward-related computations within human OFC. We found that high-frequency activity (HFA) (70-200 Hz) reflected multiple valuation components grouped in two classes of valuation signals that were dissociable in temporal profile and information content: (1) fast, transient responses reflecting signals associated with choice and outcome processing, including anticipated risk and outcome regret, and (2) sustained responses explicitly encoding what happened in the immediately preceding trial. Anatomically, these responses were widely distributed in partially overlapping networks, including regions in the central OFC (Brodmann areas 11 and 13), which have been consistently implicated in reward processing in animal single-unit studies. Together, these results integrate insights drawn from human and animal studies and provide evidence for a role of human OFC in representing multiple reward computations.
Collapse
Affiliation(s)
- Ignacio Saez
- University of California, Berkeley, Berkeley, CA 94720, USA
| | - Jack Lin
- University of California, Irvine, Irvine, CA 92697, USA
| | - Arjen Stolk
- University of California, Berkeley, Berkeley, CA 94720, USA
| | - Edward Chang
- University of California, San Francisco, San Francisco, CA 94143, USA
| | | | - Gerwin Schalk
- Wadsworth Center, New York State Department of Health, Albany, NY 12201, USA
| | - Robert T Knight
- University of California, Berkeley, Berkeley, CA 94720, USA.
| | - Ming Hsu
- University of California, Berkeley, Berkeley, CA 94720, USA.
| |
Collapse
|
34
|
Chiang FK, Wallis JD. Neuronal Encoding in Prefrontal Cortex during Hierarchical Reinforcement Learning. J Cogn Neurosci 2018; 30:1197-1208. [PMID: 29694261 DOI: 10.1162/jocn_a_01272] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Reinforcement learning models have proven highly effective for understanding learning in both artificial and biological systems. However, these models have difficulty in scaling up to the complexity of real-life environments. One solution is to incorporate the hierarchical structure of behavior. In hierarchical reinforcement learning, primitive actions are chunked together into more temporally abstract actions, called "options," that are reinforced by attaining a subgoal. These subgoals are capable of generating pseudoreward prediction errors, which are distinct from reward prediction errors that are associated with the final goal of the behavior. Studies in humans have shown that pseudoreward prediction errors positively correlate with activation of ACC. To determine how pseudoreward prediction errors are encoded at the single neuron level, we trained two animals to perform a primate version of the task used to generate these errors in humans. We recorded the electrical activity of neurons in ACC during performance of this task, as well as neurons in lateral prefrontal cortex and OFC. We found that the firing rate of a small population of neurons encoded pseudoreward prediction errors, and these neurons were restricted to ACC. Our results provide support for the idea that ACC may play an important role in encoding subgoals and pseudoreward prediction errors to support hierarchical reinforcement learning. One caveat is that neurons encoding pseudoreward prediction errors were relatively few in number, especially in comparison to neurons that encoded information about the main goal of the task.
Collapse
|
35
|
Blanchard TC, Piantadosi ST, Hayden BY. Robust mixture modeling reveals category-free selectivity in reward region neuronal ensembles. J Neurophysiol 2018; 119:1305-1318. [PMID: 29212924 PMCID: PMC5966738 DOI: 10.1152/jn.00808.2017] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2017] [Revised: 11/30/2017] [Accepted: 12/01/2017] [Indexed: 12/29/2022] Open
Abstract
Classification of neurons into clusters based on their response properties is an important tool for gaining insight into neural computations. However, it remains unclear to what extent neurons fall naturally into discrete functional categories. We developed a Bayesian method that models the tuning properties of neural populations as a mixture of multiple types of task-relevant response patterns. We applied this method to data from several cortical and striatal regions in economic choice tasks. In all cases, neurons fell into only two clusters: one multiple-selectivity cluster containing all cells driven by task variables of interest and another of no selectivity for those variables. The single cluster of task-sensitive cells argues against robust categorical tuning in these areas. The no-selectivity cluster was unanticipated and raises important questions about what distinguishes these neurons and what role they play. Moreover, the ability to formally identify these nonselective cells allows for more accurate measurement of ensemble effects by excluding or appropriately down-weighting them in analysis. Our findings provide a valuable tool for analysis of neural data, challenge simple categorization schemes previously proposed for these regions, and place useful constraints on neurocomputational models of economic choice and control. NEW & NOTEWORTHY We present a Bayesian method for formally detecting whether a population of neurons can be naturally classified into clusters based on their response tuning properties. We then examine several data sets of reward system neurons for variables and find in all cases that neurons can be classified into only two categories: a functional class and a non-task-driven class. These results provide important constraints for neural models of the reward system.
Collapse
Affiliation(s)
- Tommy C Blanchard
- Department of Brain and Cognitive Sciences, Center for Visual Science, and Center for the Origins of Cognition, University of Rochester , Rochester, New York
| | - Steven T Piantadosi
- Department of Brain and Cognitive Sciences, Center for Visual Science, and Center for the Origins of Cognition, University of Rochester , Rochester, New York
| | - Benjamin Y Hayden
- Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota , Minneapolis, Minnesota
| |
Collapse
|
36
|
Rich EL, Stoll FM, Rudebeck PH. Linking dynamic patterns of neural activity in orbitofrontal cortex with decision making. Curr Opin Neurobiol 2018; 49:24-32. [PMID: 29169086 PMCID: PMC5889957 DOI: 10.1016/j.conb.2017.11.002] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2017] [Revised: 10/30/2017] [Accepted: 11/04/2017] [Indexed: 01/12/2023]
Abstract
Humans and animals demonstrate extraordinary flexibility in choice behavior, particularly when deciding based on subjective preferences. We evaluate options on different scales, deliberate, and often change our minds. Little is known about the neural mechanisms that underlie these dynamic aspects of decision-making, although neural activity in orbitofrontal cortex (OFC) likely plays a central role. Recent evidence from studies in macaques shows that attention modulates value responses in OFC, and that ensembles of OFC neurons dynamically signal different options during choices. When contexts change, these ensembles flexibly remap to encode the new task. Determining how these dynamic patterns emerge and relate to choices will inform models of decision-making and OFC function.
Collapse
Affiliation(s)
- Erin L Rich
- Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10014, USA.
| | - Frederic M Stoll
- Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10014, USA
| | - Peter H Rudebeck
- Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY 10014, USA.
| |
Collapse
|
37
|
Balasubramani PP, Moreno-Bote R, Hayden BY. Using a Simple Neural Network to Delineate Some Principles of Distributed Economic Choice. Front Comput Neurosci 2018; 12:22. [PMID: 29643773 PMCID: PMC5882864 DOI: 10.3389/fncom.2018.00022] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2017] [Accepted: 03/12/2018] [Indexed: 01/03/2023] Open
Abstract
The brain uses a mixture of distributed and modular organization to perform computations and generate appropriate actions. While the principles under which the brain might perform computations using modular systems have been more amenable to modeling, the principles by which the brain might make choices using distributed principles have not been explored. Our goal in this perspective is to delineate some of those distributed principles using a neural network method and use its results as a lens through which to reconsider some previously published neurophysiological data. To allow for direct comparison with our own data, we trained the neural network to perform binary risky choices. We find that value correlates are ubiquitous and are always accompanied by non-value information, including spatial information (i.e., no pure value signals). Evaluation, comparison, and selection were not distinct processes; indeed, value signals even in the earliest stages contributed directly, albeit weakly, to action selection. There was no place, other than at the level of action selection, at which dimensions were fully integrated. No units were specialized for specific offers; rather, all units encoded the values of both offers in an anti-correlated format, thus contributing to comparison. Individual network layers corresponded to stages in a continuous rotation from input to output space rather than to functionally distinct modules. While our network is likely to not be a direct reflection of brain processes, we propose that these principles should serve as hypotheses to be tested and evaluated for future studies.
Collapse
Affiliation(s)
- Pragathi P. Balasubramani
- Brain and Cognitive Sciences, Center for Visual Science, Center for the Origins of Cognition, University of Rochester, Rochester, NY, United States
| | - Rubén Moreno-Bote
- Department of Information and Communications Technologies, Center for Brain and Cognition, University Pompeu Fabra, Barcelona, Spain
- Serra Húnter Fellow Programme, University Pompeu Fabra, Barcelona, Spain
| | - Benjamin Y. Hayden
- Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minnesota, MN, United States
| |
Collapse
|
38
|
Stolyarova A. Solving the Credit Assignment Problem With the Prefrontal Cortex. Front Neurosci 2018; 12:182. [PMID: 29636659 PMCID: PMC5881225 DOI: 10.3389/fnins.2018.00182] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2017] [Accepted: 03/06/2018] [Indexed: 12/13/2022] Open
Abstract
In naturalistic multi-cue and multi-step learning tasks, where outcomes of behavior are delayed in time, discovering which choices are responsible for rewards can present a challenge, known as the credit assignment problem. In this review, I summarize recent work that highlighted a critical role for the prefrontal cortex (PFC) in assigning credit where it is due in tasks where only a few of the multitude of cues or choices are relevant to the final outcome of behavior. Collectively, these investigations have provided compelling support for specialized roles of the orbitofrontal (OFC), anterior cingulate (ACC), and dorsolateral prefrontal (dlPFC) cortices in contingent learning. However, recent work has similarly revealed shared contributions and emphasized rich and heterogeneous response properties of neurons in these brain regions. Such functional overlap is not surprising given the complexity of reciprocal projections spanning the PFC. In the concluding section, I overview the evidence suggesting that the OFC, ACC and dlPFC communicate extensively, sharing the information about presented options, executed decisions and received rewards, which enables them to assign credit for outcomes to choices on which they are contingent. This account suggests that lesion or inactivation/inhibition experiments targeting a localized PFC subregion will be insufficient to gain a fine-grained understanding of credit assignment during learning and instead poses refined questions for future research, shifting the focus from focal manipulations to experimental techniques targeting cortico-cortical projections.
Collapse
Affiliation(s)
- Alexandra Stolyarova
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, United States
| |
Collapse
|
39
|
Hayden BY, Moreno-Bote R. A neuronal theory of sequential economic choice. Brain Neurosci Adv 2018; 2:2398212818766675. [PMID: 32166137 PMCID: PMC7058205 DOI: 10.1177/2398212818766675] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2018] [Accepted: 02/27/2018] [Indexed: 11/16/2022] Open
Abstract
Results of recent studies point towards a new framework for the neural bases of economic choice. The principles of this framework include the idea that evaluation is limited to a single option within the focus of attention and that we accept or reject that option relative to the entire set of alternatives. Rejection leads attention to a new option, although it can later switch back to a previously rejected one. The option to which a neuron's firing rate refers is determined dynamically by attention and not stably by labelled lines. Value is always computed relative to the value of rejection. Comparison results not from explicit competition between discrete populations of neurons, but indirectly, as in a horse race, from the fact that the first option whose value crosses a threshold is selected. Consequently, comparison can occur within a single pool of neurons rather than by competition between two or more neuronal populations. The computations that constitute comparison thus occur at multiple levels, including premotor levels, simultaneously (i.e. the brain uses a distributed consensus), and not in discrete stages. This framework suggests a solution to a set of otherwise unresolved neuronal binding problems that result from the need to link options to values, comparisons to actions, and choices to outcomes.
Collapse
Affiliation(s)
- Benjamin Y. Hayden
- Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA
| | - Rubén Moreno-Bote
- Department of Information and Communications Technologies, Pompeu Fabra University, Barcelona, Spain
- Center for Brain and Cognition, Pompeu Fabra University, Barcelona, Spain
- Serra Húnter Fellow Programme, Pompeu Fabra University, Barcelona, Spain
| |
Collapse
|
40
|
Meder D, Kolling N, Verhagen L, Wittmann MK, Scholl J, Madsen KH, Hulme OJ, Behrens TEJ, Rushworth MFS. Simultaneous representation of a spectrum of dynamically changing value estimates during decision making. Nat Commun 2017; 8:1942. [PMID: 29208968 PMCID: PMC5717172 DOI: 10.1038/s41467-017-02169-w] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2017] [Accepted: 11/07/2017] [Indexed: 01/26/2023] Open
Abstract
Decisions are based on value expectations derived from experience. We show that dorsal anterior cingulate cortex and three other brain regions hold multiple representations of choice value based on different timescales of experience organized in terms of systematic gradients across the cortex. Some parts of each area represent value estimates based on recent reward experience while others represent value estimates based on experience over the longer term. The value estimates within these areas interact with one another according to their temporal scaling. Some aspects of the representations change dynamically as the environment changes. The spectrum of value estimates may act as a flexible selection mechanism for combining experience-derived value information with other aspects of value to allow flexible and adaptive decisions in changing environments.
Collapse
Affiliation(s)
- David Meder
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK. .,Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark.
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK
| | - Marco K Wittmann
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Jacqueline Scholl
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK
| | - Kristoffer H Madsen
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark
| | - Oliver J Hulme
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark
| | - Timothy E J Behrens
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| |
Collapse
|
41
|
Azab H, Hayden BY. Correlates of decisional dynamics in the dorsal anterior cingulate cortex. PLoS Biol 2017; 15:e2003091. [PMID: 29141002 PMCID: PMC5706721 DOI: 10.1371/journal.pbio.2003091] [Citation(s) in RCA: 38] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2017] [Revised: 11/29/2017] [Accepted: 10/27/2017] [Indexed: 02/01/2023] Open
Abstract
We hypothesized that during binary economic choice, decision makers use the first option they attend as a default to which they compare the second. To test this idea, we recorded activity of neurons in the dorsal anterior cingulate cortex (dACC) of macaques choosing between gambles presented asynchronously. We find that ensemble encoding of the value of the first offer includes both choice-dependent and choice-independent aspects, as if reflecting a partial decision. That is, its responses are neither entirely pre- nor post-decisional. In contrast, coding of the value of the second offer is entirely decision dependent (i.e., post-decisional). This result holds even when offer-value encodings are compared within the same time period. Additionally, we see no evidence for 2 pools of neurons linked to the 2 offers; instead, all comparison appears to occur within a single functionally homogenous pool of task-selective neurons. These observations suggest that economic choices reflect a context-dependent evaluation of attended options. Moreover, they raise the possibility that value representations reflect, to some extent, a tentative commitment to a choice.
Collapse
Affiliation(s)
- Habiba Azab
- Department of Brain and Cognitive Sciences and Center for Visual Sciences, University of Rochester, Rochester, New York, United States of America
- * E-mail:
| | - Benjamin Y. Hayden
- Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, Minnesota, United States of America
| |
Collapse
|
42
|
Lăzăroiu G, Pera A, Ștefănescu-Mihăilă RO, Mircică N, Negurită O. Can Neuroscience Assist Us in Constructing Better Patterns of Economic Decision-Making? Front Behav Neurosci 2017; 11:188. [PMID: 29066963 PMCID: PMC5641305 DOI: 10.3389/fnbeh.2017.00188] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2017] [Accepted: 09/26/2017] [Indexed: 12/22/2022] Open
Abstract
We draw on outstanding research (Sanfey et al., 2006; McCabe, 2008; Bernheim, 2009; Camerer, 2013; Radu and McClure, 2013; Declerck and Boone, 2016) to substantiate that neuroeconomics covers the investigation of the biological microfoundations of economic cognition and economic conduct, attempts to prove that a superior grasp of how choices are made brings about superior expectations regarding which options are selected, preserves the strictness of economic analysis in defining value-based decision, and associates imaging techniques with economic pattern to explain how individuals decide on a strategy taking into account various possible choices. Neuroeconomics is adequately prepared to regulate the notion of how choices are determined by mental states. The position that will be elaborated in this article is that neuroeconomic patterns are enabled and enhanced in descriptive capacity by psychological outcomes and substantiated in biological processes. Advancement in neuroeconomics takes place when outcomes from distinct procedures are coherent with an ordinary mechanistic clarification of what generates choice, construed by a computational pattern. We will develop this point further by proving that economics improves the concerted effort of neuroeconomics by using its observations in the various results that may stem from the planned and market interplays of diverse participants, and via a series of accurate, explicit, mathematical patterns to construe such interplays and results. Neuroeconomics experiments employ a mixture of brain imaging/stimulation tests advanced in the cognitive neurosciences and microeconomic systems/game theory tests advanced in the economic sciences. Our analyses indicate that neuroeconomics aims to employ the supplementary input gained from brain investigations, associated with the decision maker's selection, with the purpose of better grasping the cogitation process and to utilize the outcomes to enhance economic patterns.
Collapse
Affiliation(s)
- George Lăzăroiu
- Department of Social-Human Sciences, Spiru Haret University, Bucharest, Romania
| | - Aurel Pera
- Department of Teacher Training, University of Craiova, Craiova, Romania
| | | | - Nela Mircică
- Department of Social-Human Sciences, Spiru Haret University, Bucharest, Romania
| | - Octav Negurită
- Department of Economic Sciences, Spiru Haret University, Constanta, Romania
| |
Collapse
|
43
|
Abstract
Research published in this issue of Neuron from McGinty et al. (2016) suggests that attention may help bind information about value to specific options in economic choice. Responses of orbitofrontal neurons are strongly modulated by the distance from gaze to the position of a reward-predicting target.
Collapse
Affiliation(s)
- Rei Akaishi
- Department of Brain and Cognitive Sciences and Center for Visual Science, University of Rochester, Rochester, NY 14627, USA.
| | - Benjamin Y Hayden
- Department of Brain and Cognitive Sciences and Center for Visual Science, University of Rochester, Rochester, NY 14627, USA
| |
Collapse
|
44
|
Contrasting Effects of Medial and Lateral Orbitofrontal Cortex Lesions on Credit Assignment and Decision-Making in Humans. J Neurosci 2017. [PMID: 28630257 DOI: 10.1523/jneurosci.0692-17.2017] [Citation(s) in RCA: 82] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023] Open
Abstract
The orbitofrontal cortex is critical for goal-directed behavior. Recent work in macaques has suggested the lateral orbitofrontal cortex (lOFC) is relatively more concerned with assignment of credit for rewards to particular choices during value-guided learning, whereas the medial orbitofrontal cortex (often referred to as ventromedial prefrontal cortex in humans; vmPFC/mOFC) is involved in constraining the decision to the relevant options. We examined whether people with damage restricted to subregions of prefrontal cortex showed the patterns of impairment observed in prior investigations of the effects of lesions to homologous regions in macaques. Groups of patients with either lOFC (predominantly right hemisphere), mOFC/vmPFC, or dorsomedial prefrontal (DMF), and a comparison group of healthy age- and education-matched controls performed a probabilistic 3-choice decision-making task. We report anatomically specific patterns of impairment. We found that credit assignment, as indexed by the normal influence of contingent relationships between choice and reward, is reduced in lOFC patients compared with Controls and mOFC/vmPFC patients. Moreover, the effects of reward contingency on choice were similar for patients with lesions in DMF or mOFC/vmPFC, compared with Controls. By contrast, mOFC/vmPFC-lesioned patients made more stochastic choices than Controls when the decision was framed by valuable distracting alternatives, suggesting that value comparisons were no longer independent of irrelevant options. Once again, there was evidence of regional specialization: patients with lOFC lesions were unimpaired relative to Controls. As in macaques, human lOFC and mOFC/vmPFC are necessary for contingent learning and value-guided decision-making, respectively.SIGNIFICANCE STATEMENT The lateral and medial regions of the orbitofrontal cortex are cytoarchitectonically distinct and have different anatomical connections. Previous investigations in macaques have shown these anatomical differences are accompanied by functional specialization for learning and decision-making. Here, for the first time, we test the predictions made by macaque studies in an experiment with humans with frontal lobe lesions, asking whether behavioral impairments can be linked to lateral or medial orbitofrontal cortex. Using equivalent tasks and computational analyses, our findings broadly replicate the pattern reported after selective lesions in monkeys. Patients with lateral orbitofrontal damage had impaired credit assignment, whereas damage to medial orbitofrontal cortex meant that patients were more likely to be distracted by irrelevant options.
Collapse
|
45
|
Hunt LT, Hayden BY. A distributed, hierarchical and recurrent framework for reward-based choice. Nat Rev Neurosci 2017; 18:172-182. [PMID: 28209978 PMCID: PMC5621622 DOI: 10.1038/nrn.2017.7] [Citation(s) in RCA: 137] [Impact Index Per Article: 19.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.
Collapse
Affiliation(s)
- Laurence T Hunt
- Wellcome Trust Centre for Neuroimaging, University College London, 12 Queen Square, London WC1N 3BG, UK
| | - Benjamin Y Hayden
- Department of Brain and Cognitive Sciences, University of Rochester, 309 Meliora Hall, Rochester, New York 14618, USA
| |
Collapse
|
46
|
Pirrone A, Azab H, Hayden BY, Stafford T, Marshall JAR. Evidence for the speed-value trade-off: human and monkey decision making is magnitude sensitive. ACTA ACUST UNITED AC 2017; 5:129-142. [PMID: 29682592 DOI: 10.1037/dec0000075] [Citation(s) in RCA: 54] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Complex natural systems from brains to bee swarms have evolved to make adaptive multifactorial decisions. Recent theoretical and empirical work suggests that many evolved systems may take advantage of common motifs across multiple domains. We are particularly interested in value sensitivity (i.e., sensitivity to the magnitude or intensity of the stimuli or reward under consideration) as a mechanism to resolve deadlocks adaptively. This mechanism favours long-term reward maximization over accuracy in a simple manner, because it avoids costly delays associated with ambivalence between similar options; speed-value trade-offs have been proposed to be evolutionarily advantageous for many kinds of decision. A key prediction of the value-sensitivity hypothesis is that choices between equally-valued options will proceed faster when the options have a high value than when they have a low value. However, value-sensitivity is not part of idealised choice models such as diffusion to bound. Here we examine two different choice behaviours in two different species, perceptual decisions in humans and economic choices in rhesus monkeys, to test this hypothesis. We observe the same value sensitivity in both human perceptual decisions and monkey value-based decisions. These results endorse the idea that neural decision systems make use of the same basic principle of value-sensitivity in order to resolve costly deadlocks and thus improve long-term reward intake.
Collapse
Affiliation(s)
- Angelo Pirrone
- Department of Psychology & Department of Computer Science, The University of Sheffield, UK
| | - Habiba Azab
- Department of Brain and Cognitive Sciences and Center for Visual Science, University of Rochester, USA
| | - Benjamin Y Hayden
- Department of Brain and Cognitive Sciences and Center for Visual Science, University of Rochester, USA
| | - Tom Stafford
- Department of Psychology, The University of Sheffield, UK
| | | |
Collapse
|
47
|
Leathers ML, Olson CR. In monkeys making value-based decisions, amygdala neurons are sensitive to cue value as distinct from cue salience. J Neurophysiol 2017; 117:1499-1511. [PMID: 28077664 DOI: 10.1152/jn.00564.2016] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2016] [Revised: 01/05/2017] [Accepted: 01/05/2017] [Indexed: 11/22/2022] Open
Abstract
Neurons in the lateral intraparietal (LIP) area of macaque monkey parietal cortex respond to cues predicting rewards and penalties of variable size in a manner that depends on the motivational salience of the predicted outcome (strong for both large reward and large penalty) rather than on its value (positive for large reward and negative for large penalty). This finding suggests that LIP mediates the capture of attention by salient events and does not encode value in the service of value-based decision making. It leaves open the question whether neurons elsewhere in the brain encode value in the identical task. To resolve this issue, we recorded neuronal activity in the amygdala in the context of the task employed in the LIP study. We found that responses to reward-predicting cues were similar between areas, with the majority of reward-sensitive neurons responding more strongly to cues that predicted large reward than to those that predicted small reward. Responses to penalty-predicting cues were, however, markedly different. In the amygdala, unlike LIP, few neurons were sensitive to penalty size, few penalty-sensitive neurons favored large over small penalty, and the dependence of firing rate on penalty size was negatively correlated with its dependence on reward size. These results indicate that amygdala neurons encoded cue value under circumstances in which LIP neurons exhibited sensitivity to motivational salience. However, the representation of negative value, as reflected in sensitivity to penalty size, was weaker than the representation of positive value, as reflected in sensitivity to reward size.NEW & NOTEWORTHY This is the first study to characterize amygdala neuronal responses to cues predicting rewards and penalties of variable size in monkeys making value-based choices. Manipulating reward and penalty size allowed distinguishing activity dependent on motivational salience from activity dependent on value. This approach revealed in a previous study that neurons of the lateral intraparietal (LIP) area encode motivational salience. Here, it reveals that amygdala neurons encode value. The results establish a sharp functional distinction between the two areas.
Collapse
Affiliation(s)
- Marvin L Leathers
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania; and .,Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Carl R Olson
- Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania; and.,Department of Neuroscience, University of Pittsburgh, Pittsburgh, Pennsylvania
| |
Collapse
|
48
|
Scholl J, Kolling N, Nelissen N, Stagg CJ, Harmer CJ, Rushworth MFS. Excitation and inhibition in anterior cingulate predict use of past experiences. eLife 2017; 6:e20365. [PMID: 28055824 PMCID: PMC5213710 DOI: 10.7554/elife.20365] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2016] [Accepted: 11/29/2016] [Indexed: 01/17/2023] Open
Abstract
Dorsal anterior cingulate cortex (dACC) mediates updating and maintenance of cognitive models of the world used to drive adaptive reward-guided behavior. We investigated the neurochemical underpinnings of this process. We used magnetic resonance spectroscopy in humans, to measure levels of glutamate and GABA in dACC. We examined their relationship to neural signals in dACC, measured with fMRI, and cognitive task performance. Both inhibitory and excitatory neurotransmitters in dACC were predictive of the strength of neural signals in dACC and behavioral adaptation. Glutamate levels were correlated, first, with stronger neural activity representing information to be learnt about the tasks' costs and benefits and, second, greater use of this information in the guidance of behavior. GABA levels were negatively correlated with the same neural signals and the same indices of behavioral influence. Our results suggest that glutamate and GABA in dACC affect the encoding and use of past experiences to guide behavior.
Collapse
Affiliation(s)
- Jacqueline Scholl
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Natalie Nelissen
- Oxford Centre for Human Brain Activity, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | - Charlotte J Stagg
- Oxford Centre for Human Brain Activity, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | | | - Matthew FS Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Oxford Centre for Functional MRI of the Brain, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
49
|
Cavanagh SE, Wallis JD, Kennerley SW, Hunt LT. Autocorrelation structure at rest predicts value correlates of single neurons during reward-guided choice. eLife 2016; 5. [PMID: 27705742 PMCID: PMC5052031 DOI: 10.7554/elife.18937] [Citation(s) in RCA: 61] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2016] [Accepted: 09/15/2016] [Indexed: 01/28/2023] Open
Abstract
Correlates of value are routinely observed in the prefrontal cortex (PFC) during reward-guided decision making. In previous work (Hunt et al., 2015), we argued that PFC correlates of chosen value are a consequence of varying rates of a dynamical evidence accumulation process. Yet within PFC, there is substantial variability in chosen value correlates across individual neurons. Here we show that this variability is explained by neurons having different temporal receptive fields of integration, indexed by examining neuronal spike rate autocorrelation structure whilst at rest. We find that neurons with protracted resting temporal receptive fields exhibit stronger chosen value correlates during choice. Within orbitofrontal cortex, these neurons also sustain coding of chosen value from choice through the delivery of reward, providing a potential neural mechanism for maintaining predictions and updating stored values during learning. These findings reveal that within PFC, variability in temporal specialisation across neurons predicts involvement in specific decision-making computations. DOI:http://dx.doi.org/10.7554/eLife.18937.001
Collapse
Affiliation(s)
- Sean E Cavanagh
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom
| | - Joni D Wallis
- Department of Psychology, University of California, Berkeley, Berkeley, United States.,Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
| | - Steven W Kennerley
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom.,Department of Psychology, University of California, Berkeley, Berkeley, United States.,Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, United States
| | - Laurence T Hunt
- Sobell Department of Motor Neuroscience, University College London, London, United Kingdom.,Wellcome Trust Centre for Neuroimaging, University College London, London, United Kingdom
| |
Collapse
|
50
|
Neural Signatures of Value Comparison in Human Cingulate Cortex during Decisions Requiring an Effort-Reward Trade-off. J Neurosci 2016; 36:10002-15. [PMID: 27683898 DOI: 10.1523/jneurosci.0292-16.2016] [Citation(s) in RCA: 132] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2016] [Accepted: 07/14/2016] [Indexed: 11/21/2022] Open
Abstract
UNLABELLED Integrating costs and benefits is crucial for optimal decision-making. Although much is known about decisions that involve outcome-related costs (e.g., delay, risk), many of our choices are attached to actions and require an evaluation of the associated motor costs. Yet how the brain incorporates motor costs into choices remains largely unclear. We used human fMRI during choices involving monetary reward and physical effort to identify brain regions that serve as a choice comparator for effort-reward trade-offs. By independently varying both options' effort and reward levels, we were able to identify the neural signature of a comparator mechanism. A network involving supplementary motor area and the caudal portion of dorsal anterior cingulate cortex encoded the difference in reward (positively) and effort levels (negatively) between chosen and unchosen choice options. We next modeled effort-discounted subjective values using a novel behavioral model. This revealed that the same network of regions involving dorsal anterior cingulate cortex and supplementary motor area encoded the difference between the chosen and unchosen options' subjective values, and that activity was best described using a concave model of effort-discounting. In addition, this signal reflected how precisely value determined participants' choices. By contrast, separate signals in supplementary motor area and ventromedial prefrontal cortex correlated with participants' tendency to avoid effort and seek reward, respectively. This suggests that the critical neural signature of decision-making for choices involving motor costs is found in human cingulate cortex and not ventromedial prefrontal cortex as typically reported for outcome-based choice. Furthermore, distinct frontal circuits seem to drive behavior toward reward maximization and effort minimization. SIGNIFICANCE STATEMENT The neural processes that govern the trade-off between expected benefits and motor costs remain largely unknown. This is striking because energetic requirements play an integral role in our day-to-day choices and instrumental behavior, and a diminished willingness to exert effort is a characteristic feature of a range of neurological disorders. We use a new behavioral characterization of how humans trade off reward maximization with effort minimization to examine the neural signatures that underpin such choices, using BOLD MRI neuroimaging data. We find the critical neural signature of decision-making, a signal that reflects the comparison of value between choice options, in human cingulate cortex, whereas two distinct brain circuits drive behavior toward reward maximization or effort minimization.
Collapse
|