1
|
Holton E, Grohn J, Ward H, Manohar SG, O'Reilly JX, Kolling N. Goal commitment is supported by vmPFC through selective attention. Nat Hum Behav 2024:10.1038/s41562-024-01844-5. [PMID: 38632389 DOI: 10.1038/s41562-024-01844-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2023] [Accepted: 02/01/2024] [Indexed: 04/19/2024]
Abstract
When striking a balance between commitment to a goal and flexibility in the face of better options, people often demonstrate strong goal perseveration. Here, using functional MRI (n = 30) and lesion patient (n = 26) studies, we argue that the ventromedial prefrontal cortex (vmPFC) drives goal commitment linked to changes in goal-directed selective attention. Participants performed an incremental goal pursuit task involving sequential decisions between persisting with a goal versus abandoning progress for better alternative options. Individuals with stronger goal perseveration showed higher goal-directed attention in an interleaved attention task. Increasing goal-directed attention also affected abandonment decisions: while pursuing a goal, people lost their sensitivity to valuable alternative goals while remaining more sensitive to changes in the current goal. In a healthy population, individual differences in both commitment biases and goal-oriented attention were predicted by baseline goal-related activity in the vmPFC. Among lesion patients, vmPFC damage reduced goal commitment, leading to a performance benefit.
Collapse
Affiliation(s)
- Eleanor Holton
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Jan Grohn
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
| | - Harry Ward
- Centre for Experimental Medicine and Rheumatology, Queen Mary University London (QMUL), London, UK
| | - Sanjay G Manohar
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Jill X O'Reilly
- Department of Experimental Psychology, University of Oxford, Oxford, UK
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging (WIN), University of Oxford, Oxford, UK
- Stem Cell and Brain Research Institute U1208, Inserm, Université Claude Bernard Lyon 1, Bron, France
| |
Collapse
|
2
|
Law CK, Kolling N, Chan CCH, Chau BKH. Frontopolar cortex represents complex features and decision value during choice between environments. Cell Rep 2023; 42:112555. [PMID: 37224014 PMCID: PMC10320831 DOI: 10.1016/j.celrep.2023.112555] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Revised: 11/23/2022] [Accepted: 05/05/2023] [Indexed: 05/26/2023] Open
Abstract
Important decisions often involve choosing between complex environments that define future item encounters. Despite its importance for adaptive behavior and distinct computational challenges, decision-making research primarily focuses on item choice, ignoring environment choice altogether. Here we contrast previously studied item choice in ventromedial prefrontal cortex with lateral frontopolar cortex (FPl) linked to environment choice. Furthermore, we propose a mechanism for how FPl decomposes and represents complex environments during decision making. Specifically, we trained a choice-optimized, brain-naive convolutional neural network (CNN) and compared predicted CNN activation with actual FPl activity. We showed that the high-dimensional FPl activity decomposes environment features to represent the complexity of an environment to make such choice possible. Moreover, FPl functionally connects with posterior cingulate cortex for guiding environment choice. Further probing FPl's computation revealed a parallel processing mechanism in extracting multiple environment features.
Collapse
Affiliation(s)
- Chun-Kit Law
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong.
| | - Nils Kolling
- Université Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, 18 Avenue Doyen Lepine, 69500 Bron, France; Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK
| | - Chetwyn C H Chan
- Department of Psychology, The Education University of Hong Kong, Hong Kong
| | - Bolton K H Chau
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong; University Research Facility in Behavioral and Systems Neuroscience, The Hong Kong Polytechnic University, Hong Kong.
| |
Collapse
|
3
|
Jahn CI, Grohn J, Cuell S, Emberton A, Bouret S, Walton ME, Kolling N, Sallet J. Neural responses in macaque prefrontal cortex are linked to strategic exploration. PLoS Biol 2023; 21:e3001985. [PMID: 36716348 PMCID: PMC9910800 DOI: 10.1371/journal.pbio.3001985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Revised: 02/09/2023] [Accepted: 01/03/2023] [Indexed: 02/01/2023] Open
Abstract
Humans have been shown to strategically explore. They can identify situations in which gathering information about distant and uncertain options is beneficial for the future. Because primates rely on scarce resources when they forage, they are also thought to strategically explore, but whether they use the same strategies as humans and the neural bases of strategic exploration in monkeys are largely unknown. We designed a sequential choice task to investigate whether monkeys mobilize strategic exploration based on whether information can improve subsequent choice, but also to ask the novel question about whether monkeys adjust their exploratory choices based on the contingency between choice and information, by sometimes providing the counterfactual feedback about the unchosen option. We show that monkeys decreased their reliance on expected value when exploration could be beneficial, but this was not mediated by changes in the effect of uncertainty on choices. We found strategic exploratory signals in anterior and mid-cingulate cortex (ACC/MCC) and dorsolateral prefrontal cortex (dlPFC). This network was most active when a low value option was chosen, which suggests a role in counteracting expected value signals, when exploration away from value should to be considered. Such strategic exploration was abolished when the counterfactual feedback was available. Learning from counterfactual outcome was associated with the recruitment of a different circuit centered on the medial orbitofrontal cortex (OFC), where we showed that monkeys represent chosen and unchosen reward prediction errors. Overall, our study shows how ACC/MCC-dlPFC and OFC circuits together could support exploitation of available information to the fullest and drive behavior towards finding more information through exploration when it is beneficial.
Collapse
Affiliation(s)
- Caroline I. Jahn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
- Sorbonne Paris Cité universités, Université Paris Descartes, Frontières du Vivant, Paris, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jan Grohn
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Steven Cuell
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Andrew Emberton
- Biomedical Science Services, University of Oxford, Oxford, United Kingdom
| | - Sebastien Bouret
- Motivation, Brain and Behavior Team, Institut du Cerveau et de la Moelle Epinière, Paris, France
| | - Mark E. Walton
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging, OBHA, University of Oxford, Headington, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| | - Jérôme Sallet
- Wellcome Centre for Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
- * E-mail: (CIJ); (JG); (NK); (JS)
| |
Collapse
|
4
|
Higgins C, Vidaurre D, Kolling N, Liu Y, Behrens T, Woolrich M. Spatiotemporally resolved multivariate pattern analysis for M/EEG. Hum Brain Mapp 2022; 43:3062-3085. [PMID: 35302683 PMCID: PMC9188977 DOI: 10.1002/hbm.25835] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Revised: 02/14/2022] [Accepted: 02/27/2022] [Indexed: 11/15/2022] Open
Abstract
An emerging goal in neuroscience is tracking what information is represented in brain activity over time as a participant completes some task. While electroencephalography (EEG) and magnetoencephalography (MEG) offer millisecond temporal resolution of how activity patterns emerge and evolve, standard decoding methods present significant barriers to interpretability as they obscure the underlying spatial and temporal activity patterns. We instead propose the use of a generative encoding model framework that simultaneously infers the multivariate spatial patterns of activity and the variable timing at which these patterns emerge on individual trials. An encoding model inversion maps from these parameters to the equivalent decoding model, allowing predictions to be made about unseen test data in the same way as in standard decoding methodology. These SpatioTemporally Resolved MVPA (STRM) models can be flexibly applied to a wide variety of experimental paradigms, including classification and regression tasks. We show that these models provide insightful maps of the activity driving predictive accuracy metrics; demonstrate behaviourally meaningful variation in the timing of pattern emergence on individual trials; and achieve predictive accuracies that are either equivalent or surpass those achieved by more widely used methods. This provides a new avenue for investigating the brain's representational dynamics and could ultimately support more flexible experimental designs in the future.
Collapse
Affiliation(s)
- Cameron Higgins
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.,Department of Psychiatry, University of Oxford, Oxford, UK
| | - Diego Vidaurre
- Department of Psychiatry, University of Oxford, Oxford, UK.,Center of Functionally Integrative Neuroscience, Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.,Chinese Institute for Brain Research, Beijing, China.,Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
| | - Tim Behrens
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.,Wellcome Trust Centre for Neuroimaging, University College London, London, UK
| | - Mark Woolrich
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.,Department of Psychiatry, University of Oxford, Oxford, UK
| |
Collapse
|
5
|
Scholl J, Trier HA, Rushworth MFS, Kolling N. The effect of apathy and compulsivity on planning and stopping in sequential decision-making. PLoS Biol 2022; 20:e3001566. [PMID: 35358177 PMCID: PMC8970514 DOI: 10.1371/journal.pbio.3001566] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2019] [Accepted: 02/03/2022] [Indexed: 11/21/2022] Open
Abstract
Real-life decision-making often comprises sequences of successive decisions about whether to take opportunities as they are encountered or keep searching for better ones instead. We investigated individual differences related to such sequential decision-making and link them especially to apathy and compulsivity in a large online sample (discovery sample: n = 449 and confirmation sample: n = 756). Our cognitive model revealed distinct changes in the way participants evaluated their environments and planned their own future behaviour. Apathy was linked to decision inertia, i.e., automatically persisting with a sequence of searches for longer than appropriate given the value of searching. Thus, despite being less motivated, they did not avoid the effort associated with longer searches. In contrast, compulsivity was linked to self-reported insensitivity to the cost of continuing with a sequence of searches. The objective measures of behavioural cost insensitivity were clearly linked to compulsivity only in the discovery sample. While the confirmation sample showed a similar effect, it did not reach significance. Nevertheless, in both samples, participants reported awareness of such bias (experienced as "overchasing"). In addition, this awareness made them report preemptively avoiding situations related to the bias. However, we found no evidence of them actually preempting more in the task, which might mean a misalignment of their metacognitive beliefs or that our behavioural measures were incomplete. In summary, individual variation in distinct, fundamental aspects of sequential decision-making can be linked to variation in 2 measures of behavioural traits associated with psychological illness in the normal population.
Collapse
Affiliation(s)
- Jacqueline Scholl
- Lyon Neuroscience Research Center, INSERM U1028, CNRS UMR5292, PSYR2 Team, University Lyon 1, Lyon, France
- Centre Hospitalier Le Vinatier, Pôle EST, Bron, France
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Oxford Centre of Human Brain Activity, Wellcome Integrative Neuroimaging (WIN), Department of Psychiatry, University of Oxford, Oxford, United Kingdom
| | - Hailey A. Trier
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Matthew F. S. Rushworth
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Centre for Functional MRI of the Brain (FMRIB), Wellcome Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Oxford Centre of Human Brain Activity, Wellcome Integrative Neuroimaging (WIN), Department of Psychiatry, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
| |
Collapse
|
6
|
Woo TF, Law CK, Ting KH, Chan CCH, Kolling N, Watanabe K, Chau BKH. Distinct Causal Influences of Dorsolateral Prefrontal Cortex and Posterior Parietal Cortex in Multiple-Option Decision Making. Cereb Cortex 2021; 32:1390-1404. [PMID: 34470053 DOI: 10.1093/cercor/bhab278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Revised: 07/14/2021] [Accepted: 07/15/2021] [Indexed: 11/13/2022] Open
Abstract
Our knowledge about neural mechanisms underlying decision making is largely based on experiments that involved few options. However, it is more common in daily life to choose between many options, in which processing choice information selectively is particularly important. The current study examined whether the dorsolateral prefrontal cortex (dlPFC) and posterior parietal cortex (PPC) are of particular importance to multiple-option decision making. Sixty-eight participants received anodal high definition-transcranial direct current stimulation (HD-tDCS) to focally enhance dlPFC or PPC in a double-blind sham-controlled design. Participants then performed a multiple-option decision making task. We found longer fixations on poorer options were related to less optimal decisions. Interestingly, this negative impact was attenuated after applying anodal HD-tDCS over dlPFC, especially in choices with many options. This suggests that dlPFC has a causal role in filtering choice-irrelevant information. In contrast, these effects were absent after participants received anodal HD-tDCS over PPC. Instead, the choices made by these participants were more biased towards the best options presented on the side contralateral to the stimulation. This suggests PPC has a causal role in value-based spatial selection. To conclude, the dlPFC has a role in filtering undesirable options, whereas the PPC emphasizes the desirable contralateral options.
Collapse
Affiliation(s)
- Tsz-Fung Woo
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong
| | - Chun-Kit Law
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong
| | - Kin-Hung Ting
- University Research Facility in Behavioral and Systems Neuroscience, The Hong Kong Polytechnic University, Hong Kong
| | - Chetwyn C H Chan
- Department of Psychology, The Education University of Hong Kong, Hong Kong
| | - Nils Kolling
- Department of Psychiatry, University of Oxford, Oxford OX3 7JX, UK.,Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, OX3 9DU, UK.,Oxford Centre for Human Brain Activity (OHBA), University of Oxford, Oxford, OX3 7JX, UK
| | - Kei Watanabe
- Department of Frontier Biosciences, Osaka University, Osaka 565-0871, Japan
| | - Bolton K H Chau
- Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong.,University Research Facility in Behavioral and Systems Neuroscience, The Hong Kong Polytechnic University, Hong Kong
| |
Collapse
|
7
|
Hunt LT, Daw ND, Kaanders P, MacIver MA, Mugan U, Procyk E, Redish AD, Russo E, Scholl J, Stachenfeld K, Wilson CRE, Kolling N. Formalizing planning and information search in naturalistic decision-making. Nat Neurosci 2021; 24:1051-1064. [PMID: 34155400 DOI: 10.1038/s41593-021-00866-w] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 03/23/2021] [Indexed: 02/05/2023]
Abstract
Decisions made by mammals and birds are often temporally extended. They require planning and sampling of decision-relevant information. Our understanding of such decision-making remains in its infancy compared with simpler, forced-choice paradigms. However, recent advances in algorithms supporting planning and information search provide a lens through which we can explain neural and behavioral data in these tasks. We review these advances to obtain a clearer understanding for why planning and curiosity originated in certain species but not others; how activity in the medial temporal lobe, prefrontal and cingulate cortices may support these behaviors; and how planning and information search may complement each other as means to improve future action selection.
Collapse
Affiliation(s)
- L T Hunt
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| | - N D Daw
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, USA
| | - P Kaanders
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - M A MacIver
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - U Mugan
- Center for Robotics and Biosystems, Department of Neurobiology, Department of Biomedical Engineering, Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - E Procyk
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - A D Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - E Russo
- Department of Theoretical Neuroscience, Central Institute of Mental Health, Mannheim, Germany.,Department of Psychiatry and Psychotherapy, University Medical Center, Johannes Gutenberg University, Mainz, Germany
| | - J Scholl
- Department of Experimental Psychology, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - C R E Wilson
- Univ Lyon, Université Claude Bernard Lyon 1, INSERM, Stem Cell and Brain Research Institute U1208, Bron, France
| | - N Kolling
- Department of Psychiatry, Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK.
| |
Collapse
|
8
|
Grohn J, Schüffelgen U, Neubert FX, Bongioanni A, Verhagen L, Sallet J, Kolling N, Rushworth MFS. Multiple systems in macaques for tracking prediction errors and other types of surprise. PLoS Biol 2020; 18:e3000899. [PMID: 33125367 PMCID: PMC7657565 DOI: 10.1371/journal.pbio.3000899] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2019] [Revised: 11/11/2020] [Accepted: 09/18/2020] [Indexed: 11/21/2022] Open
Abstract
Animals learn from the past to make predictions. These predictions are adjusted after prediction errors, i.e., after surprising events. Generally, most reward prediction errors models learn the average expected amount of reward. However, here we demonstrate the existence of distinct mechanisms for detecting other types of surprising events. Six macaques learned to respond to visual stimuli to receive varying amounts of juice rewards. Most trials ended with the delivery of either 1 or 3 juice drops so that animals learned to expect 2 juice drops on average even though instances of precisely 2 drops were rare. To encourage learning, we also included sessions during which the ratio between 1 and 3 drops changed. Additionally, in all sessions, the stimulus sometimes appeared in an unexpected location. Thus, 3 types of surprising events could occur: reward amount surprise (i.e., a scalar reward prediction error), rare reward surprise, and visuospatial surprise. Importantly, we can dissociate scalar reward prediction errors—rewards that deviated from the average reward amount expected—and rare reward events—rewards that accorded with the average reward expectation but that rarely occurred. We linked each type of surprise to a distinct pattern of neural activity using functional magnetic resonance imaging. Activity in the vicinity of the dopaminergic midbrain only reflected surprise about the amount of reward. Lateral prefrontal cortex had a more general role in detecting surprising events. Posterior lateral orbitofrontal cortex specifically detected rare reward events regardless of whether they followed average reward amount expectations, but only in learnable reward environments. Animals learn from the past to make predictions, and predictions are adjusted whenever surprising outcomes violate those predictions. This study shows that macaques use multiple systems to detect different types of rare reward events; while activity in the vicinity of the dopaminergic midbrain reflected scalar reward prediction errors (i.e. reward amount), activity in the orbitofrontal cortex reflected a rare reward signal.
Collapse
Affiliation(s)
- Jan Grohn
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Urs Schüffelgen
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Franz-Xaver Neubert
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Alessandro Bongioanni
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Lennart Verhagen
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Jerome Sallet
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Univ Lyon, Université Lyon 1, Inserm, Stem Cell and Brain Research Institute U1208, Bron, France
| | - Nils Kolling
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Wellcome Integrative Neuroimaging (WIN), Oxford Centre for Human Brain Activity (OHBA), University of Oxford, Oxford, United Kingdom
| | - Matthew F S Rushworth
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
9
|
Le Heron C, Kolling N, Plant O, Kienast A, Janska R, Ang YS, Fallon S, Husain M, Apps MAJ. Dopamine Modulates Dynamic Decision-Making during Foraging. J Neurosci 2020; 40:5273-5282. [PMID: 32457071 PMCID: PMC7329313 DOI: 10.1523/jneurosci.2586-19.2020] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2019] [Revised: 03/10/2020] [Accepted: 04/28/2020] [Indexed: 01/11/2023] Open
Abstract
The mesolimbic dopaminergic system exerts a crucial influence on incentive processing. However, the contribution of dopamine in dynamic, ecological situations where reward rates vary, and decisions evolve over time, remains unclear. In such circumstances, current (foreground) reward accrual needs to be compared continuously with potential rewards that could be obtained by traveling elsewhere (background reward rate), to determine the opportunity cost of staying versus leaving. We hypothesized that dopamine specifically modulates the influence of background, but not foreground, reward information when making a dynamic comparison of these variables for optimal behavior. On a novel foraging task based on an ecological account of animal behavior (marginal value theorem), human participants of either sex decided when to leave locations in situations where foreground rewards depleted at different rates, either in rich or poor environments with high or low background reward rates. In line with theoretical accounts, people's decisions to move from current locations were independently modulated by changes in both foreground and background reward rates. Pharmacological manipulation of dopamine D2 receptor activity using the agonist cabergoline significantly affected decisions to move on, specifically modulating the effect of background reward rates. In particular, when on cabergoline, people left patches in poor environments much earlier. These results demonstrate a role of dopamine in signaling the opportunity cost of rewards, not value per se. Using this ecologically derived framework, we uncover a specific mechanism by which D2 dopamine receptor activity modulates decision-making when foreground and background reward rates are dynamically compared.SIGNIFICANCE STATEMENT Many decisions, across economic, political, and social spheres, involve choices to "leave". Such decisions depend on a continuous comparison of a current location's value, with that of other locations you could move on to. However, how the brain makes such decisions is poorly understood. Here, we developed a computerized task, based around theories of how animals make decisions to move on when foraging for food. Healthy human participants had to decide when to leave collecting financial rewards in a location, and travel to collect rewards elsewhere. Using a pharmacological manipulation, we show that the activity of dopamine in the brain modulates decisions to move on, with people valuing other locations differently depending on their dopaminergic state.
Collapse
Affiliation(s)
- Campbell Le Heron
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX39DU, United Kingdom
- New Zealand Brain Research Institute, Christchurch 8011, New Zealand
- Department of Medicine, University of Otago, Christchurch 8011, New Zealand
| | - Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford OX3 9DU, United Kingdom
| | - Olivia Plant
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - Annika Kienast
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - Rebecca Janska
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - Yuen-Siang Ang
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX39DU, United Kingdom
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
| | - Sean Fallon
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
- Bristol Medical School, University of Bristol, Bristol BS8 1UD, United Kingdom
| | - Masud Husain
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford OX39DU, United Kingdom
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford OX3 9DU, United Kingdom
| | - Matthew A J Apps
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford OX3 9DU, United Kingdom
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, B15 2TT, United Kingdom
| |
Collapse
|
10
|
Fouragnan EF, Chau BKH, Folloni D, Kolling N, Verhagen L, Klein-Flügge M, Tankelevitch L, Papageorgiou GK, Aubry JF, Sallet J, Rushworth MFS. The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change. Nat Neurosci 2019; 22:797-808. [PMID: 30988525 PMCID: PMC7116825 DOI: 10.1038/s41593-019-0375-6] [Citation(s) in RCA: 100] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2018] [Accepted: 03/06/2019] [Indexed: 12/17/2022]
Abstract
The neural mechanisms mediating sensory-guided decision making have received considerable attention but animals often pursue behaviors for which there is currently no sensory evidence. Such behaviors are guided by internal representations of choice values that have to be maintained even when these choices are unavailable. We investigated how four macaque monkeys maintained representations of the value of counterfactual choices– choices that could not be taken at the current moment but which could be taken in the future. Using functional magnetic resonance imaging, we found two different patterns of activity co-varying with values of counterfactual choices in a circuit spanning hippocampus, anterior lateral prefrontal cortex, and anterior cingulate cortex (ACC). ACC activity also reflected whether the internal value representations would be translated into actual behavioral change. To establish the causal importance of ACC for this translation process, we used a novel technique, Transcranial Focused Ultrasound Stimulation, to reversibly disrupt ACC activity.
Collapse
Affiliation(s)
- Elsa F Fouragnan
- School of Psychology, University of Plymouth, Plymouth, UK. .,Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Bolton K H Chau
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, China
| | - Davide Folloni
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Nils Kolling
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Lennart Verhagen
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Miriam Klein-Flügge
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Lev Tankelevitch
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Georgios K Papageorgiou
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,McGovern Institute for Brain Research and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Jean-Francois Aubry
- Physics for Medicine Paris, Inserm, ESPCI Paris, CNRS, PSL Research University, Paris, France
| | - Jerome Sallet
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Matthew F S Rushworth
- Wellcome Integrative Neuroimaging, Department of Experimental Psychology, University of Oxford, Oxford, UK.,Wellcome Integrative Neuroimaging, Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| |
Collapse
|
11
|
Kolling N, Scholl J, Chekroud A, Trier HA, Rushworth MFS. Prospection, Perseverance, and Insight in Sequential Behavior. Neuron 2018; 99:1069-1082.e7. [PMID: 30189202 PMCID: PMC6127030 DOI: 10.1016/j.neuron.2018.08.018] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2018] [Revised: 06/14/2018] [Accepted: 08/16/2018] [Indexed: 12/29/2022]
Abstract
Real-world decisions have benefits occurring only later and dependent on additional decisions taken in the interim. We investigated this in a novel decision-making task in humans (n = 76) while measuring brain activity with fMRI (n = 24). Modeling revealed that participants computed the prospective value of decisions: they planned their future behavior taking into account how their decisions might affect which states they would encounter and how they themselves might respond in these states. They considered their own likely future behavioral biases (e.g., failure to adapt to changes in prospective value) and avoided situations in which they might be prone to such biases. Three neural networks in adjacent medial frontal regions were linked to distinct components of prospective decision making: activity in dorsal anterior cingulate cortex, area 8 m/9, and perigenual anterior cingulate cortex reflected prospective value, anticipated changes in prospective value, and the degree to which prospective value influenced decisions.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; Oxford Centre of Human Brain Activity, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.
| | - Jacqueline Scholl
- Department of Experimental Psychology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.
| | - Adam Chekroud
- Department of Experimental Psychology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; Department of Psychiatry, School of Medicine, Yale University, New Haven, CT, USA
| | - Hailey A Trier
- Department of Experimental Psychology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Matthew F S Rushworth
- Department of Experimental Psychology, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| |
Collapse
|
12
|
Abstract
Different kinds of decision making can be categorized by their differential effect on the agent’s current and future states as well as the computational challenges they pose. Here, we draw a distinction between within-state and state-change decision-making, and propose that a dedicated decision mechanism exists in dorsomedial prefrontal cortex (dmPFC) that is specialized for state-change decisions. We set out a formal framework in which state change decisions may be made on the basis of the integrated momentary reward rate, over the intended time to be spent in a state. A key feature of this framework is that reward rate is expressed as a function of continuous time. We argue that dmPFC is suited for this type of decision making partly due to its ability to track the passage of time. This proposed function of dmPFC is placed in contrast to other evaluative systems such as the orbitofrontal cortex, which is important for careful deliberation within a specific model-space or option-space and within a decision strategy.
Collapse
Affiliation(s)
- Nils Kolling
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.,Oxford Centre of Human Brain Activity, Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford, UK
| | - Jill X O'Reilly
- Wellcome Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Oxford, UK.,Wellcome Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, UK.,Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
13
|
Meder D, Kolling N, Verhagen L, Wittmann MK, Scholl J, Madsen KH, Hulme OJ, Behrens TEJ, Rushworth MFS. Simultaneous representation of a spectrum of dynamically changing value estimates during decision making. Nat Commun 2017; 8:1942. [PMID: 29208968 PMCID: PMC5717172 DOI: 10.1038/s41467-017-02169-w] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2017] [Accepted: 11/07/2017] [Indexed: 01/26/2023] Open
Abstract
Decisions are based on value expectations derived from experience. We show that dorsal anterior cingulate cortex and three other brain regions hold multiple representations of choice value based on different timescales of experience organized in terms of systematic gradients across the cortex. Some parts of each area represent value estimates based on recent reward experience while others represent value estimates based on experience over the longer term. The value estimates within these areas interact with one another according to their temporal scaling. Some aspects of the representations change dynamically as the environment changes. The spectrum of value estimates may act as a flexible selection mechanism for combining experience-derived value information with other aspects of value to allow flexible and adaptive decisions in changing environments.
Collapse
Affiliation(s)
- David Meder
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK. .,Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark.
| | - Nils Kolling
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Lennart Verhagen
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK
| | - Marco K Wittmann
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Jacqueline Scholl
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK
| | - Kristoffer H Madsen
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark
| | - Oliver J Hulme
- Danish Research Centre for Magnetic Resonance, Centre for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark
| | - Timothy E J Behrens
- Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK.,Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, OX3 9DU, UK
| |
Collapse
|
14
|
Kolling N, Akam T. (Reinforcement?) Learning to forage optimally. Curr Opin Neurobiol 2017; 46:162-169. [PMID: 28918312 DOI: 10.1016/j.conb.2017.08.008] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2017] [Revised: 08/06/2017] [Accepted: 08/17/2017] [Indexed: 11/24/2022]
Abstract
Foraging effectively is critical to the survival of all animals and this imperative is thought to have profoundly shaped brain evolution. Decisions made by foraging animals often approximate optimal strategies, but the learning and decision mechanisms generating these choices remain poorly understood. Recent work with laboratory foraging tasks in humans suggest their behaviour is poorly explained by model-free reinforcement learning, with simple heuristic strategies better describing behaviour in some tasks, and in others evidence of prospective prediction of the future state of the environment. We suggest that model-based average reward reinforcement learning may provide a common framework for understanding these apparently divergent foraging strategies.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, United Kingdom
| | - Thomas Akam
- Department of Experimental Psychology, University of Oxford, United Kingdom; Champalimaud Neuroscience Program, Champalimaud Center for the Unknown, Portugal.
| |
Collapse
|
15
|
Wittmann MK, Kolling N, Faber NS, Scholl J, Nelissen N, Rushworth MFS. Self-Other Mergence in the Frontal Cortex during Cooperation and Competition. Neuron 2017; 91:482-93. [PMID: 27477020 PMCID: PMC4961240 DOI: 10.1016/j.neuron.2016.06.022] [Citation(s) in RCA: 88] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2016] [Revised: 04/29/2016] [Accepted: 06/13/2016] [Indexed: 10/25/2022]
Abstract
To survive, humans must estimate their own ability and the abilities of others. We found that, although people estimated their abilities on the basis of their own performance in a rational manner, their estimates of themselves were partly merged with the performance of others. Reciprocally, their ability estimates for others also reflected their own, as well as the others', performance. Self-other mergence operated in a context-dependent manner: interacting with high or low performers, respectively, enhanced and diminished own ability estimates in cooperative contexts, but the opposite occurred in competitive contexts. Self-other mergence not only influenced subjective evaluations, it also affected how people subsequently objectively adjusted their performance. Perigenual anterior cingulate cortex tracked one's own performance. Dorsomedial frontal area 9 tracked others' performances, but also integrated contextual and self-related information. Self-other mergence increased with the strength of self and other representations in area 9, suggesting it carries interdependent representations of self and other.
Collapse
Affiliation(s)
- Marco K Wittmann
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK.
| | - Nils Kolling
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK
| | - Nadira S Faber
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK
| | - Jacqueline Scholl
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK; Department of Psychiatry, University of Oxford, Warneford Hospital, OX3 7JX Oxford, UK
| | - Natalie Nelissen
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK; Department of Psychiatry, University of Oxford, Warneford Hospital, OX3 7JX Oxford, UK
| | - Matthew F S Rushworth
- Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD Oxford, UK; Centre for Functional MRI of the Brain, University of Oxford, John Radcliffe Hospital, OX3 9DU Oxford, UK
| |
Collapse
|
16
|
Kolling N, Wittmann MK, Behrens TEJ, Boorman ED, Mars RB, Rushworth MFS. Value, search, persistence and model updating in anterior cingulate cortex. Nat Neurosci 2017; 19:1280-5. [PMID: 27669988 DOI: 10.1038/nn.4382] [Citation(s) in RCA: 255] [Impact Index Per Article: 36.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2015] [Accepted: 08/08/2016] [Indexed: 12/13/2022]
Abstract
Dorsal anterior cingulate cortex (dACC) carries a wealth of value-related information necessary for regulating behavioral flexibility and persistence. It signals error and reward events informing decisions about switching or staying with current behavior. During decision-making, it encodes the average value of exploring alternative choices (search value), even after controlling for response selection difficulty, and during learning, it encodes the degree to which internal models of the environment and current task must be updated. dACC value signals are derived in part from the history of recent reward integrated simultaneously over multiple time scales, thereby enabling comparison of experience over the recent and extended past. Such ACC signals may instigate attentionally demanding and difficult processes such as behavioral change via interactions with prefrontal cortex. However, the signal in dACC that instigates behavioral change need not itself be a conflict or difficulty signal.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Marco K Wittmann
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Tim E J Behrens
- Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Erie D Boorman
- Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| | - Rogier B Mars
- Department of Experimental Psychology, University of Oxford, Oxford, UK.,Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK.,Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, Nijmegen, the Netherlands
| | - Matthew F S Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, UK.,Centre for Functional MRI of the Brain (MRI), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, Oxford, UK
| |
Collapse
|
17
|
Scholl J, Kolling N, Nelissen N, Stagg CJ, Harmer CJ, Rushworth MFS. Excitation and inhibition in anterior cingulate predict use of past experiences. eLife 2017; 6:e20365. [PMID: 28055824 PMCID: PMC5213710 DOI: 10.7554/elife.20365] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2016] [Accepted: 11/29/2016] [Indexed: 01/17/2023] Open
Abstract
Dorsal anterior cingulate cortex (dACC) mediates updating and maintenance of cognitive models of the world used to drive adaptive reward-guided behavior. We investigated the neurochemical underpinnings of this process. We used magnetic resonance spectroscopy in humans, to measure levels of glutamate and GABA in dACC. We examined their relationship to neural signals in dACC, measured with fMRI, and cognitive task performance. Both inhibitory and excitatory neurotransmitters in dACC were predictive of the strength of neural signals in dACC and behavioral adaptation. Glutamate levels were correlated, first, with stronger neural activity representing information to be learnt about the tasks' costs and benefits and, second, greater use of this information in the guidance of behavior. GABA levels were negatively correlated with the same neural signals and the same indices of behavioral influence. Our results suggest that glutamate and GABA in dACC affect the encoding and use of past experiences to guide behavior.
Collapse
Affiliation(s)
- Jacqueline Scholl
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
| | - Natalie Nelissen
- Oxford Centre for Human Brain Activity, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | - Charlotte J Stagg
- Oxford Centre for Human Brain Activity, Department of Psychiatry, Oxford University, Oxford, United Kingdom
| | | | - Matthew FS Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
- Oxford Centre for Functional MRI of the Brain, University of Oxford, Oxford, United Kingdom
| |
Collapse
|
18
|
Chakraborty S, Kolling N, Walton ME, Mitchell AS. Critical role for the mediodorsal thalamus in permitting rapid reward-guided updating in stochastic reward environments. eLife 2016; 5. [PMID: 27136677 PMCID: PMC4887209 DOI: 10.7554/elife.13588] [Citation(s) in RCA: 41] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2015] [Accepted: 05/01/2016] [Indexed: 11/13/2022] Open
Abstract
Adaptive decision-making uses information gained when exploring alternative options to decide whether to update the current choice strategy. Magnocellular mediodorsal thalamus (MDmc) supports adaptive decision-making, but its causal contribution is not well understood. Monkeys with excitotoxic MDmc damage were tested on probabilistic three-choice decision-making tasks. They could learn and track the changing values in object-reward associations, but they were severely impaired at updating choices after reversals in reward contingencies or when there were multiple options associated with reward. These deficits were not caused by perseveration or insensitivity to negative feedback though. Instead, monkeys with MDmc lesions exhibited an inability to use reward to promote choice repetition after switching to an alternative option due to a diminished influence of recent past choices and the last outcome to guide future behavior. Together, these data suggest MDmc allows for the rapid discovery and persistence with rewarding options, particularly in uncertain or changing environments. DOI:http://dx.doi.org/10.7554/eLife.13588.001 A small structure deep inside the brain, called the mediodorsal thalamus, is a critical part of a brain network that is important for learning new information and making decisions. However, the exact role of this brain area is still not understood, and there is little evidence showing that this area is actually needed to make the best choices. To explore the role of this area further, Chakraborty et al. trained macaque monkeys to choose between three colorful objects displayed on a touchscreen that was controlled by a computer. Some of their choices resulted in the monkeys getting a tasty food pellet as a reward. However the probability of receiving a reward changed during testing, and in some cases, reversed, meaning that the highest rewarded object was no longer rewarded when chosen and vice versa. While at first the monkeys did not know which choice was the right one, they quickly learned and changed their choices during the test according to which option resulted in them receiving the most reward. Next, the mediodorsal thalamus in each monkey was damaged and the tests were repeated. Previous research had suggested that such damage might result in animals repeatedly choosing the same option, even though it is clearly the wrong choice. However, Chakraborty et al. showed that it is not as simple as that. Instead monkeys with damage to the mediodorsal thalamus could make different choices but they struggled to use information from their most recent choices to best guide their future behavior. Specifically, the pattern of the monkeys’ choices suggests that the mediodorsal thalamus helps to quickly link recent choices that resulted in a reward in order to allow an individual to choose the best option as their next choice. Further studies are now needed to understand the messages that are relayed between the mediodorsal thalamus and interconnected areas during this rapid linking of recent choices, rewards and upcoming decisions. This will help reveal how these brain areas support normal thought processes and how these processes might be altered in mental health disorders involving learning information and making decisions. DOI:http://dx.doi.org/10.7554/eLife.13588.002
Collapse
Affiliation(s)
| | - Nils Kolling
- Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| | - Mark E Walton
- Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| | - Anna S Mitchell
- Department of Experimental Psychology, Oxford University, Oxford, United Kingdom
| |
Collapse
|
19
|
Kolling N, Behrens T, Wittmann MK, Rushworth M. Multiple signals in anterior cingulate cortex. Curr Opin Neurobiol 2016; 37:36-43. [PMID: 26774693 PMCID: PMC4863523 DOI: 10.1016/j.conb.2015.12.007] [Citation(s) in RCA: 144] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2015] [Revised: 12/15/2015] [Accepted: 12/17/2015] [Indexed: 12/23/2022]
Abstract
There are multiple signals in anterior cingulate cortex (ACC). ACC activity reflects value of behavioural change even after controlling for difficulty. ACC activity reflects updating of internal models even after controlling for difficulty.
Activity in anterior cingulate cortex (ACC) has been linked both to commitment to a course of action, even when it is associated with costs, and to exploring or searching for alternative courses of action. Here we review evidence that this is due to the presence of multiple signals in ACC reflecting the updating of beliefs and internal models of the environment and encoding aspects of choice value, including the average value of choices afforded by the environment (‘search value’). We contrast this evidence with the influential view that ACC activity is better described as reflecting task difficulty. A consideration of cortical neural network properties explains why ACC may carry such signals and also exhibit sensitivity to task difficulty.
Collapse
Affiliation(s)
- N Kolling
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | - Tej Behrens
- Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, UK
| | - M K Wittmann
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Mfs Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Centre for Functional MRI of the Brain (FMRIB), Nuffield Department of Clinical Neurosciences, John Radcliffe Hospital, University of Oxford, UK
| |
Collapse
|
20
|
Abstract
Cost-benefit analysis in decision making takes place in everyday life for animals and humans alike. In this issue, a neural circuit specific for modulating these behaviors is identified in rats and reveals elusive functional distinctions between long-mysterious anatomical features of the brain.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK
| | | |
Collapse
|
21
|
Abstract
Sometimes when a choice is made, the outcome is not guaranteed and there is only a probability of its occurrence. Each individual’s attitude to probability, sometimes called risk proneness or aversion, has been assumed to be static. Behavioral ecological studies, however, suggest such attitudes are dynamically modulated by the context an organism finds itself in; in some cases, it may be optimal to pursue actions with a low probability of success but which are associated with potentially large gains. We show that human subjects rapidly adapt their use of probability as a function of current resources, goals, and opportunities for further foraging. We demonstrate that dorsal anterior cingulate cortex (dACC) carries signals indexing the pressure to pursue unlikely choices and signals related to the taking of such choices. We show that dACC exerts this control over behavior when it, rather than ventromedial prefrontal cortex, interacts with posterior cingulate cortex. Against common belief, risk preference is not fixed but is dynamically updated Environments exert varying “risk pressure” tracked by ACC Risk-related values and behavior are then regulated by ACC Two mechanisms regulate competition for control of behavior between ACC and vmPFC
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK.
| | - Marco Wittmann
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK
| | - Matthew F S Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK; Centre for Functional MRI of the Brain, University of Oxford, Oxford OX3 9DU, UK
| |
Collapse
|
22
|
Abstract
The orbitofrontal cortex and adjacent ventromedial prefrontal cortex carry reward representations and mediate flexible behaviour when circumstances change. Here we review how recent experiments in humans and macaques have confirmed the existence of a major difference between the functions of the ventromedial prefrontal cortex and adjacent medial orbitofrontal cortex (mOFC) on the one hand and the lateral orbitofrontal cortex (lOFC) on the other. These differences, however, may not be best accounted for in terms of specializations for reward and error/punishment processing as is commonly assumed. Instead we argue that both lesion and functional magnetic resonance imaging studies reveal that the lOFC is concerned with the assignment of credit for both reward and error outcomes to the choice of specific stimuli and with the linking of specific stimulus representations to representations of specific types of reward outcome. By contrast, we argue that the ventromedial prefrontal cortex/mOFC is concerned with evaluation, value-guided decision-making and maintenance of a choice over successive decisions. Despite the popular view that they cause perseveration of behaviour and inability to inhibit repetition of a previously made choice, we found that lesions in neither orbitofrontal subdivision caused perseveration. On the contrary, lesions in the lOFC made animals switch more rapidly between choices when they were finding it difficult to assign reward values to choices. Lesions in the mOFC caused animals to lose their normal predisposition to repeat previously successful choices, suggesting that the mOFC does not just mediate value comparison in choice but also facilitates maintenance of the same choice if it has been successful.
Collapse
Affiliation(s)
- M P Noonan
- Decision and Action Laboratory, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford OX1 3UD, UK
| | | | | | | |
Collapse
|
23
|
Rushworth MFS, Kolling N, Sallet J, Mars RB. Valuation and decision-making in frontal cortex: one or many serial or parallel systems? Curr Opin Neurobiol 2012; 22:946-55. [PMID: 22572389 DOI: 10.1016/j.conb.2012.04.011] [Citation(s) in RCA: 197] [Impact Index Per Article: 16.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2012] [Revised: 03/29/2012] [Accepted: 04/15/2012] [Indexed: 10/28/2022]
Abstract
We evaluate the merits of different conceptualizations of frontal cortex function in value-guided decision-making. According to one view each frontal cortical region is concerned with a different aspect of the process of learning about and evaluating choices and then selecting actions. An alternative view, however, sees sets of decision-making circuits working in parallel within the frontal lobes in order to make different types of decisions. While there is a neural circuit for making choices between pairs of simultaneously presented items in the manner that is frequently assessed in the laboratory, there is also evidence that other frontal lobe circuits have evolved to make other types of choices such as those made during the course of foraging.
Collapse
|
24
|
Abstract
Behavioral economic studies involving limited numbers of choices have provided key insights into neural decision-making mechanisms. By contrast, animals' foraging choices arise in the context of sequences of encounters with prey or food. On each encounter, the animal chooses whether to engage or, if the environment is sufficiently rich, to search elsewhere. The cost of foraging is also critical. We demonstrate that humans can alternate between two modes of choice, comparative decision-making and foraging, depending on distinct neural mechanisms in ventromedial prefrontal cortex (vmPFC) and anterior cingulate cortex (ACC) using distinct reference frames; in ACC, choice variables are represented in invariant reference to foraging or searching for alternatives. Whereas vmPFC encodes values of specific well-defined options, ACC encodes the average value of the foraging environment and cost of foraging.
Collapse
Affiliation(s)
- Nils Kolling
- Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK.
| | | | | | | |
Collapse
|
25
|
Mars RB, Shea NJ, Kolling N, Rushworth MFS. Model-based analyses: Promises, pitfalls, and example applications to the study of cognitive control. Q J Exp Psychol (Hove) 2011; 65:252-67. [PMID: 20437297 PMCID: PMC3335278 DOI: 10.1080/17470211003668272] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
We discuss a recent approach to investigating cognitive control, which has the potential to deal with some of the challenges inherent in this endeavour. In a model-based approach, the researcher defines a formal, computational model that performs the task at hand and whose performance matches that of a research participant. The internal variables in such a model might then be taken as proxies for latent variables computed in the brain. We discuss the potential advantages of such an approach for the study of the neural underpinnings of cognitive control and its pitfalls, and we make explicit the assumptions underlying the interpretation of data obtained using this approach.
Collapse
Affiliation(s)
- Rogier B Mars
- Department of Experimental Psychology, University of Oxford, Oxford, UK.
| | | | | | | |
Collapse
|