1
|
Cheng Y, Magnard R, Langdon AJ, Lee D, Janak PH. Chronic Ethanol Exposure Produces Persistent Impairment in Cognitive Flexibility and Decision Signals in the Striatum. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.03.10.584332. [PMID: 38585868 PMCID: PMC10996555 DOI: 10.1101/2024.03.10.584332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Lack of cognitive flexibility is a hallmark of substance use disorders and has been associated with drug-induced synaptic plasticity in the dorsomedial striatum (DMS). Yet the possible impact of altered plasticity on real-time striatal neural dynamics during decision-making is unclear. Here, we identified persistent impairments induced by chronic ethanol (EtOH) exposure on cognitive flexibility and striatal decision signals. After a substantial withdrawal period from prior EtOH vapor exposure, male, but not female, rats exhibited reduced adaptability and exploratory behavior during a dynamic decision-making task. Reinforcement learning models showed that prior EtOH exposure enhanced learning from rewards over omissions. Notably, neural signals in the DMS related to the decision outcome were enhanced, while those related to choice and choice-outcome conjunction were reduced, in EtOH-treated rats compared to the controls. These findings highlight the profound impact of chronic EtOH exposure on adaptive decision-making, pinpointing specific changes in striatal representations of actions and outcomes as underlying mechanisms for cognitive deficits.
Collapse
Affiliation(s)
- Yifeng Cheng
- Department Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD
| | - Robin Magnard
- Department Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD
| | - Angela J. Langdon
- Intramural Research Program, National Institute of Mental Health, National Institutes of Health, Bethesda, MD
| | - Daeyeol Lee
- Department Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD
- Zanvyl Krieger Mind/Brain Institute, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD
- Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, MD
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD
| | - Patricia H. Janak
- Department Psychological and Brain Sciences, Krieger School of Arts and Sciences, Johns Hopkins University, Baltimore, MD
- Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, MD
- Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD
| |
Collapse
|
2
|
Lowet AS, Zheng Q, Meng M, Matias S, Drugowitsch J, Uchida N. An opponent striatal circuit for distributional reinforcement learning. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.02.573966. [PMID: 38260354 PMCID: PMC10802299 DOI: 10.1101/2024.01.02.573966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Machine learning research has achieved large performance gains on a wide range of tasks by expanding the learning target from mean rewards to entire probability distributions of rewards - an approach known as distributional reinforcement learning (RL)1. The mesolimbic dopamine system is thought to underlie RL in the mammalian brain by updating a representation of mean value in the striatum2,3, but little is known about whether, where, and how neurons in this circuit encode information about higher-order moments of reward distributions4. To fill this gap, we used high-density probes (Neuropixels) to acutely record striatal activity from well-trained, water-restricted mice performing a classical conditioning task in which reward mean, reward variance, and stimulus identity were independently manipulated. In contrast to traditional RL accounts, we found robust evidence for abstract encoding of variance in the striatum. Remarkably, chronic ablation of dopamine inputs disorganized these distributional representations in the striatum without interfering with mean value coding. Two-photon calcium imaging and optogenetics revealed that the two major classes of striatal medium spiny neurons - D1 and D2 MSNs - contributed to this code by preferentially encoding the right and left tails of the reward distribution, respectively. We synthesize these findings into a new model of the striatum and mesolimbic dopamine that harnesses the opponency between D1 and D2 MSNs5-15 to reap the computational benefits of distributional RL.
Collapse
Affiliation(s)
- Adam S Lowet
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
- Program in Neuroscience, Harvard University, Boston, MA, USA
| | - Qiao Zheng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Melissa Meng
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Sara Matias
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| | - Jan Drugowitsch
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Naoshige Uchida
- Center for Brain Science, Harvard University, Cambridge, MA, USA
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA
| |
Collapse
|
3
|
Hattori R, Hedrick NG, Jain A, Chen S, You H, Hattori M, Choi JH, Lim BK, Yasuda R, Komiyama T. Meta-reinforcement learning via orbitofrontal cortex. Nat Neurosci 2023; 26:2182-2191. [PMID: 37957318 PMCID: PMC10689244 DOI: 10.1038/s41593-023-01485-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Accepted: 10/06/2023] [Indexed: 11/15/2023]
Abstract
The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.
Collapse
Affiliation(s)
- Ryoma Hattori
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA.
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA.
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA.
- Department of Neuroscience, The Herbert Wertheim UF Scripps Institute for Biomedical Innovation & Technology, University of Florida, Jupiter, FL, USA.
| | - Nathan G Hedrick
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA
| | - Anant Jain
- Max Planck Florida Institute for Neuroscience, Jupiter, FL, USA
| | - Shuqi Chen
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA
| | - Hanjia You
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA
| | - Mariko Hattori
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA
| | - Jun-Hyeok Choi
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
| | - Byung Kook Lim
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA
| | - Ryohei Yasuda
- Max Planck Florida Institute for Neuroscience, Jupiter, FL, USA
| | - Takaki Komiyama
- Department of Neurobiology, University of California San Diego, La Jolla, CA, USA.
- Center for Neural Circuits and Behavior, University of California San Diego, La Jolla, CA, USA.
- Department of Neurosciences, University of California San Diego, La Jolla, CA, USA.
- Halıcıoğlu Data Science Institute, University of California San Diego, La Jolla, CA, USA.
| |
Collapse
|
4
|
Farries MA, Faust TW, Mohebi A, Berke JD. Selective encoding of reward predictions and prediction errors by globus pallidus subpopulations. Curr Biol 2023; 33:4124-4135.e5. [PMID: 37703876 PMCID: PMC10591972 DOI: 10.1016/j.cub.2023.08.042] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 07/04/2023] [Accepted: 08/15/2023] [Indexed: 09/15/2023]
Abstract
Basal ganglia (BG) circuits help guide and invigorate actions using predictions of future rewards (values). Within the BG, the globus pallidus pars externa (GPe) may play an essential role in aggregating and distributing value information. We recorded from the GPe in unrestrained rats performing both Pavlovian and instrumental tasks to obtain rewards and distinguished neuronal subtypes by their firing properties across the wake/sleep cycle and optogenetic tagging. In both tasks, the parvalbumin-positive (PV+), faster-firing "prototypical" neurons showed strong, sustained modulation by value, unlike other subtypes, including the "arkypallidal" cells that project back to striatum. Furthermore, we discovered that a distinct minority (7%) of GP cells display slower, pacemaker-like firing and encode reward prediction errors (RPEs) almost identically to midbrain dopamine neurons. These cell-specific forms of GPe value representation help define the circuit mechanisms by which the BG contribute to motivation and reinforcement learning.
Collapse
Affiliation(s)
- Michael A Farries
- Knoebel Institute for Healthy Aging, University of Denver, Denver, CO 80210, USA
| | - Thomas W Faust
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94143, USA
| | - Ali Mohebi
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94143, USA
| | - Joshua D Berke
- Department of Neurology, University of California, San Francisco, San Francisco, CA 94143, USA; Department of Psychiatry and Behavioral Sciences, Neuroscience Graduate Program, Kavli Institute for Fundamental Neuroscience, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA 94143, USA.
| |
Collapse
|
5
|
Maisson DJN, Cervera RL, Voloh B, Conover I, Zambre M, Zimmermann J, Hayden BY. Widespread coding of navigational variables in prefrontal cortex. Curr Biol 2023; 33:3478-3488.e3. [PMID: 37541250 PMCID: PMC10984098 DOI: 10.1016/j.cub.2023.07.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 06/01/2023] [Accepted: 07/13/2023] [Indexed: 08/06/2023]
Abstract
To navigate effectively, we must represent information about our location in the environment. Traditional research highlights the role of the hippocampal complex in this process. Spurred by recent research highlighting the widespread cortical encoding of cognitive and motor variables previously thought to have localized function, we hypothesized that navigational variables would be likewise encoded widely, especially in the prefrontal cortex, which is associated with volitional behavior. We recorded neural activity from six prefrontal regions while macaques performed a foraging task in an open enclosure. In all regions, we found strong encoding of allocentric position, allocentric head direction, boundary distance, and linear and angular velocity. These encodings were not accounted for by distance, time to reward, or motor factors. The strength of coding of all variables increased along a ventral-to-dorsal gradient. Together, these results argue that encoding of navigational variables is not localized to the hippocampus and support the hypothesis that navigation is continuous with other forms of flexible cognition in the service of action.
Collapse
Affiliation(s)
- David J-N Maisson
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Roberto Lopez Cervera
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Voloh
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Indirah Conover
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Mrunal Zambre
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Jan Zimmermann
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA
| | - Benjamin Y Hayden
- Department of Neuroscience, Center for Magnetic Resonance Research, Center for Neuroengineering, Department of Biomedical Engineering, University of Minnesota, Minneapolis, MN 55455, USA.
| |
Collapse
|
6
|
Perisse E, Miranda M, Trouche S. Modulation of aversive value coding in the vertebrate and invertebrate brain. Curr Opin Neurobiol 2023; 79:102696. [PMID: 36871400 DOI: 10.1016/j.conb.2023.102696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 01/30/2023] [Accepted: 01/31/2023] [Indexed: 03/06/2023]
Abstract
Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.
Collapse
Affiliation(s)
- Emmanuel Perisse
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| | - Magdalena Miranda
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France
| | - Stéphanie Trouche
- Institute of Functional Genomics, University of Montpellier, CNRS, Inserm, 141 rue de la Cardonille, 34094 Montpellier Cedex 5, France.
| |
Collapse
|
7
|
Yun M, Hwang JY, Jung MW. Septotemporal variations in hippocampal value and outcome processing. Cell Rep 2023; 42:112094. [PMID: 36763498 DOI: 10.1016/j.celrep.2023.112094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 11/11/2022] [Accepted: 01/26/2023] [Indexed: 02/11/2023] Open
Abstract
A large body of evidence indicates functional variations along the hippocampal longitudinal axis. To investigate whether and how value and outcome processing vary between the dorsal (DH) and the ventral hippocampus (VH), we examined neuronal activity and inactivation effects of the DH and VH in mice performing probabilistic classical conditioning tasks. Inactivation of either structure disrupts value-dependent anticipatory licking, and value-coding neurons are found in both structures, indicating their involvement in value processing. However, the DH neuronal population increases activity as a function of value, while the VH neuronal population is preferentially responsive to the highest-value sensory cue. Also, signals related to outcome-dependent value learning are stronger in the DH. VH neurons instead show rapid responses to punishment and strongly biased responses to negative prediction error. These findings suggest that the DH faithfully represents the external value landscape, whereas the VH preferentially represents behaviorally relevant, salient features of experienced events.
Collapse
Affiliation(s)
- Miru Yun
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea; Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon 34141, Korea
| | - Ji Young Hwang
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea; Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon 34141, Korea
| | - Min Whan Jung
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea; Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon 34141, Korea.
| |
Collapse
|
8
|
Cox J, Minerva AR, Fleming WT, Zimmerman CA, Hayes C, Zorowitz S, Bandi A, Ornelas S, McMannon B, Parker NF, Witten IB. A neural substrate of sex-dependent modulation of motivation. Nat Neurosci 2023; 26:274-284. [PMID: 36646878 DOI: 10.1038/s41593-022-01229-9] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 11/01/2022] [Indexed: 01/18/2023]
Abstract
While there is emerging evidence of sex differences in decision-making behavior, the neural substrates that underlie such differences remain largely unknown. Here we demonstrate that in mice performing a value-based decision-making task, while choices are similar between the sexes, motivation to engage in the task is modulated by action value more strongly in females than in males. Inhibition of activity in anterior cingulate cortex (ACC) neurons that project to the dorsomedial striatum (DMS) preferentially disrupts this relationship between value and motivation in females, without affecting choice in either sex. In line with these effects, in females compared to males, ACC-DMS neurons have stronger representations of negative outcomes and more neurons are active when the value of the chosen option is low. By contrast, the representation of each choice is similar between the sexes. Thus, we identify a neural substrate that contributes to sex-specific modulation of motivation by value.
Collapse
Affiliation(s)
- Julia Cox
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Neuroscience, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.
| | - Adelaide R Minerva
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Weston T Fleming
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | | | - Cameron Hayes
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Samuel Zorowitz
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Akhil Bandi
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Sharon Ornelas
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Brenna McMannon
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Nathan F Parker
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
9
|
Hu J, Konovalov A, Ruff CC. A unified neural account of contextual and individual differences in altruism. eLife 2023; 12:80667. [PMID: 36752704 PMCID: PMC9908080 DOI: 10.7554/elife.80667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 01/19/2023] [Indexed: 02/09/2023] Open
Abstract
Altruism is critical for cooperation and productivity in human societies but is known to vary strongly across contexts and individuals. The origin of these differences is largely unknown, but may in principle reflect variations in different neurocognitive processes that temporally unfold during altruistic decision making (ranging from initial perceptual processing via value computations to final integrative choice mechanisms). Here, we elucidate the neural origins of individual and contextual differences in altruism by examining altruistic choices in different inequality contexts with computational modeling and electroencephalography (EEG). Our results show that across all contexts and individuals, wealth distribution choices recruit a similar late decision process evident in model-predicted evidence accumulation signals over parietal regions. Contextual and individual differences in behavior related instead to initial processing of stimulus-locked inequality-related value information in centroparietal and centrofrontal sensors, as well as to gamma-band synchronization of these value-related signals with parietal response-locked evidence-accumulation signals. Our findings suggest separable biological bases for individual and contextual differences in altruism that relate to differences in the initial processing of choice-relevant information.
Collapse
Affiliation(s)
- Jie Hu
- Zurich Center for Neuroeconomics, Department of Economics, University of ZurichZurichSwitzerland
| | - Arkady Konovalov
- Zurich Center for Neuroeconomics, Department of Economics, University of ZurichZurichSwitzerland,Centre for Human Brain Health, School of Psychology, University of BirminghamBirminghamUnited Kingdom
| | - Christian C Ruff
- Zurich Center for Neuroeconomics, Department of Economics, University of ZurichZurichSwitzerland,University Research Priority Program 'Adaptive Brain Circuits in Development and Learning' (URPP AdaBD), University of ZurichZurichSwitzerland
| |
Collapse
|
10
|
Grossman CD, Cohen JY. Neuromodulation and Neurophysiology on the Timescale of Learning and Decision-Making. Annu Rev Neurosci 2022; 45:317-337. [PMID: 35363533 DOI: 10.1146/annurev-neuro-092021-125059] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Nervous systems evolved to effectively navigate the dynamics of the environment to achieve their goals. One framework used to study this fundamental problem arose in the study of learning and decision-making. In this framework, the demands of effective behavior require slow dynamics-on the scale of seconds to minutes-of networks of neurons. Here, we review the phenomena and mechanisms involved. Using vignettes from a few species and areas of the nervous system, we view neuromodulators as key substrates for temporal scaling of neuronal dynamics. Expected final online publication date for the Annual Review of Neuroscience, Volume 45 is July 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.
Collapse
Affiliation(s)
- Cooper D Grossman
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, and Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA;
| | - Jeremiah Y Cohen
- The Solomon H. Snyder Department of Neuroscience, Brain Science Institute, and Kavli Neuroscience Discovery Institute, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA;
| |
Collapse
|
11
|
Wong AL, Green AL, Isaacs MW. Motor Plans under Uncertainty Reflect a Trade-Off between Maximizing Reward and Success. eNeuro 2022; 9:ENEURO.0503-21.2022. [PMID: 35346958 PMCID: PMC9007409 DOI: 10.1523/eneuro.0503-21.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Revised: 03/10/2022] [Accepted: 03/15/2022] [Indexed: 11/21/2022] Open
Abstract
When faced with multiple potential movement options, individuals either reach directly to one of the options, or initiate a reach intermediate between the options. It remains unclear why people generate these two types of behaviors. Using the go-before-you-know task (commonly used to study behavior under choice uncertainty) in humans, we examined two key questions. First, do these two types of responses actually reflect distinct movement strategies? If so, the relative desirability (i.e., weighing the success likelihood vs the attainable reward) of the two target options would not need to be computed identically for direct and intermediate reaches. We showed that indeed, when reward and success likelihood differed between the two options, reach direction was preferentially biased toward different directions for direct versus intermediate reaches. Importantly, this suggests that the computation of subjective values depends on the choice of movement strategy. Second, what drives individual differences in how people respond under uncertainty? We found that risk/reward-seeking individuals tended to generate more intermediate reaches and were more responsive to changes in reward, suggesting these movements may reflect a strategy to maximize reward versus success. Together, these findings suggest that when faced with choice uncertainty, individuals adopt movement strategies consistent with their risk/reward attitude, preferentially biasing behavior toward exogenous rewards or endogenous success and consequently modulating the relative desirability of the available options.
Collapse
Affiliation(s)
- Aaron L Wong
- Moss Rehabilitation Research Institute, Elkins Park, PA 19027
| | - Audrey L Green
- Department of Neuroscience, Holy Family University, Philadelphia, PA 19114
| | | |
Collapse
|
12
|
Food reward derives from nutrient content and sensory qualities. Proc Natl Acad Sci U S A 2021; 118:2109735118. [PMID: 34266961 DOI: 10.1073/pnas.2109735118] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
13
|
Shin EJ, Jang Y, Kim S, Kim H, Cai X, Lee H, Sul JH, Lee SH, Chung Y, Lee D, Jung MW. Robust and distributed neural representation of action values. eLife 2021; 10:53045. [PMID: 33876728 PMCID: PMC8104958 DOI: 10.7554/elife.53045] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Accepted: 04/19/2021] [Indexed: 11/13/2022] Open
Abstract
Studies in rats, monkeys, and humans have found action-value signals in multiple regions of the brain. These findings suggest that action-value signals encoded in these brain structures bias choices toward higher expected rewards. However, previous estimates of action-value signals might have been inflated by serial correlations in neural activity and also by activity related to other decision variables. Here, we applied several statistical tests based on permutation and surrogate data to analyze neural activity recorded from the striatum, frontal cortex, and hippocampus. The results show that previously identified action-value signals in these brain areas cannot be entirely accounted for by concurrent serial correlations in neural activity and action value. We also found that neural activity related to action value is intermixed with signals related to other decision variables. Our findings provide strong evidence for broadly distributed neural signals related to action value throughout the brain.
Collapse
Affiliation(s)
- Eun Ju Shin
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Yunsil Jang
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Soyoun Kim
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon, Republic of Korea
| | - Hoseok Kim
- Department of Neuroscience, Biomedicum, Karolinska Institutet, Stockholm, Sweden
| | - Xinying Cai
- New York University Shanghai, NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, and Shanghai Key Laboratory of Brain Functional Genomics (Ministry of Education), School of Psychology and Cognitive Science, East China Normal University, Shanghai, China
| | - Hyunjung Lee
- Department of Anatomy, Kyungpook National University School of Medicine, Daegu, Republic of Korea
| | - Jung Hoon Sul
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea
| | - Sung-Hyun Lee
- Neuroscience Graduate Program, Ajou University School of Medicine, Suwon, Republic of Korea
| | - Yeonseung Chung
- Department of Mathematical Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Daeyeol Lee
- The Zanvyl Krieger Mind/Brain Institute, Kavli Neuroscience Discovery Institute, Department of Neuroscience, and Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, United States
| | - Min Whan Jung
- Center for Synaptic Brain Dysfunctions, Institute for Basic Science, Daejeon, Republic of Korea.,Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| |
Collapse
|