1
|
Mueller D, Giglio E, Chen CS, Holm A, Ebitz RB, Grissom NM. Touchscreen Response Precision Is Sensitive to the Explore/Exploit Trade-off. eNeuro 2025; 12:ENEURO.0538-24.2025. [PMID: 40246556 PMCID: PMC12061356 DOI: 10.1523/eneuro.0538-24.2025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2024] [Revised: 04/04/2025] [Accepted: 04/11/2025] [Indexed: 04/19/2025] Open
Abstract
The explore/exploit trade-off is a fundamental property of choice selection during reward-guided decision making, where the "same" choice can reflect either of these internal cognitive states. An unanswered question is whether the execution of a decision provides an underexplored measure of internal cognitive states. Touchscreens are increasingly used across species for cognitive testing and afford the ability to measure the precise location of choice touch responses. We examined how male and female mice in a restless bandit decision making task interacted with a touchscreen to determine if the explore/exploit trade-off, prior reward, and/or sex differences change the variability in the kinetics of touchscreen choices. During exploit states, successive touch responses are closer together than those made in an explore state, suggesting exploit states reflect periods of increased motor stereotypy. Although exploit decisions might be expected to be rewarded more frequently than explore decisions, we find that immediate past reward reduces choice variability independently of explore/exploit state. Male mice are more variable in their interactions with the touchscreen than females, even in low-variability trials such as exploit or following reward. These results suggest that as exploit behavior emerges in reward-guided decision making, all mice become less variable and more automated in both their choice and the actions taken to make that choice, but this occurs on a background of increased male variability. These data uncover the hidden potential for touchscreen decision making tasks to uncover the latent neural states that unite cognition and movement.
Collapse
Affiliation(s)
- Dana Mueller
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| | - Erin Giglio
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| | - Cathy S Chen
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| | - Aspen Holm
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| | - R Becket Ebitz
- Department of Neuroscience, University of Montreal, Montreal, Quebec H3T 1J4, Canada
| | - Nicola M Grissom
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
| |
Collapse
|
2
|
Chen CS, Knep E, Laurie VJ, Calvin O, Ebitz RB, Fisher M, Schallmo MP, Sponheim SR, Chafee MV, Heilbronner SR, Grissom NM, Redish AD, MacDonald AW, Vinogradov S, Demro C. Beyond reward learning deficits: Exploration-exploitation instability reveals computational heterogeneity in value-based decision making in early psychosis. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2025:2025.04.29.25326698. [PMID: 40343017 PMCID: PMC12060966 DOI: 10.1101/2025.04.29.25326698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 05/11/2025]
Abstract
Psychosis spectrum illnesses are characterized by impaired goal-directed behavior and significant neurophysiological heterogeneity. To investigate the neurocomputational underpinnings of this heterogeneity, 75 participants with Early Psychosis (EP) and 68 controls completed a dynamic decision-making task. Consistent with prior studies, EP exhibited more choice switching, not explained by reward learning deficits, but instead by increased transition to exploration from exploitation. Bayesian modeling implicated elevated uncertainty intolerance and decision noise as independent contributors to suboptimal transition dynamics across individuals, which identified three computational subtypes with unique cognitive and symptom profiles. Replicating prior studies, a high decision-noise subtype emerged showing learning deficits and worse negative symptoms; our analyses further uncovered a normative subtype with worse mood symptoms and a novel uncertainty-intolerance subtype with higher hospitalization rates. These specific microcognitive disruptions underlying the distinct neurocomputational subtypes are individually measurable and may have the potential for targeted interventions.
Collapse
Affiliation(s)
- Cathy S. Chen
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota, United States
| | - Evan Knep
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota, United States
| | | | - Olivia Calvin
- Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, United States
| | - R. Becket Ebitz
- Department of Neurosciences, University of Montréal, Québec, Canada
| | - Melissa Fisher
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
| | - Michael-Paul Schallmo
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
| | - Scott R. Sponheim
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
- Minneapolis VA Health Care System, Minneapolis, MN, United States
| | - Matthew V. Chafee
- Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, United States
| | - Sarah R. Heilbronner
- Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, United States
- Department of Neurosurgery, Baylor College of Medicine, Houston, Texas, United States
| | - Nicola M. Grissom
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota, United States
| | - A. David Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota, United States
| | - Angus W. MacDonald
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota, United States
| | - Sophia Vinogradov
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
| | - Caroline Demro
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, Minnesota, United States
| |
Collapse
|
3
|
Abbaszadeh M, Ozanick E, Magen N, Darrow D, Yan X, Grissom N, Herman AB, Ebitz BR. Individual differences in sequential decision-making. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.04.04.647306. [PMID: 40236038 PMCID: PMC11996512 DOI: 10.1101/2025.04.04.647306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 04/17/2025]
Abstract
1People differ widely in how they make decisions in uncertain environments. While many studies leverage this variability to measure differences in specific cognitive processes and parameters, the key dimension(s) of individual variability in uncertain decision-making tasks has not been identified. Here, we analyzed behavioral data from 1001 participants performing a restless three-armed bandit task, where reward probabilities fluctuated unpredictably over time. Using a novel analytical approach that controlled for the stochasticity in this tasks, we identified a dominant nonlinear axis of individual variability. We found that this primary axis of variability was strongly and selectively correlated with the probability of exploration, as inferred by latent state modeling. This suggests that the major factor shaping individual differences in bandit task performance is the tendency to explore (versus exploit), rather than personality characteristics, reinforcement learning model parameters, or low-level strategies. Certain demographic characteristics also predicted variance along this principle axis: participants at the exploratory end tended to be younger than participants at the exploitative end, and self-identified men were overrepresented at both extremes. Together, these findings offer a principled framework for understanding individual differences in task behavior while highlighting the cognitive and demographic factors that shape individual differences in decision-making under uncertainty.
Collapse
|
4
|
Lloyd A, Roiser JP, Skeen S, Freeman Z, Badalova A, Agunbiade A, Busakhwe C, DeFlorio C, Marcu A, Pirie H, Saleh R, Snyder T, Fearon P, Viding E. Reviewing explore/exploit decision-making as a transdiagnostic target for psychosis, depression, and anxiety. COGNITIVE, AFFECTIVE & BEHAVIORAL NEUROSCIENCE 2024; 24:793-815. [PMID: 38653937 PMCID: PMC11390819 DOI: 10.3758/s13415-024-01186-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/27/2024] [Indexed: 04/25/2024]
Abstract
In many everyday decisions, individuals choose between trialling something novel or something they know well. Deciding when to try a new option or stick with an option that is already known to you, known as the "explore/exploit" dilemma, is an important feature of cognition that characterises a range of decision-making contexts encountered by humans. Recent evidence has suggested preferences in explore/exploit biases are associated with psychopathology, although this has typically been examined within individual disorders. The current review examined whether explore/exploit decision-making represents a promising transdiagnostic target for psychosis, depression, and anxiety. A systematic search of academic databases was conducted, yielding a total of 29 studies. Studies examining psychosis were mostly consistent in showing that individuals with psychosis explored more compared with individuals without psychosis. The literature on anxiety and depression was more heterogenous; some studies found that anxiety and depression were associated with more exploration, whereas other studies demonstrated reduced exploration in anxiety and depression. However, examining a subset of studies that employed case-control methods, there was some evidence that both anxiety and depression also were associated with increased exploration. Due to the heterogeneity across the literature, we suggest that there is insufficient evidence to conclude whether explore/exploit decision-making is a transdiagnostic target for psychosis, depression, and anxiety. However, alongside our advisory groups of lived experience advisors, we suggest that this context of decision-making is a promising candidate that merits further investigation using well-powered, longitudinal designs. Such work also should examine whether biases in explore/exploit choices are amenable to intervention.
Collapse
Affiliation(s)
- Alex Lloyd
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK.
| | - Jonathan P Roiser
- Institute of Cognitive Neuroscience, University College London, London, UK
| | - Sarah Skeen
- Institute for Life Course Health Research, Stellenbosch University, Stellenbosch, South Africa
| | - Ze Freeman
- Department of Psychology, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Aygun Badalova
- Institute of Neurology, University College London, London, UK
| | | | | | | | - Anna Marcu
- Young People's Advisor Group, London, UK
| | | | | | | | - Pasco Fearon
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK
- Centre for Family Research, Department of Psychology, University of Cambridge, Cambridge, UK
| | - Essi Viding
- Clinical, Educational and Health Psychology, Psychology and Language Sciences, University College London, 26 Bedford Way, London, WC1H 0AP, UK
| |
Collapse
|
5
|
Yan X, Ebitz RB, Grissom N, Darrow DP, Herman AB. Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.04.597412. [PMID: 38895240 PMCID: PMC11185698 DOI: 10.1101/2024.06.04.597412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
Decision-making in uncertain environments often leads to varied outcomes. Understanding how individuals interpret the causes of unexpected feedback is crucial for adaptive behavior and mental well-being. Uncertainty can be broadly categorized into two components: volatility and stochasticity. Volatility is about how quickly conditions change, impacting results. Stochasticity, on the other hand, refers to outcomes affected by random chance or "luck". Understanding these factors enables individuals to have more effective environmental analysis and strategy implementation (explore or exploit) for future decisions. This study investigates how anxiety and apathy, two prevalent affective states, influence the perceptions of uncertainty and exploratory behavior. Participants (N = 1001) completed a restless three-armed bandit task that was analyzed using latent state models. Anxious individuals perceived uncertainty as more volatile, leading to increased exploration and learning rates, especially after reward omission. Conversely, apathetic individuals viewed uncertainty as more stochastic, resulting in decreased exploration and learning rates. The perceived volatility-to-stochasticity ratio mediated the anxiety-exploration relationship post-adverse outcomes. Dimensionality reduction showed exploration and uncertainty estimation to be distinct but related latent factors shaping a manifold of adaptive behavior that is modulated by anxiety and apathy. These findings reveal distinct computational mechanisms for how anxiety and apathy influence decision-making, providing a framework for understanding cognitive and affective processes in neuropsychiatric disorders.
Collapse
Affiliation(s)
- Xinyuan Yan
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| | - R. Becket Ebitz
- Department of Neuroscience, Universite de Montreal, 2900 Edouard Montpetit Blvd, Montreal, Quebec H3T 1J4, Canada
| | - Nicola Grissom
- Department of Psychology, University of Minnesota, 75 E River Rd, Minneapolis, MN 55455, USA
| | - David P. Darrow
- Department of Neurosurgery, University of Minnesota, Minneapolis, MN 55455, USA
| | - Alexander B. Herman
- Department of Psychiatry and Behavioral Sciences, University of Minnesota, Minneapolis, MN 55455, USA
| |
Collapse
|
6
|
Chen CS, Mueller D, Knep E, Ebitz RB, Grissom NM. Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.09.523322. [PMID: 36711959 PMCID: PMC9881999 DOI: 10.1101/2023.01.09.523322] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.
Collapse
|
7
|
Yan X, Ebitz RB, Grissom N, Darrow DP, Herman AB. A low dimensional manifold of human exploratory behavior reveals opposing roles for apathy and anxiety. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.06.19.545645. [PMID: 37425723 PMCID: PMC10327047 DOI: 10.1101/2023.06.19.545645] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/11/2023]
Abstract
Exploration-exploitation decision-making is a feature of daily life that is altered in a number of neuropsychiatric conditions. Humans display a range of exploration and exploitation behaviors, which can be affected by apathy and anxiety. It remains unknown how factors underlying decision-making generate the spectrum of observed exploration-exploitation behavior and how they relate to states of anxiety and apathy. Here, we report a latent structure underlying sequential exploration and exploitation decisions that explains variation in anxiety and apathy. 1001 participants in a gender-balanced sample completed a three-armed restless bandit task along with psychiatric symptom surveys. Using dimensionality reduction methods, we found that decision sequences reduced to a low-dimensional manifold. The axes of this manifold explained individual differences in the balance between states of exploration and exploitation and the stability of those states, as determined by a statistical mechanics model of decision-making. Position along the balance axis was correlated with opposing symptoms of behavioral apathy and anxiety, while position along the stability axis correlated with the level of emotional apathy. This result resolves a paradox over how these symptoms can be correlated in samples but have opposite effects on behavior. Furthermore, this work provides a basis for using behavioral manifolds to reveal relationships between behavioral dynamics and affective states, with important implications for behavioral measurement approaches to neuropsychiatric conditions.
Collapse
|
8
|
Shourkeshti A, Marrocco G, Jurewicz K, Moore T, Ebitz RB. Pupil size predicts the onset of exploration in brain and behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.24.541981. [PMID: 37292773 PMCID: PMC10245915 DOI: 10.1101/2023.05.24.541981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
In uncertain environments, intelligent decision-makers exploit actions that have been rewarding in the past, but also explore actions that could be even better. Several neuromodulatory systems are implicated in exploration, based, in part, on work linking exploration to pupil size-a peripheral correlate of neuromodulatory tone and index of arousal. However, pupil size could instead track variables that make exploration more likely, like volatility or reward, without directly predicting either exploration or its neural bases. Here, we simultaneously measured pupil size, exploration, and neural population activity in the prefrontal cortex while two rhesus macaques explored and exploited in a dynamic environment. We found that pupil size under constant luminance specifically predicted the onset of exploration, beyond what could be explained by reward history. Pupil size also predicted disorganized patterns of prefrontal neural activity at both the single neuron and population levels, even within periods of exploitation. Ultimately, our results support a model in which pupil-linked mechanisms promote the onset of exploration via driving the prefrontal cortex through a critical tipping point where prefrontal control dynamics become disorganized and exploratory decisions are possible.
Collapse
Affiliation(s)
- Akram Shourkeshti
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| | - Gabriel Marrocco
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| | - Katarzyna Jurewicz
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
- Department of Physiology, McGill University, Montréal, QC, Canada
| | - Tirin Moore
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, USA
- Howard Hughes Medical Institute, Chevy Chase, MD, USA
| | - R. Becket Ebitz
- Department of Neurosciences, Université de Montréal, Montréal, QC, Canada
| |
Collapse
|