Working memory contributions to reinforcement learning impairments in schizophrenia

Developing algorithmic psychiatry via multi-level spanning computational models

Modern psychiatry faces challenges in translating neurobiological insights into treatments for severe illnesses. The mid-20th century witnessed the rise of molecular mechanisms as pathophysiological and treatment models, with recent holistic proposals keeping this focus unaltered. In this perspective, we explore how psychiatry can utilize systems neuroscience to develop a vertically integrated understanding of brain function to inform treatment. Using schizophrenia as a case study, we discuss scale-related challenges faced by researchers studying molecules, circuits, networks, and cognition and clinicians operating within existing frameworks. We emphasize computation as a bridging language, with algorithmic models like hierarchical predictive processing offering explanatory potential for targeted interventions. Developing such models will not only facilitate new interventions but also optimize combining existing treatments by predicting their multi-level effects. We conclude with the prognosis that the future is bright, but that continued investment in research closely driven by clinical realities will be critical.

Evidence for shallow cognitive maps in Schizophrenia

Individuals with schizophrenia can have marked deficits in goal-directed decision making. Prominent theories differ in whether schizophrenia (SZ) affects the ability to exert cognitive control or the motivation to exert control. An alternative explanation is that schizophrenia negatively impacts the formation of cognitive maps, the internal representations of the way the world is structured, necessary for the formation of effective action plans. That is, deficits in decision-making could arise when goal-directed control and motivation are intact but used to plan over ill-formed maps. We tested the hypothesis that individuals with SZ are impaired in constructing cognitive maps. We combine a behavioral representational similarity analysis technique with a sequential decision-making task. This enables us to examine how relationships between choice options change when individuals with SZ and healthy age-matched controls build a cognitive map of the task structure. Our results indicate that SZ affects how people represent the structure of the task, focusing more on simpler visual features and less on abstract, higher-order, planning-relevant features. At the same time, we find that individuals with SZ were able to display similar performance on this task compared with controls, emphasizing the need for a distinction between cognitive map formation and changes in goal-directed control in understanding cognitive deficits in schizophrenia.

Neural correlates of uncertainty processing in psychosis spectrum disorder

Psychotic beliefs are typically held with high certainty. Altered computation of uncertainty about a belief and about environmental dynamics may be an underlying mechanism of psychotic symptoms. We set out to shed light on behavioural and neural correlates of uncertainty processing and how it drives belief updating in psychosis. This cross-sectional study included 19 participants with psychosis spectrum disorder (5 female and 14 male) and 40 healthy control participants (21 female and 19 male) between 18 and 65 years of age. Participants performed a predictive inference task that required belief updating of a noisy outcome in a suddenly changing environment during functional magnetic resonance imaging. Behavioural and imaging data were analysed with a computational model that approximates an ideal Bayesian observer. The model expects beliefs to be updated based on the relative belief uncertainty and environmental change point probability. Task performance, model parameters and associated neural activation were compared between groups and associated with self-reported delusional ideation and cognitive functioning. While the belief updating speed overall did not differ between groups, the psychosis group showed lower task performance. Lower performance was associated with higher self-reported delusional ideation, even when controlling for cognitive functioning. Persons with psychosis spectrum disorder tended to persevere on beliefs after large prediction errors that signal environmental changes. They informed belief updates less by the probability of environmental change points, although this capacity seemed to depend on general cognitive functioning. The psychosis group also encoded the change point probability less in the superior occipital and fusiform gyrus, as well as a cluster comprising pre-central to middle frontal gyrus. Activity in these clusters was associated with lower self-reported delusional ideation across the whole sample and lower general and negative symptoms in the clinical sample. Persons with psychosis spectrum disorder did not seem to overestimate environmental volatility in general. Instead, they showed altered processing of information that occurred after environmental change points, whose probability was less well represented in brain regions encoding visual surprise and motor responses. Possibly, persons with psychosis spectrum disorder inadequately integrated visual surprise signals, leading to ineffective transmission to motor regions that eventually guide behaviour. Summarizing, our study suggests that delusions could result from a tendency to stick to old beliefs even in the light of contrary evidence, due to a failure to integrate uncertainty information based on inferred environmental dynamics.

Probabalistic reinforcement learning impairments predict negative symptom severity and risk for conversion in youth at clinical high-risk for psychosis

BACKGROUND

Elucidation of transphasic mechanisms (i.e., mechanisms that occur across illness phases) underlying negative symptoms could inform early intervention and prevention efforts and additionally identify treatment targets that could be effective regardless of illness stage. This study examined whether a key reinforcement learning behavioral pattern characterized by reduced difficulty learning from rewards that have been found to underlie negative symptoms in those with a schizophrenia diagnosis also contributes to negative symptoms in those at clinical high-risk (CHR) for psychosis.

METHODS

CHR youth (n = 46) and 51 healthy controls (CN) completed an explicit reinforcement learning task with two phases. During the acquisition phase, participants learned to select between pairs of stimuli probabilistically reinforced with feedback indicating receipt of monetary gains or avoidance of losses. Following training, the transfer phase required participants to select between pairs of previously presented stimuli during the acquisition phase and novel stimuli without receiving feedback. These test phase pairings allowed for inferences about the contributions of prediction error and value representation mechanisms to reinforcement learning deficits.

RESULTS

In acquisition, CHR participants displayed impaired learning from gains specifically that were associated with greater negative symptom severity. Transfer performance indicated these acquisition deficits were largely driven by value representation deficits. In addition to negative symptoms, this profile of deficits was associated with a greater risk of conversion to psychosis and lower functioning.

CONCLUSIONS

Impairments in positive reinforcement learning, specifically effectively representing reward value, may be an important transphasic mechanism of negative symptoms and a marker of psychosis liability.

Disentangling negative reinforcement, working memory, and deductive reasoning deficits in elevated BMI

Neuropsychological data suggest that being overweight or obese is associated with a tendency to perseverate behavior despite negative feedback. This deficit might be observed due to other cognitive factors, such as working memory (WM) deficits or decreased ability to deduce model-based strategies when learning by trial-and-error. In the present study, a group of subjects with overweight or obesity (Ow/Ob, n = 30) was compared to normal-weight individuals (n = 42) in a modified Reinforcement Learning (RL) task. The task was designed to control WM effects on learning by manipulating cognitive load and to foster model-based learning via deductive reasoning. Computational modelling and analysis were conducted to isolate parameters related to RL mechanisms, WM use, and model-based learning (deduction parameter). Results showed that subjects with Ow/Ob had a higher number of perseverative errors and used a weaker deduction mechanism in their performance than control individuals, indicating impairments in negative reinforcement and model-based learning, whereas WM impairments were not responsible for deficits in RL. The present data suggests that obesity is associated with impairments in negative reinforcement and model-based learning.

Working Memory Guides Action Valuation in Model-based Decision-making Strategy

Humans use both model-free (or habitual) and model-based (or goal-directed) strategies in sequential decision-making. Working memory (WM) is essential for the model-based strategy; however, its exact role in these processes remains elusive. This study investigates the influence of WM processes on decision-making and the underlying cognitive computing mechanisms. Specifically, we used experimental data from two-stage decision tasks and found that delay and load, two WM-specific variables, impact goal-revisiting behaviors. Then, we proposed possible computational mechanisms by which WM participates in information processing and integrated them into the model-based system. The proposed Hybrid-WM model reproduced the observed experimental effects and fit human behavior better than the classic hybrid reinforcement learning model. These results were verified with independent data sets. Furthermore, differences in model parameters explain the age-related difference in sequential decision-making. Overall, this study suggests that WM guides action valuation in model-based strategies, highlighting the contribution of higher cognitive functions to sequential decision-making.

Modeling Decision-Making in Schizophrenia: Associations Between Computationally Derived Risk Propensity and Self-Reported Risk Perception

BACKGROUND AND HYPOTHESIS

Schizophrenia is associated with a decreased pursuit of risky rewards during uncertain-risk decision-making. However, putative mechanisms subserving this disadvantageous risky reward pursuit, such as contributions of cognition and relevant traits, remain poorly understood.

STUDY DESIGN

Participants (30 schizophrenia/schizoaffective disorder [SZ]; 30 comparison participants [CP]) completed the Balloon Analogue Risk Task (BART). Computational modeling captured subprocesses of uncertain-risk decision-making: Risk Propensity, Prior Belief of Success, Learning Rate, and Behavioral Consistency. IQ, self-reported risk-specific processes (ie, Perceived Risks and Expected Benefit of Risks), and non-risk-specific traits (ie, defeatist beliefs; hedonic tone) were examined for relationships with Risk Propensity to determine what contributed to differences in risky reward pursuit.

STUDY RESULTS

On the BART, the SZ group exhibited lower Risk Propensity, higher Prior Beliefs of Success, and comparable Learning Rates. Furthermore, Risk Propensity was positively associated with IQ across groups. Linear models predicting Risk Propensity revealed 2 interactions: 1 between group and Perceived Risk, and 1 between IQ and Perceived Risk. Specifically, in both the SZ group and individuals with below median IQ, lower Perceived Risks was related to lower Risk Propensity. Thus, lower perception of financial risks was associated with a less advantageous pursuit of uncertain-risk rewards.

CONCLUSIONS

Findings suggest consistently decreased risk-taking on the BART in SZ may reflect risk imperception, the failure to accurately perceive and leverage relevant information to guide the advantageous pursuit of risky rewards. Additionally, our results highlight the importance of cognition in uncertain-risk decision-making.

Effect of prior beliefs and cognitive deficits on learning in first-episode schizophrenia patients

Introduction

It is known that cognitive deficits are a core feature of schizophrenia and that in the general population, prior beliefs significantly influence learning and reasoning processes. However, the interaction of prior beliefs with cognitive deficits and their impact on performance in schizophrenia patients is still poorly understood. This study investigates the role of beliefs and cognitive variables (CVs) like working memory, associative learning, and processing speed on learning processes in individuals with schizophrenia. We hypothesize that beliefs will influence the ability to learn correct predictions and that first-episode schizophrenia patients (FEP) will show impaired learning due to cognitive deficits.

Methods

We used a predictive-learning task to examine how FEP (n = 23) and matched controls (n = 23) adjusted their decisional criteria concerning physical properties during the learning process when predicting the sinking behavior of two transparent containers filled with aluminum discs when placed in water.

Results

On accuracy, initial differences by group, trial type, and interaction effects of these variables disappeared when CVs were controlled. The differences by conditions, associated with differential beliefs about why the objects sink slower or faster, were seen in patients and controls, despite controlling the CVs' effect.

Conclusions

Differences between groups were mainly explained by CVs, proving that they play an important role than what is assumed in this type of task. However, beliefs about physical events were not affected by CVs, and beliefs affect in the same way the decisional criteria of the control or FEP patients' groups.

Levo-Stepholidine as a Potential Cognitive Enhancer: Insights into Executive Function and Memory Improvements

BACKGROUND/OBJECTIVES

Levo-Stepholidine (l-SPD), a compound extracted from Chinese herbs, has the potential to treat psychotic disorders where cognitive deficits are a critical challenge. L-SPD displays a D1R agonism/D2R antagonism pharmacological profile, and its effect on cognitive function is still vague and lacks comprehensive study. Here, we investigated the impact of l-SPD on two core indexes of executive function, working memory and response inhibition, and learning and memory.

METHODS

Using a delayed alternation T-maze task (DAT), we investigated the impact of l-SPD on working memory, evaluated its effect on response inhibition using the stop-signal task (SST), and assessed the impact on learning and memory using trace fear conditioning in Sprague-Dawley rats. We further evaluated its effects on prefrontal glutamate receptor expression using western blot.

RESULTS

Rats receiving l-SPD made fewer errors in the T-maze, exhibited faster stop action in response to the stop signal, and showed longer-lasting memory retention. Molecular mechanism investigations reveal that l-SPD upregulates the expression of prefrontal glutamate receptors. These results demonstrate that l-SPD improves executive function and memory.

CONCLUSIONS

Here, we show the enhancement effect of l-SPD on cognitive function, which provides essential implicants for the treatment of cognitive deficits, which is a critical unmet need in psychiatric care.

Working memory as a representational template for reinforcement learning

Working memory (WM) and reinforcement learning (RL) both influence decision-making, but how they interact to affect behaviour remains unclear. We assessed whether RL is influenced by the format of visual stimuli held in WM, either feature-based or unified, object-based representations. In a pre-registered paradigm, participants learned stimulus-action combinations that provided reward through 80% probabilistic feedback. In parallel, participants retained the RL stimulus in WM and were asked to recall this stimulus after each RL choice. Crucially, the format of representation probed in WM was manipulated, with blocks encouraging either separate features or bound objects to be remembered. Incentivising a feature-based WM representation facilitated feature-based learning, shown by an improved choice strategy. This reveals a role of WM in providing sustained internal representations that are harnessed by RL, providing a framework by which these two cognitive processes cooperate.

Working memory gating in obesity is moderated by striatal dopaminergic gene variants

Everyday life requires an adaptive balance between distraction-resistant maintenance of information and the flexibility to update this information when needed. These opposing mechanisms are proposed to be balanced through a working memory gating mechanism. Prior research indicates that obesity may elevate the risk of working memory deficits, yet the underlying mechanisms remain elusive. Dopaminergic alterations have emerged as a potential mediator. However, current models suggest these alterations should only shift the balance in working memory tasks, not produce overall deficits. The empirical support for this notion is currently lacking, however. To address this gap, we pooled data from three studies (N = 320) where participants performed a working memory gating task. Higher BMI was associated with overall poorer working memory, irrespective of whether there was a need to maintain or update information. However, when participants, in addition to BMI level, were categorized based on certain putative dopamine-signaling characteristics (single-nucleotide polymorphisms [SNPs]; specifically, Taq1A and DARPP-32), distinct working memory gating effects emerged. These SNPs, primarily associated with striatal dopamine transmission, appear to be linked with differences in updating, specifically, among high-BMI individuals. Moreover, blood amino acid ratio, which indicates central dopamine synthesis capacity, combined with BMI shifted the balance between distractor-resistant maintenance and updating. These findings suggest that both dopamine-dependent and dopamine-independent cognitive effects exist in obesity. Understanding these effects is crucial if we aim to modify maladaptive cognitive profiles in individuals with obesity.

Global reward processing deficits predict negative symptoms transdiagnostically and transphasically in a severe mental illness-spectrum sample

Reward processing impairments are a key factor associated with negative symptoms in those with severe mental illnesses. However, past findings are inconsistent regarding which reward processing components are impaired and most strongly linked to negative symptoms. The current study examined the hypothesis that these mixed findings may be the result of multiple reward processing pathways (i.e., equifinality) to negative symptoms that cut across diagnostic boundaries and phases of illness. Participants included healthy controls (n = 100) who served as a reference sample and a severe mental illness-spectrum sample (n = 92) that included psychotic-like experiences, clinical high-risk for psychosis, bipolar disorder, and schizophrenia participants. All participants completed tasks measuring four RDoC Positive Valence System constructs: value representation, reinforcement learning, effort-cost computation, and hedonic reactivity. A k-means cluster analysis of the severe mental illness-spectrum samples identified three clusters with differential reward processing profiles that were characterized by: (1) global reward processing deficits (22.8%), (2) selective impairments in hedonic reactivity alone (40.2%), and (3) preserved reward processing (37%). Elevated negative symptoms were only observed in the global reward processing cluster. All clusters contained participants from each clinical group, and the distribution of these groups did not significantly differ among the clusters. Findings identified one pathway contributing to negative symptoms that was transdiagnostic and transphasic. Future work further characterizing divergent pathways to negative symptoms may help to improve symptom trajectories and personalized treatments.

Resource-rational psychopathology

Psychopathology is vast and diverse. Across distinct disease states, individuals exhibit symptoms that appear counter to the standard view of rationality (expected utility maximization). We argue that some aspects of psychopathology can be described as resource-rational, reflecting a rational trade-off between reward and cognitive resources. We review work on two theories of this kind: rational inattention, where a capacity limit applies to perceptual channels, and policy compression, where the capacity limit applies to action channels. We show how these theories can parsimoniously explain many forms of psychopathology, including affective, primary psychotic, and neurodevelopmental disorders, as well as many effects of psychoactive medications on these disorders. While there are important disorder-specific differences and the theories are by no means universal, we argue that resource rationality offers a useful new perspective on psychopathology. By emphasizing the role of cognitive resource constraints, this approach offers a more inclusive picture of rationality. Some aspects of psychopathology may reflect rational trade-offs rather than the breakdown of rationality. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Trying Harder: How Cognitive Effort Sculpts Neural Representations during Working Memory

While the exertion of mental effort improves performance on cognitive tasks, the neural mechanisms by which motivational factors impact cognition remain unknown. Here, we used fMRI to test how changes in cognitive effort, induced by changes in task difficulty, impact neural representations of working memory (WM). Participants (both sexes) were precued whether WM difficulty would be hard or easy. We hypothesized that hard trials demanded more effort as a later decision required finer mnemonic precision. Behaviorally, pupil size was larger and response times were slower on hard compared with easy trials suggesting our manipulation of effort succeeded. Neurally, we observed robust persistent activity during delay periods in the prefrontal cortex (PFC), especially during hard trials. Yet, details of the memoranda could not be decoded from patterns in prefrontal activity. In the patterns of activity in the visual cortex, however, we found strong decoding of memorized targets, where accuracy was higher on hard trials. To potentially link these across-region effects, we hypothesized that effort, carried by persistent activity in the PFC, impacts the quality of WM representations encoded in the visual cortex. Indeed, we found that the amplitude of delay period activity in the frontal cortex predicted decoded accuracy in the visual cortex on a trial-wise basis. These results indicate that effort-related feedback signals sculpt population activity in the visual cortex, improving mnemonic fidelity.

The challenge of learning adaptive mental behavior

Many psychotherapies aim to help people replace maladaptive mental behaviors (such as those leading to unproductive worry) with more adaptive ones (such as those leading to active problem solving). Yet, little is known empirically about how challenging it is to learn adaptive mental behaviors. Mental behaviors entail taking mental operations and thus may be more challenging to perform than motor actions; this challenge may enhance or impair learning. In particular, challenge when learning is often desirable because it improves retention. Yet, it is also plausible that the necessity of carrying out mental operations interferes with learning the expected values of mental actions by impeding credit assignment: the process of updating an action's value after reinforcement. Then, it may be more challenging not only to perform-but also to learn the consequences of-mental (vs. motor) behaviors. We designed a task to assess learning to take adaptive mental versus motor actions via matched probabilistic feedback. In two experiments (N = 300), most participants found it more difficult to learn to select optimal mental (vs. motor) actions, as evident in worse accuracy not only in a learning but also test (retention) phase. Computational modeling traced this impairment to an indicator of worse credit assignment (impaired construction and maintenance of expected values) when learning mental actions, accounting for worse accuracy in the learning and retention phases. The results suggest that people have particular difficulty learning adaptive mental behavior and pave the way for novel interventions to scaffold credit assignment and promote adaptive thinking. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Dysfunctional feedback processing in male methamphetamine abusers: Evidence from neurophysiological and computational approaches

Methamphetamine use disorder (MUD) as a major public health risk is associated with dysfunctional neural feedback processing. Although dysfunctional feedback processing in people who are substance dependent has been explored in several behavioral, computational, and electrocortical studies, this mechanism in MUDs requires to be well understood. Furthermore, the current understanding of latent components of their behavior such as learning speed and exploration-exploitation dilemma is still limited. In addition, the association between the latent cognitive components and the related neural mechanisms also needs to be explored. Therefore, in this study, the underlying neurocognitive mechanisms of feedback processing of such impairment, and age/gender-matched healthy controls are evaluated within a probabilistic learning task with rewards and punishments. Mathematical modeling results based on the Q-learning paradigm suggested that MUDs show less sensitivity in distinguishing optimal options. Additionally, it may be worth noting that MUDs exhibited a slight decrease in their ability to learn from negative feedback compared to healthy controls. Also through the lens of underlying neural mechanisms, MUDs showed lower theta power at the medial-frontal areas while responding to negative feedback. However, other EEG measures of reinforcement learning including feedback-related negativity, parietal-P300, and activity flow from the medial frontal to lateral prefrontal regions, remained intact in MUDs. On the other hand, the elimination of the linkage between value sensitivity and medial-frontal theta activity in MUDs was observed. The observed dysfunction could be due to the adverse effects of methamphetamine on the cortico-striatal dopamine circuit, which is reflected in the anterior cingulate cortex activity as the most likely region responsible for efficient behavior adjustment. These findings could help us to pave the way toward tailored therapeutic approaches.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Artificial neural networks for model identification and parameter estimation in computational cognitive models

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Associative learning and facial expression recognition in schizophrenic patients: Effects of social presence

Schizophrenia (SCZ) is a psychiatric disorder that alters both general and social cognition. However, the exact mechanisms that are altered remain to be elucidated. In this study, we investigated associative learning (AL) and facial expression recognition (FER) in the same patients, using emotional expressions and abstract images. Our main aim was to investigate how these cognitive abilities are affected by SCZ and to assess the role of mere social presence, a factor that has not been considered before. The study compared the behavioral performance of 60 treated outpatients with SCZ and 103 demographically matched healthy volunteers. In the AL task, participants had to associate abstract images or facial expressions with key presses, guided by feedback on each trial. In the FER task, they had to report whether two successively presented facial expressions were the same or different. All participants performed the two tasks under two social context conditions: alone or in the presence of a passive but attentive audience. The results showed a severe learning impairment in patients compared to controls, with a slight advantage for facial expressions compared to abstract images, and a gender-dependent effect of social presence. In contrast, facial expression recognition was partially spared in patients and facilitated by social presence. We conclude that cognitive abilities are impaired in patients with SCZ, but their investigation needs to take into account the social context in which they are assessed.

Evidence for shallow cognitive maps in schizophrenia

Individuals with schizophrenia can have marked deficits in goal-directed decision making. Prominent theories differ in whether schizophrenia (SZ) affects the ability to exert cognitive control, or the motivation to exert control. An alternative explanation is that schizophrenia negatively impacts the formation of cognitive maps, the internal representations of the way the world is structured, necessary for the formation of effective action plans. That is, deficits in decision-making could also arise when goal-directed control and motivation are intact, but used to plan over ill-formed maps. Here, we test the hypothesis that individuals with SZ are impaired in the construction of cognitive maps. We combine a behavioral representational similarity analysis technique with a sequential decision-making task. This enables us to examine how relationships between choice options change when individuals with SZ and healthy age-matched controls build a cognitive map of the task structure. Our results indicate that SZ affects how people represent the structure of the task, focusing more on simpler visual features and less on abstract, higher-order, planning-relevant features. At the same time, we find that SZ were able to display similar performance on this task compared to controls, emphasizing the need for a distinction between cognitive map formation and changes in goal-directed control in understanding cognitive deficits in schizophrenia.

Exploring the dynamic interplay between learning and working memory within various cognitive contexts

Introduction

The intertwined relationship between reinforcement learning and working memory in the brain is a complex subject, widely studied across various domains in neuroscience. Research efforts have focused on identifying the specific brain areas responsible for these functions, understanding their contributions in accomplishing the related tasks, and exploring their adaptability under conditions such as cognitive impairment or aging.

Methods

Numerous models have been introduced to formulate either these two subsystems of reinforcement learning and working memory separately or their combination and relationship in executing cognitive tasks. This study adopts the RLWM model as a computational framework to analyze the behavioral parameters of subjects with varying cognitive abilities due to age or cognitive status. A related RLWM task is employed to assess a group of subjects across different age groups and cognitive abilities, as measured by the Montreal Cognitive Assessment tool (MoCA).

Results

Analysis reveals a decline in overall performance accuracy and speed with differing age groups (young vs. middle-aged). Significant differences are observed in model parameters such as learning rate, WM decay, and decision noise. Furthermore, among the middle-aged group, distinctions emerge between subjects categorized as normal vs. MCI based on MoCA scores, notably in speed, performance accuracy, and decision noise.

Naturalistic reinforcement learning

Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.

Out of control: computational dynamic control dysfunction in stress- and anxiety-related disorders

Control theory, which has played a central role in technological progress over the last 150 years, has also yielded critical insights into biology and neuroscience. Recently, there has been a surging interest in integrating control theory with computational psychiatry. Here, we review the state of the field of using control theory approaches in computational psychiatry and show that recent research has mapped a neural control circuit consisting of frontal cortex, parietal cortex, and the cerebellum. This basic feedback control circuit is modulated by estimates of reward and cost via the basal ganglia as well as by arousal states coordinated by the insula, dorsal anterior cingulate cortex, amygdala, and locus coeruleus. One major approach within the broader field of control theory, known as proportion-integral-derivative (PID) control, has shown promise as a model of human behavior which enables precise and reliable estimates of underlying control parameters at the individual level. These control parameters correlate with self-reported fear and with both structural and functional variation in affect-related brain regions. This suggests that dysfunctional engagement of stress and arousal systems may suboptimally modulate parameters of domain-general goal-directed control algorithms, impairing performance in complex tasks involving movement, cognition, and affect. Future directions include clarifying the causal role of control deficits in stress- and anxiety-related disorders and developing clinically useful tools based on insights from control theory.

Transdiagnostic inflexible learning dynamics explain deficits in depression and schizophrenia

Deficits in reward learning are core symptoms across many mental disorders. Recent work suggests that such learning impairments arise by a diminished ability to use reward history to guide behaviour, but the neuro-computational mechanisms through which these impairments emerge remain unclear. Moreover, limited work has taken a transdiagnostic approach to investigate whether the psychological and neural mechanisms that give rise to learning deficits are shared across forms of psychopathology. To provide insight into this issue, we explored probabilistic reward learning in patients diagnosed with major depressive disorder (n = 33) or schizophrenia (n = 24) and 33 matched healthy controls by combining computational modelling and single-trial EEG regression. In our task, participants had to integrate the reward history of a stimulus to decide whether it is worthwhile to gamble on it. Adaptive learning in this task is achieved through dynamic learning rates that are maximal on the first encounters with a given stimulus and decay with increasing stimulus repetitions. Hence, over the course of learning, choice preferences would ideally stabilize and be less susceptible to misleading information. We show evidence of reduced learning dynamics, whereby both patient groups demonstrated hypersensitive learning (i.e. less decaying learning rates), rendering their choices more susceptible to misleading feedback. Moreover, there was a schizophrenia-specific approach bias and a depression-specific heightened sensitivity to disconfirmational feedback (factual losses and counterfactual wins). The inflexible learning in both patient groups was accompanied by altered neural processing, including no tracking of expected values in either patient group. Taken together, our results thus provide evidence that reduced trial-by-trial learning dynamics reflect a convergent deficit across depression and schizophrenia. Moreover, we identified disorder distinct learning deficits.

Reinforcement learning and working memory in mood disorders: A computational analysis in a developmental transdiagnostic sample

BACKGROUND

Mood disorders commonly onset during adolescence and young adulthood and are conceptually and empirically related to reinforcement learning abnormalities. However, the nature of abnormalities associated with acute symptom severity versus lifetime diagnosis remains unclear, and prior research has often failed to disentangle working memory from reward processes.

METHODS

The present sample (N = 220) included adolescents and young adults with a lifetime history of unipolar disorders (n = 127), bipolar disorders (n = 28), or no history of psychopathology (n = 62), and varying severity of mood symptoms. Analyses fitted a reinforcement learning and working memory model to an instrumental learning task that varied working memory load, and tested associations between model parameters and diagnoses or current symptoms.

RESULTS

Current severity of manic or anhedonic symptoms negatively correlated with task performance. Participants reporting higher severity of current anhedonia, or with lifetime unipolar or bipolar disorders, showed lower reward learning rates. Participants reporting higher severity of current manic symptoms showed faster working memory decay and reduced use of working memory.

LIMITATIONS

Computational parameters should be interpreted in the task environment (a deterministic reward learning paradigm), and developmental population. Future work should test replication in other paradigms and populations.

CONCLUSIONS

Results indicate abnormalities in reinforcement learning processes that either scale with current symptom severity, or correspond with lifetime mood diagnoses. Findings may have implications for understanding reward processing anomalies related to state-like (current symptom) or trait-like (lifetime diagnosis) aspects of mood disorders.

A novel technique for delineating the effect of variation in the learning rate on the neural correlates of reward prediction errors in model-based fMRI

Introduction

Computational models play an increasingly important role in describing variation in neural activation in human neuroimaging experiments, including evaluating individual differences in the context of psychiatric neuroimaging. In particular, reinforcement learning (RL) techniques have been widely adopted to examine neural responses to reward prediction errors and stimulus or action values, and how these might vary as a function of clinical status. However, there is a lack of consensus around the importance of the precision of free parameter estimation for these methods, particularly with regard to the learning rate. In the present study, I introduce a novel technique which may be used within a general linear model (GLM) to model the effect of mis-estimation of the learning rate on reward prediction error (RPE)-related neural responses.

Methods

Simulations employed a simple RL algorithm, which was used to generate hypothetical neural activations that would be expected to be observed in functional magnetic resonance imaging (fMRI) studies of RL. Similar RL models were incorporated within a GLM-based analysis method including derivatives, with individual differences in the resulting GLM-derived beta parameters being evaluated with respect to the free parameters of the RL model or being submitted to other validation analyses.

Results

Initial simulations demonstrated that the conventional approach to fitting RL models to RPE responses is more likely to reflect individual differences in a reinforcement efficacy construct (lambda) rather than learning rate (alpha). The proposed method, adding a derivative regressor to the GLM, provides a second regressor which reflects the learning rate. Validation analyses were performed including examining another comparable method which yielded highly similar results, and a demonstration of sensitivity of the method in presence of fMRI-like noise.

Conclusion

Overall, the findings underscore the importance of the lambda parameter for interpreting individual differences in RPE-coupled neural activity, and validate a novel neural metric of the modulation of such activity by individual differences in the learning rate. The method is expected to find application in understanding aberrant reinforcement learning across different psychiatric patient groups including major depression and substance use disorder.

Trying harder: how cognitive effort sculpts neural representations during working memory

The neural mechanisms by which motivational factors influence cognition remain unknown. Using fMRI, we tested how cognitive effort impacts working memory (WM). Participants were precued whether WM difficulty would be hard or easy. Hard trials demanded more effort as a later decision required finer mnemonic precision. Behaviorally, pupil size was larger and response times were slower on hard trials suggesting our manipulation of effort succeeded. Neurally, we observed robust persistent activity in prefrontal cortex, especially during hard trials. We found strong decoding of location in visual cortex, where accuracy was higher on hard trials. Connecting these across-region effects, we found that the amplitude of delay period activity in frontal cortex predicted decoded accuracy in visual cortex on a trial-wise basis. We conclude that the gain of persistent activity in frontal cortex may be the source of effort-related feedback signals that improve the quality of WM representations stored in visual cortex.

Back to the future: Progressing memory research in eating disorders

OBJECTIVE

Human behaviors, thoughts, and emotions are guided by memories of the past. Thus, there can be little doubt that memory plays a fundamental role in the behaviors (e.g., binging), thoughts (e.g., body-image concerns), and emotions (e.g., guilt) that characterize eating disorders (EDs). Although a growing body of research has begun to investigate the role of memory in EDs, this literature is limited in numerous ways and has yet to be integrated into an overarching framework.

METHODS

In the present article, we provide an operational framework for characterizing different domains of memory, briefly review existing ED memory research within this framework, and highlight crucial gaps in the literature.

RESULTS

We distinguish between three domains of memory-episodic, procedural, and working-which differ based on functional attributes and underlying neural systems. Most recent ED memory research has focused on procedural memory broadly defined (e.g., reinforcement learning), and findings within all three memory domains are highly mixed. Further, few studies have attempted to assess these different domains simultaneously, though most behavior is achieved through coordination and competition between memory systems. We, therefore, offer recommendations for how to move ED research forward within each domain of memory and how to study the interactions between memory systems, using illustrative examples from other areas of basic and clinical research.

DISCUSSION

A stronger and more integrated understanding of the mechanisms that connect memory of past experiences to present ED behavior may yield more comprehensive theoretical models of EDs that guide novel treatment approaches.

PUBLIC SIGNIFICANCE

Memories of previous eating-related experiences may contribute to the onset and maintenance of eating disorders (EDs). However, research on the role of memory in EDs is limited, and distinct domains of ED memory research are rarely connected. We, therefore, offer a framework for organizing, progressing, and integrating ED memory research, to provide a better foundation for improving ED treatment and intervention going forward.

Switching to online: Testing the validity of supervised remote testing for online reinforcement learning experiments

Online experiments are an alternative for researchers interested in conducting behavioral research outside the laboratory. However, an online assessment might become a challenge when long and complex experiments need to be conducted in a specific order or with supervision from a researcher. The aim of this study was to test the computational validity and the feasibility of a remote and synchronous reinforcement learning (RL) experiment conducted during the social-distancing measures imposed by the pandemic. An additional feature of this study was to describe how a behavioral experiment originally created to be conducted in-person was transformed into an online supervised remote experiment. Open-source software was used to collect data, conduct statistical analysis, and do computational modeling. Python codes were created to replicate computational models that simulate the effect of working memory (WM) load over RL performance. Our behavioral results indicated that we were able to replicate remotely and with a modified behavioral task the effects of working memory (WM) load over RL performance observed in previous studies with in-person assessments. Our computational analyses using Python code also captured the effects of WM load over RL as expected, which suggests that the algorithms and optimization methods were reliable in their ability to reproduce behavior. The behavioral and computational validation shown in this study and the detailed description of the supervised remote testing may be useful for researchers interested in conducting long and complex experiments online.

Lowered inter-stimulus discriminability hurts incremental contributions to learning

How does the similarity between stimuli affect our ability to learn appropriate response associations for them? In typical laboratory experiments learning is investigated under somewhat ideal circumstances, where stimuli are easily discriminable. This is not representative of most real-life learning, where overlapping "stimuli" can result in different "rewards" and may be learned simultaneously (e.g., you may learn over repeated interactions that a specific dog is friendly, but that a very similar looking one isn't). With two experiments, we test how humans learn in three stimulus conditions: one "best case" condition in which stimuli have idealized and highly discriminable visual and semantic representations, and two in which stimuli have overlapping representations, making them less discriminable. We find that, unsurprisingly, decreasing stimuli discriminability decreases performance. We develop computational models to test different hypotheses about how reinforcement learning (RL) and working memory (WM) processes are affected by different stimulus conditions. Our results replicate earlier studies demonstrating the importance of both processes to capture behavior. However, our results extend previous studies by demonstrating that RL, and not WM, is affected by stimulus distinctness: people learn slower and have higher across-stimulus value confusion at decision when stimuli are more similar to each other. These results illustrate strong effects of stimulus type on learning and demonstrate the importance of considering parallel contributions of different cognitive processes when studying behavior.

Integrative Brain Network and Salience Models of Psychopathology and Cognitive Dysfunction in Schizophrenia

Brain network models of cognitive control are central to advancing our understanding of psychopathology and cognitive dysfunction in schizophrenia. This review examines the role of large-scale brain organization in schizophrenia, with a particular focus on a triple-network model of cognitive control and its role in aberrant salience processing. First, we provide an overview of the triple network involving the salience, frontoparietal, and default mode networks and highlight the central role of the insula-anchored salience network in the aberrant mapping of salient external and internal events in schizophrenia. We summarize the extensive evidence that has emerged from structural, neurochemical, and functional brain imaging studies for aberrancies in these networks and their dynamic temporal interactions in schizophrenia. Next, we consider the hypothesis that atypical striatal dopamine release results in misattribution of salience to irrelevant external stimuli and self-referential mental events. We propose an integrated triple-network salience-based model incorporating striatal dysfunction and sensitivity to perceptual and cognitive prediction errors in the insula node of the salience network and postulate that dysregulated dopamine modulation of salience network-centered processes contributes to the core clinical phenotype of schizophrenia. Thus, a powerful paradigm to characterize the neurobiology of schizophrenia emerges when we combine conceptual models of salience with large-scale cognitive control networks in a unified manner. We conclude by discussing potential therapeutic leads on restoring brain network dysfunction in schizophrenia.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

Cognitive impairment in adults under compulsory psychiatric care: association with psychotic symptoms and high-dose antipsychotics

BACKGROUND

There is limited evidence on the association between cognitive function, psychotic symptoms and doses of antipsychotics in adults under compulsory psychiatric care.

AIMS

We assessed (a) the degree of cognitive impairment in adults involuntarily hospitalised for compulsory psychiatric care and (b) correlation of Montreal Cognitive Assessment (MoCA) score with psychotic symptoms, polypharmacy and prescription of high-dose antipsychotics.

METHOD

This was a nationwide, cross-sectional study, conducted at the only referral state hospital for compulsory psychiatric care in Cyprus (December 2016-February 2018). Τhe MoCA was applied for the assessment of cognitive functioning. The Positive and Negative Syndrome Scale (PANSS) was applied for the assessment of psychotic symptoms.

RESULTS

The sample comprised 187 men and 116 women. The mean MoCA score was 22.09 (reported scale range (RSR): 3-30); the mean PANSS general symptoms subscale score was 49.60 (RSR = 41-162). The participants who reported positive psychiatric history (mean 21.71, s.d. 5.37), non-adherence to pharmacotherapy (mean 21.32, s.d. 5.56) and prescription of high-dose antipsychotics (with medication prescribed as needed: mean 21.31, s.d. 5.70; without medication prescribed as needed: mean 20.71, s.d. 5.78) had lower mean MoCA scores compared with those who reported negative psychiatric history (mean 23.42, s.d. 4.51; P = 0.017), adherence to pharmacotherapy (mean 23.10, s.d. 6.61; P = 0.003) and no prescription of high-dose antipsychotics (with medication prescribed as needed: mean 22.56, s.d. 4.90; without medication prescribed as needed: mean 22.60 s.d. 4.94; P = 0.045-0.005), respectively. Mean MoCA score was mildly and inversely associated with total PANSS score (r = -0.15, P = 0.03), PANSS general (r = -0.18, P = 0.002) and PANSS negative (r = -0.16, P = 0.005) symptoms subscales, respectively.

CONCLUSIONS

Our findings support the evaluation of cognitive functioning in adults under compulsory psychiatric care via the MoCA tool, with focus on those prescribed high-dose antipsychotics, with positive mental health history and non-adherence to pharmacotherapy.

Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty

People radically differ in how they cope with uncertainty. Clinical researchers describe a dispositional characteristic known as "intolerance of uncertainty", a tendency to find uncertainty aversive, reported to be elevated across psychiatric and neurodevelopmental conditions. Concurrently, recent research in computational psychiatry has leveraged theoretical work to characterise individual differences in uncertainty processing. Under this framework, differences in how people estimate different forms of uncertainty can contribute to mental health difficulties. In this review, we briefly outline the concept of intolerance of uncertainty within its clinical context, and we argue that the mechanisms underlying this construct may be further elucidated through modelling how individuals make inferences about uncertainty. We will review the evidence linking psychopathology to different computationally specified forms of uncertainty and consider how these findings might suggest distinct mechanistic routes towards intolerance of uncertainty. We also discuss the implications of this computational approach for behavioural and pharmacological interventions, as well as the importance of different cognitive domains and subjective experiences in studying uncertainty processing.

Human generalization of internal representations through prototype learning with goal-directed attention

The world is overabundant with feature-rich information obscuring the latent causes of experience. How do people approximate the complexities of the external world with simplified internal representations that generalize to novel examples or situations? Theories suggest that internal representations could be determined by decision boundaries that discriminate between alternatives, or by distance measurements against prototypes and individual exemplars. Each provide advantages and drawbacks for generalization. We therefore developed theoretical models that leverage both discriminative and distance components to form internal representations via action-reward feedback. We then developed three latent-state learning tasks to test how humans use goal-oriented discrimination attention and prototypes/exemplar representations. The majority of participants attended to both goal-relevant discriminative features and the covariance of features within a prototype. A minority of participants relied only on the discriminative feature. Behaviour of all participants could be captured by parameterizing a model combining prototype representations with goal-oriented discriminative attention.

Moderators of metacognitive strategy training for executive functioning in early schizophrenia and psychosis risk

Goal Management Training (GMT) improved self-reported executive functioning in a recent randomized, controlled trial in early intervention for psychosis participants. Little is known about the mechanism for this benefit, so this study investigates objectively measured executive function, the difference between subjective and objective executive function, independent living and employment status as potential moderators of efficacy of GMT. Baseline scores from 81 participants (GMT n = 39 vs Treatment-as-usual; TAU n = 42) were analyzed in a linear mixed model analysis for repeated measures as predictors of improvement on the self-reported Behavior Rating Inventory of Executive Function-Adult version (BRIEF-A) immediately and 30 weeks after GMT. Potential moderators were scores from objective measures of executive functioning, discrepancy between subjective and objective measures, independent living and employment status. Discrepancy was assessed by comparing four clusters of participants with differing patterns of scores. The effect of GMT remained significant regardless of initial objective executive functioning at baseline. Those with higher subjective complaints at baseline in two clusters with (i) both objective and subjective executive dysfunction, and (ii) mostly subjective executive dysfunction experienced greater change after treatment. Living arrangements or participation in education or work did not significantly moderate the effects of GMT. Poor performance on neuropsychological tasks is not an obstacle to making use of GMT, but further knowledge is needed about the benefits of strategy training for individuals with a combination of poor performance with few subjective complaints.

Antipsychotics-induced improvement of cool executive function in individuals living with schizophrenia

Cool executive dysfunction is a crucial feature in people living with schizophrenia which is related to cognition impairment and the severity of the clinical symptoms. Based on electroencephalogram (EEG), our current study explored the change of brain network under the cool executive tasks in individuals living with schizophrenia before and after atypical antipsychotic treatment (before_TR vs. after_TR). 21 patients with schizophrenia and 24 healthy controls completed the cool executive tasks, involving the Tower of Hanoi Task (THT) and Trail-Marking Test A-B (TMT A-B). The results of this study uncovered that the reaction time of the after_TR group was much shorter than that of the before_TR group in the TMT-A and TMT-B. And the after_TR group showed fewer error numbers in the TMT-B than those of the before_TR group. Concerning the functional network, stronger DMN-like linkages were found in the before_TR group compared to the control group. Finally, we adopted a multiple linear regression model based on the change network properties to predict the patient's PANSS change ratio. Together, the findings deepened our understanding of cool executive function in individuals living with schizophrenia and might provide physiological information to reliably predict the clinical efficacy of schizophrenia after atypical antipsychotic treatment.

Choice Type Impacts Human Reinforcement Learning

In reinforcement learning (RL) experiments, participants learn to make rewarding choices in response to different stimuli; RL models use outcomes to estimate stimulus-response values that change incrementally. RL models consider any response type indiscriminately, ranging from more concretely defined motor choices (pressing a key with the index finger), to more general choices that can be executed in a number of ways (selecting dinner at the restaurant). However, does the learning process vary as a function of the choice type? In Experiment 1, we show that it does: Participants were slower and less accurate in learning correct choices of a general format compared with learning more concrete motor actions. Using computational modeling, we show that two mechanisms contribute to this. First, there was evidence of irrelevant credit assignment: The values of motor actions interfered with the values of other choice dimensions, resulting in more incorrect choices when the correct response was not defined by a single motor action; second, information integration for relevant general choices was slower. In Experiment 2, we replicated and further extended the findings from Experiment 1 by showing that slowed learning was attributable to weaker working memory use, rather than slowed RL. In both experiments, we ruled out the explanation that the difference in performance between two condition types was driven by difficulty/different levels of complexity. We conclude that defining a more abstract choice space used by multiple learning systems for credit assignment recruits executive resources, limiting how much such processes then contribute to fast learning.

Implicit sensorimotor adaptation is preserved in Parkinson's disease

Our ability to enact successful goal-directed actions involves multiple learning processes. Among these processes, implicit motor adaptation ensures that the sensorimotor system remains finely tuned in response to changes in the body and environment. Whether Parkinson's disease impacts implicit motor adaptation remains a contentious area of research: whereas multiple reports show impaired performance in this population, many others show intact performance. While there is a range of methodological differences across studies, one critical issue is that performance in many of the studies may reflect a combination of implicit adaptation and strategic re-aiming. Here, we revisited this controversy using a visuomotor task designed to isolate implicit adaptation. In two experiments, we found that adaptation in response to a wide range of visual perturbations was similar in Parkinson's disease and matched control participants. Moreover, in a meta-analysis of previously published and unpublished work, we found that the mean effect size contrasting Parkinson's disease and controls across 16 experiments involving over 200 participants was not significant. Together, these analyses indicate that implicit adaptation is preserved in Parkinson's disease, offering a fresh perspective on the role of the basal ganglia in sensorimotor learning.

The interpretation of computational model parameters depends on the context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8-30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.

Variables influencing conditioning-evoked hallucinations: overview and future applications

Hallucinations occur in the absence of sensory stimulation and result in vivid perceptual experiences of nonexistent events that manifest across a range of sensory modalities. Approaches from the field of experimental and cognitive psychology have leveraged the idea that associative learning experiences can evoke conditioning-induced hallucinations in both animals and humans. In this review, we describe classical and contemporary findings and highlight the variables eliciting these experiences. We also provide an overview of the neurobiological mechanisms, along with the associative and computational factors that may explain hallucinations that are generated by representation-mediated conditioning phenomena. Through the integration of animal and human research, significant advances into the psychobiology of hallucinations are possible, which may ultimately translate to more effective clinical applications.

Using Computational Modeling to Capture Schizophrenia-Specific Reinforcement Learning Differences and Their Implications on Patient Classification

BACKGROUND

Psychiatric diagnosis and treatment have historically taken a symptom-based approach, with less attention on identifying underlying symptom-producing mechanisms. Recent efforts have illuminated the extent to which different underlying circuitry can produce phenotypically similar symptomatology (e.g., psychosis in bipolar disorder vs. schizophrenia). Computational modeling makes it possible to identify and mathematically differentiate behaviorally unobservable, specific reinforcement learning differences in patients with schizophrenia versus other disorders, likely owing to a higher reliance on prediction error-driven learning associated with basal ganglia and underreliance on explicit value representations associated with orbitofrontal cortex.

METHODS

We used a well-established probabilistic reinforcement learning task to replicate those findings in individuals with schizophrenia both on (n = 120) and off (n = 44) antipsychotic medications and included a patient comparison group of bipolar patients with psychosis (n = 60) and healthy control subjects (n = 72).

RESULTS

Using accuracy, there was a main effect of group (F3,279 = 7.87, p < .001), such that all patient groups were less accurate than control subjects. Using computationally derived parameters, both medicated and unmediated individuals with schizophrenia, but not patients with bipolar disorder, demonstrated a reduced mixing parameter (F3,295 = 13.91, p < .001), indicating less dependence on learning explicit value representations as well as greater learning decay between training and test (F1,289 = 12.81, p < .001). Unmedicated patients with schizophrenia also showed greater decision noise (F3,295 = 2.67, p = .04).

CONCLUSIONS

Both medicated and unmedicated patients showed overreliance on prediction error-driven learning as well as significantly higher noise and value-related memory decay, compared with the healthy control subjects and the patients with bipolar disorder. Additionally, the computational model parameters capturing these processes can significantly improve patient/control classification, potentially providing useful diagnosis insight.

Cognitive [Computational] Neuroscience Test Reliability and Clinical Applications for Serious Mental Illness (CNTRaCS) Consortium: Progress and Future Directions

The development of treatments for impaired cognition in schizophrenia has been characterized as the most important challenge facing psychiatry at the beginning of the twenty-first century. The Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) project was designed to build on the potential benefits of using tasks and tools from cognitive neuroscience to better understanding and treat cognitive impairments in psychosis. These benefits include: (1) the use of fine-grained tasks that measure discrete cognitive processes; (2) the ability to design tasks that distinguish between specific cognitive domain deficits and poor performance due to generalized deficits resulting from sedation, low motivation, poor test taking skills, etc.; and (3) the ability to link cognitive deficits to specific neural systems, using animal models, neuropsychology, and functional imaging. CNTRICS convened a series of meetings to identify paradigms from cognitive neuroscience that maximize these benefits and identified the steps need for translation into use in clinical populations. The Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia (CNTRaCS) Consortium was developed to help carry out these steps. CNTRaCS consists of investigators at five different sites across the country with diverse expertise relevant to a wide range of the cognitive systems identified as critical as part of CNTRICs. This work reports on the progress and current directions in the evaluation and optimization carried out by CNTRaCS of the tasks identified as part of the original CNTRICs process, as well as subsequent extensions into the Positive Valence systems domain of Research Domain Criteria (RDoC). We also describe the current focus of CNTRaCS, which involves taking a computational psychiatry approach to measuring cognitive and motivational function across the spectrum of psychosis. Specifically, the current iteration of CNTRaCS is using computational modeling to isolate parameters reflecting potentially more specific cognitive and visual processes that may provide greater interpretability in understanding shared and distinct impairments across psychiatric disorders.

Functional neuroimaging in psychiatry and the case for failing better

Psychiatric disorders encompass complex aberrations of cognition and affect and are among the most debilitating and poorly understood of any medical condition. Current treatments rely primarily on interventions that target brain function (drugs) or learning processes (psychotherapy). A mechanistic understanding of how these interventions mediate their therapeutic effects remains elusive. From the early 1990s, non-invasive functional neuroimaging, coupled with parallel developments in the cognitive neurosciences, seemed to signal a new era of neurobiologically grounded diagnosis and treatment in psychiatry. Yet, despite three decades of intense neuroimaging research, we still lack a neurobiological account for any psychiatric condition. Likewise, functional neuroimaging plays no role in clinical decision making. Here, we offer a critical commentary on this impasse and suggest how the field might fare better and deliver impactful neurobiological insights.

Reward and loss incentives improve spatial working memory by shaping trial-by-trial posterior frontoparietal signals

Integrating motivational signals with cognition is critical for goal-directed activities. The mechanisms that link neural changes with motivated working memory continue to be understood. Here, we tested how externally cued and non-cued (internally represented) reward and loss impact spatial working memory precision and neural circuits in human subjects using fMRI. We translated the classic delayed-response spatial working memory paradigm from non-human primate studies to take advantage of a continuous numeric measure of working memory precision, and the wealth of translational neuroscience yielded by these studies. Our results demonstrated that both cued and non-cued reward and loss improved spatial working memory precision. Visual association regions of the posterior prefrontal and parietal cortices, specifically the precentral sulcus (PCS) and intraparietal sulcus (IPS), had increased BOLD signal during incentivized spatial working memory. A subset of these regions had trial-by-trial increases in BOLD signal that were associated with better working memory precision, suggesting that these regions may be critical for linking neural signals with motivated working memory. In contrast, regions straddling executive networks, including areas in the dorsolateral prefrontal cortex, anterior parietal cortex and cerebellum displayed decreased BOLD signal during incentivized working memory. While reward and loss similarly impacted working memory processes, they dissociated during feedback when money won or avoided in loss was given based on working memory performance. During feedback, the trial-by-trial amount and valence of reward/loss received was dissociated amongst regions such as the ventral striatum, habenula and periaqueductal gray. Overall, this work suggests motivated spatial working memory is supported by complex sensory processes, and that the IPS and PCS in the posterior frontoparietal cortices may be key regions for integrating motivational signals with spatial working memory precision.

Psychological Dimensions Relevant to Motivation and Pleasure in Schizophrenia

Motivation and pleasure deficits are common in schizophrenia, strongly linked with poorer functioning, and may reflect underlying alterations in brain functions governing reward processing and goal pursuit. While there is extensive research examining cognitive and reward mechanisms related to these deficits in schizophrenia, less attention has been paid to psychological characteristics that contribute to resilience against, or risk for, motivation and pleasure impairment. For example, psychological tendencies involving positive future expectancies (e.g., optimism) and effective affect management (e.g., reappraisal, mindfulness) are associated with aspects of reward anticipation and evaluation that optimally guide goal-directed behavior. Conversely, maladaptive thinking patterns (e.g., defeatist performance beliefs, asocial beliefs) and tendencies that amplify negative cognitions (e.g., rumination), may divert cognitive resources away from goal pursuit or reduce willingness to exert effort. Additionally, aspects of sociality, including the propensity to experience social connection as positive reinforcement may be particularly relevant for pursuing social goals. In the current review, we discuss the roles of several psychological characteristics with respect to motivation and pleasure in schizophrenia. We argue that individual variation in these psychological dimensions is relevant to the study of motivation and reward processing in schizophrenia, including interactions between these psychological dimensions and more well-characterized cognitive and reward processing contributors to motivation. We close by emphasizing the value of considering a broad set of modulating factors when studying motivation and pleasure functions in schizophrenia.

Event segmentation reveals working memory forgetting rate

We encounter the world as a continuous flow and effortlessly segment sequences of events into episodes. This process of event segmentation engages working memory (WM) for tracking the flow of events and impacts subsequent memory accuracy. WM is limited in how much information (i.e., WM capacity) and for how long the information is retained (i.e., forgetting rate). In this study, across multiple tasks, we estimated participants’ WM capacity and forgetting rate in a dynamic context and evaluated their relationship to event segmentation. A U-shaped relationship across tasks shows that individuals who segmented the movie more finely or coarsely than the average have a faster WM forgetting rate. A separate task assessing long-term memory retrieval revealed that the coarse-segmenters have better recognition of temporal order of events compared to the fine-segmenters. These findings show that event segmentation employs dissociable memory strategies and correlates with how long information is retained in WM

•

The event segmentation grain is variable across individuals

•

The event segmentation grain has a U-shaped relationship with the WM forgetting rate

•

The temporal order memory accuracy decreases with the increasing event segmentation

•

The number of recalled events increases with the increasing event segmentation

Reduced learning bias towards the reward context in medication-naive first-episode schizophrenia patients

BACKGROUND

Reinforcement learning has been proposed to contribute to the development of amotivation in individuals with schizophrenia (SZ). Accumulating evidence suggests dysfunctional learning in individuals with SZ in Go/NoGo learning and expected value representation. However, previous findings might have been confounded by the effects of antipsychotic exposure. Moreover, reinforcement learning also rely on the learning context. Few studies have examined the learning performance in reward and loss-avoidance context separately in medication-naïve individuals with first-episode SZ. This study aimed to explore the behaviour profile of reinforcement learning performance in medication-naïve individuals with first-episode SZ, including the contextual performance, the Go/NoGo learning and the expected value representation performance.

METHODS

Twenty-nine medication-naïve individuals with first-episode SZ and 40 healthy controls (HCs) who have no significant difference in age and gender, completed the Gain and Loss Avoidance Task, a reinforcement learning task involving stimulus pairs presented in both the reward and loss-avoidance context. We assessed the group difference in accuracy in the reward and loss-avoidance context, the Go/NoGo learning and the expected value representation. The correlations between learning performance and the negative symptom severity were examined.

RESULTS

Individuals with SZ showed significantly lower accuracy when learning under the reward than the loss-avoidance context as compared to HCs. The accuracies under the reward context (90%win- 10%win) in the Acquisition phase was significantly and negatively correlated with the Scale for the Assessment of Negative Symptoms (SANS) avolition scores in individuals with SZ. On the other hand, individuals with SZ showed spared ability of Go/NoGo learning and expected value representation.

CONCLUSIONS

Despite our small sample size and relatively modest findings, our results suggest possible reduced learning bias towards reward context among medication-naïve individuals with first-episode SZ. The reward learning performance was correlated with amotivation symptoms. This finding may facilitate our understanding of the underlying mechanism of negative symptoms. Reinforcement learning performance under the reward context may be important to better predict and prevent the development of schizophrenia patients' negative symptom, especially amotivation.

Impairment in acquisition of conditioned fear in schizophrenia

Individuals with schizophrenia show impairments in associative learning. One well-studied, quantifiable form of associative learning is Pavlovian fear conditioning. However, to date, studies of fear conditioning in schizophrenia have been inconclusive, possibly because they lacked sufficient power. To address this issue, we pooled data from four independent fear conditioning studies that included a total of 77 individuals with schizophrenia and 74 control subjects. Skin conductance responses (SCRs) to stimuli that were paired (the CS + ) or not paired (CS-) with an aversive, unconditioned stimulus were measured, and the success of acquisition of differential conditioning (the magnitude of CS + vs. CS- SCRs) and responses to CS + and CS- separately were assessed. We found that acquisition of differential conditioned fear responses was significantly lower in individuals with schizophrenia than in healthy controls (Cohen's d = 0.53). This effect was primarily related to a significantly higher response to the CS- stimulus in the schizophrenia compared to the control group. Moreover, the magnitude of this response to the CS- in the schizophrenia group was correlated with the severity of delusional ideation (p = 0.006). Other symptoms or antipsychotic dose were not associated with fear conditioning measures. In conclusion, individuals with schizophrenia who endorse delusional beliefs may be over-responsive to neutral stimuli during fear conditioning. This finding is consistent with prior models of abnormal associative learning in psychosis.