Differential reinforcement learning responses to positive and negative information in unmedicated individuals with depression

Aberrant reward learning, but not negative reinforcement learning, is related to depressive symptoms: an attentional perspective

BACKGROUND

Aberrant reward functioning is implicated in depression. While attention precedes behavior and guides higher-order cognitive processes, reward learning from an attentional perspective - the effects of prior reward-learning on subsequent attention allocation - has been mainly overlooked.

METHODS

The present study explored the effects of reward-based attentional learning in depression using two separate, yet complimentary, studies. In study 1, participants with high (HD) and low (LD) levels of depression symptoms were trained to divert their gaze toward one type of stimuli over another using a novel gaze-contingent music reward paradigm - music played when fixating the desired stimulus type and stopped when gazing the alternate one. Attention allocation was assessed before, during, and following training. In study 2, using negative reinforcement, the same attention allocation pattern was trained while substituting the appetitive music reward for gazing the desired stimulus type with the removal of an aversive sound (i.e. white noise).

RESULTS

In study 1 both groups showed the intended shift in attention allocation during training (online reward learning), while generalization of learning at post-training was only evident among LD participants. Conversely, in study 2 both groups showed post-training generalization. Results were maintained when introducing anxiety as a covariate, and when using a more powerful sensitivity analysis. Finally, HD participants showed higher learning speed than LD participants during initial online learning, but only when using negative, not positive, reinforcement.

CONCLUSIONS

Deficient generalization of learning characterizes the attentional system of HD individuals, but only when using reward-based positive reinforcement, not negative reinforcement.

A feasibility study of goal-directed network-based real-time fMRI neurofeedback for anhedonic depression

Anhedonia is a hallmark symptom of depression that often lacks adequate interventions. The translational gap remains in clinical treatments based on neural substrates of anhedonia. Our pilot study found that depressed individuals depended less on goal-directed (GD) reward learning (RL), with reduced reward prediction error (RPE) BOLD signal. Previous studies have found that anhedonia is related to abnormal activities and/or functional connectivities of the central executive network (CEN) and salience network (SN), both of which belong to the goal-directed system. In addition, it was found that real-time functional magnetic resonance imaging (rt-fMRI) neurofeedback (NF) could improve the balance between CEN and SN in healthy individuals. Therefore, we speculate that rt-fMRI NF of the CEN and SN associated with the GD system may improve depressive and/or anhedonic symptoms. Therefore, this study (1) will examine individuals with anhedonic depression using GD-RL behavioral task, combined with functional magnetic resonance imaging and computational modeling to explore the role of CEN/SN deficits in anhedonic depression; and (2) will utilize network-based rt-fMRI NF to investigate whether it is feasible to regulate the differential signals of brain CEN/SN of GD system through rt-fMRI NF to alleviate depressive and/or anhedonic symptoms. This study highlights the need to elucidate the intervention effects of rt-fMRI NF and the underlying computational network neural mechanisms.

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Adaptive human learning utilizes reward prediction errors (RPEs) that scale the differences between expected and actual outcomes to optimize future choices. Depression has been linked with biased RPE signaling and an exaggerated impact of negative outcomes on learning which may promote amotivation and anhedonia. The present proof-of-concept study combined computational modeling and multivariate decoding with neuroimaging to determine the influence of the selective competitive angiotensin II type 1 receptor antagonist losartan on learning from positive or negative outcomes and the underlying neural mechanisms in healthy humans. In a double-blind, between-subjects, placebo-controlled pharmaco-fMRI experiment, 61 healthy male participants (losartan, n = 30; placebo, n = 31) underwent a probabilistic selection reinforcement learning task incorporating a learning and transfer phase. Losartan improved choice accuracy for the hardest stimulus pair via increasing expected value sensitivity towards the rewarding stimulus relative to the placebo group during learning. Computational modeling revealed that losartan reduced the learning rate for negative outcomes and increased exploitatory choice behaviors while preserving learning for positive outcomes. These behavioral patterns were paralleled on the neural level by increased RPE signaling in orbitofrontal-striatal regions and enhanced positive outcome representations in the ventral striatum (VS) following losartan. In the transfer phase, losartan accelerated response times and enhanced VS functional connectivity with left dorsolateral prefrontal cortex when approaching maximum rewards. These findings elucidate the potential of losartan to reduce the impact of negative outcomes during learning and subsequently facilitate motivational approach towards maximum rewards in the transfer of learning. This may indicate a promising therapeutic mechanism to normalize distorted reward learning and fronto-striatal functioning in depression.

Plasticity of synapses and reward circuit function in the genesis and treatment of depression

What changes in brain function cause the debilitating symptoms of depression? Can we use the answers to this question to invent more effective, faster acting antidepressant drug therapies? This review provides an overview and update of the converging human and preclinical evidence supporting the hypothesis that changes in the function of excitatory synapses impair the function of the circuits they are embedded in to give rise to the pathological changes in mood, hedonic state, and thought processes that characterize depression. The review also highlights complementary human and preclinical findings that classical and novel antidepressant drugs relieve the symptoms of depression by restoring the functions of these same synapses and circuits. These findings offer a useful path forward for designing better antidepressant compounds.

Disorder-specific impaired neurocognitive function in major depression and generalized anxiety disorder

BACKGROUND

Generalized anxiety disorder (GAD) and major depressive disorder (MDD) are both highly prevalent and comorbid psychiatric disorders. Neurocognitive dysfunction has been commonly found in MDD, but the findings in GAD are inconsistent. Few studies have directly compared cognitive performance between GAD and MDD. Therefore, the present study aimed to reveal the similar and distinct cognitive impairments between both disorders.

METHODS

Three non-overlapping and non-comorbid groups were enrolled in the current study including patients with GAD (n = 37), MDD (n = 107) and healthy controls (n = 74). Levels of anxiety and depression were assessed using the Hamilton Anxiety Rating Scale (HAMA) and the Hamilton Depression Rating Scale (HAMD) respectively. The Cambridge Neuropsychological Test Automated Battery (CANTAB) was used to compare the cognitive performance, including sustained attention, visual memory, executive functions and learning.

RESULTS

Both MDD and GAD groups demonstrated common significant deficits in sustained attention, visual memory, working memory and learning when compared to healthy controls. Despite the similarities, the MDD group had significantly greater impairment in learning, particularly generalization, while the GAD group demonstrated more pronounced deficits in visual memory.

LIMITATIONS

Patients involved were medicated and the sample size for GAD was relatively small.

CONCLUSIONS

The significant differences in visual memory and learning between MDD and GAD groups might be indicators to distinguishing both disorders. These results confirm that cognitive function is of great importance as a future target for treatment in order to improve wellbeing, quality of life and functionality in both GAD and MDD.

Affective Control Training (AffeCT) reduces negative affect in depressed individuals

Depression is the leading cause of disability worldwide, with prevalence rates rising. Despite the scale of the problem, available pharmacological and psychological interventions only have limited efficacy. The National Institute of Health's Science of Behaviour Change framework proposes to address this issue by capitalising on insights from basic science to identify mechanisms that can be targeted by novel interventions. The current study evaluated the potential of a computerized programme aimed at improving affective control, a mechanistic target involved in both risk and maintenance of depression. In a first phase the cognitive profiles of 48 depressed individuals (mean age: 39 years, 75 % female) were compared to cognitive functioning in 16 never-depressed individuals (mean age: 31 years, 56 % female). The sole index of functioning that differed between diagnostic groups was reaction time across negative and positively valanced trials on an affective Stroop task (d = 0.58). This index was then used to evaluate an affective control training (AffeCT) against a placebo training. Results showed no significant changes on tasks that showed no differences with never-depressed individuals in Phase I. However, compared to placebo training, AffeCT led to significantly greater improvement in the target index, affective Stroop performance (d = 1.17). Importantly, AffeCT led to greater reductions in negative affect as measured by the Positive Affect and Negative Affect Schedule compared to the placebo training (d = 0.98). This proof-of-concept study shows promising benefits of AffeCT on depressed individuals' affect, but not depressive symptoms. It further supports the utility of the Science of Behaviour Change framework, highlighting the need for determining meaningful assays of target mechanisms when evaluating novel interventions.

Neurofunctional mapping of reward anticipation and outcome for major depressive disorder: a voxel-based meta-analysis

Aberrations in how people form expectations about rewards and how they respond to receiving rewards are thought to underlie major depressive disorder (MDD). However, the underlying mechanism linking the appetitive reward system, specifically anticipation and outcome, is still not fully understood. To examine the neural correlates of monetary anticipation and outcome in currently depressed subjects with MDD, we performed two separate voxel-wise meta-analyses of functional neuroimaging studies using the monetary incentive delay task. During reward anticipation, the depressed patients exhibited an increased response in the bilateral middle cingulate cortex (MCC) extending to the anterior cingulate cortex, the medial prefrontal cortex, the left inferior frontal gyrus (IFG), and the postcentral gyrus, but a reduced response in the mesolimbic circuit, including the left striatum, insula, amygdala, right cerebellum, striatum, and IFG, compared to controls. During the outcome stage, MDD showed higher activity in the left inferior temporal gyrus, and lower activity in the mesocortical pathway, including the bilateral MCC, left caudate nucleus, precentral gyrus, thalamus, cerebellum, right striatum, insula, IFG, middle frontal gyrus, and temporal pole. Our findings suggest that cMDD may be characterised by state-dependent hyper-responsivity in cortical regions during the anticipation phase, and hypo-responsivity of the mesocortico-limbic circuit across the two phases of the reward response. Our study showed dissociable neural circuit responses to monetary stimuli during reward anticipation and outcome, which help to understand the dysfunction in different aspects of reward processing, particularly motivational v. hedonic deficits in depression.

Trauma Disrupts Reinforcement Learning in Rats-A Novel Animal Model of Chronic Stress Exposure

Trauma, as well as chronic stress that characterizes a modern fast-paced lifestyle, contributes to numerous psychopathologies and psychological problems. Psychiatric patients with traumas, as well as healthy individuals who experienced traumas in the past, are often characterized by diminished cognitive abilities. In our protocol, we used an animal model to explore the influence of chronic trauma on cognitive abilities and behavior in the group of 20 rats (Rattus norvegicus). The experimental group was introduced to chronic (12 consecutive days) exposure to predator odor (bobcat urine). We measured the reinforcement learning of each individual before and after the exposition via the Probabilistic Selection Task (PST) and we used Social Interaction Test (SIT) to assess the behavioral changes of each individual before and after the trauma. In the experimental group, there was a significant decrease in reinforcement learning after exposure to a single trauma (Wilcoxon Test, p = 0.034) as well as after 11 days of chronic trauma (Wilcoxon-test, p = 0.01) in comparison to pre-trauma performance. The control group, which was not exposed to predator odor but underwent the same testing protocol, did not present significant deterioration in reinforcement learning. In cross-group comparisons, there was no difference between the experimental and control group in PST before odor protocol (U Mann-Whitney two-sided, p = 0.909). After exposure to chronic trauma, the experimental group deteriorated in PST performance compared to control (U Mann-Whitney Two-sided, p = 0.0005). In SIT, the experimental group spent less time in an Interaction Zone with an unfamiliar rat after trauma protocol (Wilcoxon two-sided test, p = 0.019). Major strengths of our models are: (1) protocol allows investigating reinforcement learning before and after exposition to chronic trauma, with the same group of rats, (2) translational scope, as the PST is displayed on touchscreen, similarly to human studies, (3) protocol delivers chronic trauma that impairs reward learning, but behaviorally does not induce full-blown anhedonia, thus rats performed voluntarily throughout all the procedures.

Confirmation Bias in the Course of Instructed Reinforcement Learning in Schizophrenia-Spectrum Disorders

A large body of research attributes learning deficits in schizophrenia (SZ) to the systems involved in value representation (prefrontal cortex, PFC) and reinforcement learning (basal ganglia, BG) as well as to the compromised connectivity of these regions. In this study, we employed learning tasks hypothesized to probe the function and interaction of the PFC and BG in patients with SZ-spectrum disorders in comparison to healthy control (HC) subjects. In the Instructed Probabilistic Selection task (IPST), participants received false instruction about one of the stimuli used in the course of probabilistic learning which creates confirmation bias, whereby the instructed stimulus is overvalued in comparison to its real experienced value. The IPST was administered to 102 patients with SZ and 120 HC subjects. We have shown that SZ patients and HC subjects were equally influenced by false instruction in reinforcement learning (RL) probabilistic task (IPST) (p-value = 0.441); however, HC subjects had significantly higher learning rates associated with the process of overcoming cognitive bias in comparison to SZ patients (p-value = 0.018). The behavioral results of our study could be hypothesized to provide further evidence for impairments in the SZ-BG circuitry; however, this should be verified by neurofunctional imaging studies.