Reinforcement learning and Tourette syndrome

When the interoceptive and conceptual clash: The case of oppositional phenomenal self-modelling in Tourette syndrome

Tourette syndrome (TS) has been associated with a rich set of symptoms that are said to be uncomfortable, unwilled, and effortful to manage. Furthermore, tics, the canonical characteristic of TS, are multifaceted, and their onset and maintenance is complex. A formal account that integrates these features of TS symptomatology within a plausible theoretical framework is currently absent from the field. In this paper, we assess the explanatory power of hierarchical generative modelling in accounting for TS symptomatology from the perspective of active inference. We propose a fourfold analysis of sensory, motor, and cognitive phenomena associated with TS. In Section 1, we characterise tics as a form of action aimed at sensory attenuation. In Section 2, we introduce the notion of epistemic ticcing and describe such behaviour as the search for evidence that there is an agent (i.e., self) at the heart of the generative hierarchy. In Section 3, we characterise both epistemic (sensation-free) and nonepistemic (sensational) tics as habitual behaviour. Finally, in Section 4, we propose that ticcing behaviour involves an inevitable conflict between distinguishable aspects of selfhood; namely, between the minimal phenomenal sense of self-which is putatively underwritten by interoceptive inference-and the explicit preferences that constitute the individual's conceptual sense of self. In sum, we aim to provide an empirically informed analysis of TS symptomatology under active inference, revealing a continuity between covert and overt features of the condition.

Breast Cancer Diagnosis in Thermography Using Pre-Trained VGG16 with Deep Attention Mechanisms

One of the most prevalent cancers in women is breast cancer. The mortality rate related to this disease can be decreased by early, accurate diagnosis to increase the chance of survival. Infrared thermal imaging is one of the breast imaging modalities in which the temperature of the breast tissue is measured using a screening tool. The previous studies did not use pre-trained deep learning (DL) with deep attention mechanisms (AMs) on thermographic images for breast cancer diagnosis. Using thermal images from the Database for Research Mastology with Infrared Image (DMR-IR), the study investigates the use of a pre-trained Visual Geometry Group with 16 layers (VGG16) with AMs that can produce good diagnosis performance utilizing the thermal images of breast cancer. The symmetry of the three models resulting from the combination of VGG16 with three types of AMs is evident in all its stages in methodology. The models were compared to state-of-art breast cancer diagnosis approaches and tested for accuracy, sensitivity, specificity, precision, F1-score, AUC score, and Cohen’s kappa. The test accuracy rates for the AMs using the VGG16 model on the breast thermal dataset were encouraging, at 99.80%, 99.49%, and 99.32%. Test accuracy for VGG16 without AMs was 99.18%, whereas test accuracy for VGG16 with AMs improved by 0.62%. The proposed approaches also performed better than previous approaches examined in the related studies.

Tourette syndrome: clinical features, pathophysiology, and treatment

Tourette syndrome is a chronic neurodevelopmental disorder characterised by motor and phonic tics that can substantially diminish the quality of life of affected individuals. Evaluating and treating Tourette syndrome is complex, in part due to the heterogeneity of symptoms and comorbidities between individuals. The underlying pathophysiology of Tourette syndrome is not fully understood, but recent research in the past 5 years has brought new insights into the genetic variations and the alterations in neurophysiology and brain networks contributing to its pathogenesis. Treatment options for Tourette syndrome are expanding with novel pharmacological therapies and increased use of deep brain stimulation for patients with symptoms that are refractory to pharmacological or behavioural treatments. Potential predictors of patient responses to therapies for Tourette syndrome, such as specific networks modulated during deep brain stimulation, can guide clinical decisions. Multicentre data sharing initiatives have enabled several advances in our understanding of the genetics and pathophysiology of Tourette syndrome and will be crucial for future large-scale research and in refining effective treatments.

A Process-Oriented View of Procedural Memory Can Help Better Understand Tourette's Syndrome

Tourette's syndrome (TS) is a neurodevelopmental disorder characterized by repetitive movements and vocalizations, also known as tics. The phenomenology of tics and the underlying neurobiology of the disorder have suggested that the altered functioning of the procedural memory system might contribute to its etiology. However, contrary to the robust findings of impaired procedural memory in neurodevelopmental disorders of language, results from TS have been somewhat mixed. We review the previous studies in the field and note that they have reported normal, impaired, and even enhanced procedural performance. These mixed findings may be at least partially be explained by the diversity of the samples in both age and tic severity, the vast array of tasks used, the low sample sizes, and the possible confounding effects of other cognitive functions, such as executive functions, working memory or attention. However, we propose that another often overlooked factor could also contribute to the mixed findings, namely the multiprocess nature of the procedural system itself. We propose that a process-oriented view of procedural memory functions could serve as a theoretical framework to help integrate these varied findings. We discuss evidence suggesting heterogeneity in the neural regions and their functional contributions to procedural memory. Our process-oriented framework can help to deepen our understanding of the complex profile of procedural functioning in TS and atypical development in general.

The sooner the better: clinical and neural correlates of impulsive choice in Tourette disorder

Reward sensitivity has been suggested as one of the central pathophysiological mechanisms in Tourette disorder. However, the subjective valuation of a reward by introduction of delay has received little attention in Tourette disorder, even though it has been suggested as a trans-diagnostic feature of numerous neuropsychiatric disorders. We aimed to assess delay discounting in Tourette disorder and to identify its brain functional correlates. We evaluated delayed discounting and its brain functional correlates in a large group of 54 Tourette disorder patients and 31 healthy controls using a data-driven approach. We identified a subgroup of 29 patients with steeper reward discounting, characterised by a higher burden of impulse-control disorders and a higher level of general impulsivity compared to patients with normal behavioural performance or to controls. Reward discounting was underpinned by resting-state activity of a network comprising the orbito-frontal, cingulate, pre-supplementary motor area, temporal and insular cortices, as well as ventral striatum and hippocampus. Within this network, (i) lower connectivity of pre-supplementary motor area with ventral striatum predicted a higher impulsivity and a steeper reward discounting and (ii) a greater connectivity of pre-supplementary motor area with anterior insular cortex predicted steeper reward discounting and more severe tics. Overall, our results highlight the heterogeneity of the delayed reward processing in Tourette disorder, with steeper reward discounting being a marker of burden in impulsivity and impulse control disorders, and the pre-supplementary motor area being a hub region for the delay discounting, impulsivity and tic severity.

Temporal discounting in adolescents and adults with Tourette syndrome

Tourette syndrome is a neurodevelopmental disorder associated with hyperactivity in dopaminergic networks. Dopaminergic hyperactivity in the basal ganglia has previously been linked to increased sensitivity to positive reinforcement and increases in choice impulsivity. In this study, we examine whether this extends to changes in temporal discounting, where impulsivity is operationalized as an increased preference for smaller-but-sooner over larger-but-later rewards. We assessed intertemporal choice in two studies including nineteen adolescents (age: mean[sd] = 14.21[±2.37], 13 male subjects) and twenty-five adult patients (age: mean[sd] = 29.88 [±9.03]; 19 male subjects) with Tourette syndrome and healthy age- and education matched controls. Computational modeling using exponential and hyperbolic discounting models via hierarchical Bayesian parameter estimation revealed reduced temporal discounting in adolescent patients, and no evidence for differences in adult patients. Results are discussed with respect to neural models of temporal discounting, dopaminergic alterations in Tourette syndrome and the developmental trajectory of temporal discounting. Specifically, adolescents might show attenuated discounting due to improved inhibitory functions that also affect choice impulsivity and/or the developmental trajectory of executive control functions. Future studies would benefit from a longitudinal approach to further elucidate the developmental trajectory of these effects.

Acute stress impairs reward learning in men

Acute stress is ubiquitous in everyday life, but the extent to which acute stress affects how people learn from the outcomes of their choices is still poorly understood. Here, we investigate how acute stress impacts reward and punishment learning in men using a reinforcement-learning task. Sixty-two male participants performed the task whilst under stress and control conditions. We observed that acute stress impaired participants' choice performance towards monetary gains, but not losses. To unravel the mechanism(s) underlying such impairment, we fitted a reinforcement-learning model to participants' trial-by-trial choices. Computational modeling indicated that under acute stress participants learned more slowly from positive prediction errors - when the outcomes were better than expected - consistent with stress-induced dopamine disruptions. Such mechanistic understanding of how acute stress impairs reward learning is particularly important given the pervasiveness of stress in our daily life and the impact that stress can have on our wellbeing and mental health.

Impairment of visually guided associative learning in children with Tourette syndrome

The major symptoms of Tourette syndrome are motor and vocal tics, but Tourette syndrome is occasionally associated with cognitive alterations as well. Although Tourette syndrome does not affect the majority of cognitive functions, some of them improve. There is scarce evidence on the impairment of learning functions in patients with Tourette syndrome. The core symptoms of Tourette syndrome are related to dysfunction of the basal ganglia and the frontostriatal loops. Acquired equivalence learning is a kind of associative learning that is related to the basal ganglia and the hippocampi. The modified Rutgers Acquired Equivalence Test was used in the present study to observe the associative learning function of patients with Tourette syndrome. The cognitive learning task can be divided into two main phases: the acquisition and test phases. The latter is further divided into two parts: retrieval and generalization. The acquisition phase of the associative learning test, which mainly depends on the function of the basal ganglia, was affected in the entire patient group, which included patients with Tourette syndrome with attention deficit hyperactivity disorder, obsessive compulsive disorder, autism spectrum disorder, or no comorbidities. Patients with Tourette syndrome performed worse in building associations. However, the retrieval and generalization parts of the test phase, which primarily depend on the function of the hippocampus, were not worsened by Tourette syndrome.

Clinical Features That Evoke the Concept of Disinhibition in Tourette Syndrome

The capacity to efficiently control motor output, by either refraining from prepotent actions or disengaging from ongoing motor behaviors, is necessary for our ability to thrive in a stimulus-rich and socially complex environment. Failure to engage in successful inhibitory motor control could lead to aberrant behaviors typified by an excess of motor performance. In tic disorders and Tourette syndrome (TS) - the most common tic disorder encountered in clinics - surplus motor output is rarely the only relevant clinical sign. A range of abnormal behaviors is often encountered which are historically viewed as "disinhibition phenomena". Here, we present the different clinical features of TS from distinct categorical domains (motor, sensory, complex behavioral) that evoke the concept of disinhibition and discuss their associations. We also present evidence for their consideration as phenomena of inhibitory dysfunction and provide an overview of studies on TS pathophysiology which support this view. We then critically dissect the concept of disinhibition in TS and illuminate other salient aspects, which should be considered in a unitary pathophysiological approach. We briefly touch upon the dangers of oversimplification and emphasize the necessity of conceptual diversity in the scientific exploration of TS, from disinhibition and beyond.

The Association of Poor Academic Performance with Tic Disorders: A Longitudinal, Mainstream School-Based Population Study

BACKGROUND

Little is known about the academic performance of students with tic disorders (TD). Our aim was to investigate the association of TD and poor academic performance over time.

METHODS

Longitudinal, observational study of mainstream schoolchildren comparing grade retention (GR) and learning disorders (LD) in students with vs. without TD between 2010 and 2014. Students with vs. without TD based on DSM-IV-TR criteria, or with vs. without GR and LD were compared in terms of comorbidities, school, and environmental characteristics. The association of TD with GR was analyzed using hazard ratios (HRs) with 95% CIs, and with LD using logistic regression analysis [Odds ratio (OR)].

RESULTS

Two hundred fifty-eight students were included (mean age 14.0 ± 1.71 years, 143 [55.4%] males). The incident rate for TD and GR was 2.6 and 3.3 per 100 persons-year, respectively. LD found in 21 (9.9%) students was associated with TD (OR 11.62, 95% CI 2.21-60.90, p = 0.004), and attention deficit hyperactivity disorder (ADHD; OR 6.63, 95% CI 1.55-28.37, p = 0.01). Low psychological support (HRs 12.79, 95% CI 3.39-48.17) and low sport participation (HRs 6.41, 95% CI 1.54-26.78) were risk factors for GR.

CONCLUSIONS

TD was associated with academic difficulties, namely, LD in conjunction with ADHD but not GR. The diagnosis of TD and comorbidities, and the initiation of proper treatment could have a favorable impact on school performance, and consequently on social development.

Novel reinforcement learning paradigm based on response patterning under interval schedules of reinforcement

There is a need to develop cognitive tasks that address valid neuropsychological constructs implicated in disease mechanisms and can be used in animals and humans to guide novel drug discovery. Present experiments aimed to characterize a novel reinforcement learning task based on a classical operant behavioral phenomenon observed in multiple species - differences in response patterning under variable (VI) vs fixed interval (FI) schedules of reinforcement. Wistar rats were trained to press a lever for food under VI30s and later weekly test sessions were introduced with reinforcement schedule switched to FI30s. During the FI30s test session, post-reinforcement pauses (PRPs) gradually grew towards the end of the session reaching 22-43% of the initial values. Animals could be retrained under VI30s conditions, and FI30s test sessions were repeated over a period of several months without appreciable signs of a practice effect. Administration of the non-competitive N-methyl-d-aspartate (NMDA) receptor antagonist MK-801 ((5S,10R)-(+)-5-Methyl-10,11-dihydro-5H-dibenzo[a,d]cyclohepten-5,10-imine maleate) prior to FI30s sessions prevented adjustment of PRPs associated with the change from VI to FI schedule. This effect was most pronounced at the highest tested dose of MK-801 and appeared to be independent of the effects of this dose on response rates. These results provide initial evidence for the possibility to use different response patterning under VI and FI schedules with equivalent reinforcement density for studying effects of drug treatment on reinforcement learning.

Specific effect of a dopamine partial agonist on counterfactual learning: evidence from Gilles de la Tourette syndrome

The dopamine partial agonist aripiprazole is increasingly used to treat pathologies for which other antipsychotics are indicated because it displays fewer side effects, such as sedation and depression-like symptoms, than other dopamine receptor antagonists. Previously, we showed that aripiprazole may protect motivational function by preserving reinforcement-related signals used to sustain reward-maximization. However, the effect of aripiprazole on more cognitive facets of human reinforcement learning, such as learning from the forgone outcomes of alternative courses of action (i.e., counterfactual learning), is unknown. To test the influence of aripiprazole on counterfactual learning, we administered a reinforcement learning task that involves both direct learning from obtained outcomes and indirect learning from forgone outcomes to two groups of Gilles de la Tourette (GTS) patients, one consisting of patients who were completely unmedicated and the other consisting of patients who were receiving aripiprazole monotherapy, and to healthy subjects. We found that whereas learning performance improved in the presence of counterfactual feedback in both healthy controls and unmedicated GTS patients, this was not the case in aripiprazole-medicated GTS patients. Our results suggest that whereas aripiprazole preserves direct learning of action-outcome associations, it may impair more complex inferential processes, such as counterfactual learning from forgone outcomes, in GTS patients treated with this medication.

Brain structure in pediatric Tourette syndrome

Previous studies of brain structure in Tourette syndrome (TS) have produced mixed results, and most had modest sample sizes. In the present multicenter study, we used structural magnetic resonance imaging (MRI) to compare 103 children and adolescents with TS to a well-matched group of 103 children without tics. We applied voxel-based morphometry methods to test gray matter (GM) and white matter (WM) volume differences between diagnostic groups, accounting for MRI scanner and sequence, age, sex and total GM+WM volume. The TS group demonstrated lower WM volume bilaterally in orbital and medial prefrontal cortex, and greater GM volume in posterior thalamus, hypothalamus and midbrain. These results demonstrate evidence for abnormal brain structure in children and youth with TS, consistent with and extending previous findings, and they point to new target regions and avenues of study in TS. For example, as orbital cortex is reciprocally connected with hypothalamus, structural abnormalities in these regions may relate to abnormal decision making, reinforcement learning or somatic processing in TS.

Reward and decision processes in the brains of humans and nonhuman primates

Choice behavior requires weighing multiple decision variables, such as utility, uncertainty, delay, or effort, that combine to define a subjective value for each considered option or course of action. This capacity is based on prior learning about potential rewards (and punishments) that result from prior actions. When made in a social context, decisions can involve strategic thinking about the intentions of others and about the impact of others' behavior on one's own outcome. Valuation is also influenced by different emotions that serve to adaptively regulate our choices in order to, for example, stay away from excessively risky gambles, prevent future regrets, or avoid personal rejection or conflicts. Drawing on economic theory and on advances in the study of neuronal mechanisms, we review relevant recent experiments in nonhuman primates and clinical observations made in neurologically impaired patients suffering from impaired decision-making capacities.