Neurocomputational models of basal ganglia function in learning, memory and choice

Modulation of EEG by Slow-Symmetric Breathing Incorporating Breath-Hold

The interplay between the brain and lungs involves intricate physiological mechanisms operating bidirectionally. Volitional breathing, unlike spontaneous breathing, offers various benefits with potential therapeutic effects. Volitional breathing involves many variables, such as breathing rate (BR) and breathing patterns. The main objective is to study the neural dynamics during slow-symmetric breathing (SSB) at different rates, with and without incorporating breath-holds post inhalation and exhalation. Electroencephalogram (EEG) is analyzed from 63 healthy adults while breathing at 10, 6, and 4 cycles per minute (cpm) guided by a visual metronome. A significant increase in coherence is observed during SSB at all breathing rates and resonance at 6 cpm (0.1 Hz). We also study EEG band power changes, coupling of the breathing phase with EEG amplitude, and EEG modulation during SSB. The power in alpha and beta bands is significantly higher during breath-hold than during inhalation and exhalation. Compared to the baseline, the modulation index increases significantly during SSB, illustrating the enhanced modulation of brain activity. Interestingly, coherence, phase-amplitude coupling, and modulation index depend mainly on the breathing rate, and the differences between symmetric breathing with and without breath-hold are not statistically significant. The observed similarities in neural dynamics across different breathing patterns suggest that achieving very low breathing rates is feasible with symmetric breathing with breath-holds while retaining the reported benefits of slow breathing.

Probabalistic reinforcement learning impairments predict negative symptom severity and risk for conversion in youth at clinical high-risk for psychosis

BACKGROUND

Elucidation of transphasic mechanisms (i.e., mechanisms that occur across illness phases) underlying negative symptoms could inform early intervention and prevention efforts and additionally identify treatment targets that could be effective regardless of illness stage. This study examined whether a key reinforcement learning behavioral pattern characterized by reduced difficulty learning from rewards that have been found to underlie negative symptoms in those with a schizophrenia diagnosis also contributes to negative symptoms in those at clinical high-risk (CHR) for psychosis.

METHODS

CHR youth (n = 46) and 51 healthy controls (CN) completed an explicit reinforcement learning task with two phases. During the acquisition phase, participants learned to select between pairs of stimuli probabilistically reinforced with feedback indicating receipt of monetary gains or avoidance of losses. Following training, the transfer phase required participants to select between pairs of previously presented stimuli during the acquisition phase and novel stimuli without receiving feedback. These test phase pairings allowed for inferences about the contributions of prediction error and value representation mechanisms to reinforcement learning deficits.

RESULTS

In acquisition, CHR participants displayed impaired learning from gains specifically that were associated with greater negative symptom severity. Transfer performance indicated these acquisition deficits were largely driven by value representation deficits. In addition to negative symptoms, this profile of deficits was associated with a greater risk of conversion to psychosis and lower functioning.

CONCLUSIONS

Impairments in positive reinforcement learning, specifically effectively representing reward value, may be an important transphasic mechanism of negative symptoms and a marker of psychosis liability.

Exploring the effectiveness of reward-based learning strategies for second-language speech sounds

Adults struggle to learn non-native speech categories in many experimental settings (Goto, Neuropsychologia, 9(3), 317-323 1971), but learn efficiently in a video game paradigm where non-native speech sounds have functional significance (Lim & Holt, Cognitive Science, 35(7), 1390-1405 2011). Behavioral and neural evidence from this and other paradigms point toward the involvement of reinforcement learning mechanisms in speech category learning (Harmon, Idemaru, & Kapatsinski, Cognition, 189, 76-88 2019; Lim, Fiez, & Holt, Proceedings of the National Academy of Sciences, 116, 201811992 2019). We formalize this hypothesis computationally and implement a deep reinforcement learning network to map between environmental input and actions. Comparing to a supervised model of learning, we show that the reinforcement network closely matches aspects of human behavior in two experiments - learning of synthesized auditory noise tokens and improvement in speech sound discrimination. Both models perform comparably and the similarity in the output of each model leads us to believe that there is little inherent computational benefit to a reward-based learning mechanism. We suggest that the specific neural circuitry engaged by the paradigm and links between striatum and superior temporal areas play a critical role in effective learning.

Invasive neurophysiological recordings in human basal ganglia. What have we learned about non-motor behaviour?

Research into the function of deep brain structures has benefited greatly from microelectrode recordings in animals. This has helped to unravel physiological processes in the healthy and malfunctioning brain. Translation to the human is necessary for improving basic understanding of subcortical structures and their implications in diseases. The use of microelectrode recordings as a standard component of deep brain stimulation surgery offers the most viable route for studying the electrophysiology of single cells and local neuronal populations in important deep structures of the human brain. Most of the studies in the basal ganglia have targeted the motor loop and movement disorder pathophysiology. In recent years, however, research has diversified to include limbic and cognitive processes. This review aims to provide an overview of advances in neuroscience made using intraoperative and post-operative recordings with a focus on non-motor activity in the basal ganglia.

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

Cognitive outcome and its neural correlates after cardiorespiratory arrest in childhood

Hypoxia-ischaemia (HI) can result in structural brain abnormalities, which in turn can lead to behavioural deficits in various cognitive and motor domains, in both adult and paediatric populations. Cardiorespiratory arrest (CA) is a major cause of hypoxia-ischaemia in adults, but it is relatively rare in infants and children. While the effects of adult CA on brain and cognition have been widely studied, to date, there are no studies examining the neurodevelopmental outcome of children who suffered CA early in life. Here, we studied the long-term outcome of 28 children who suffered early CA (i.e., before age 16). They were compared to a group of control participants (n = 28) matched for age, sex and socio-economic status. The patient group had impairments in the domains of memory, language and academic attainment (measured using standardised tests). Individual scores within the impaired range were most commonly found within the memory domain (79%), followed by academic attainment (50%), and language (36%). The patient group also had reduced whole brain grey matter volume, and reduced volume and fractional anisotropy of the white matter. In addition, lower performance on memory tests was correlated with bilaterally reduced volume of the hippocampi, thalami, and striatum, while lower attainment scores were correlated with bilateral reduction of fractional anisotropy in the superior cerebellar peduncle, the main output tract of the cerebellum. We conclude that patients who suffered early CA are at risk of developing specific cognitive deficits associated with structural brain abnormalities. RESEARCH HIGHLIGHTS: Our data shed light on the long-term outcome and associated neural mechanisms after paediatric hypoxia-ischaemia as a result of cardiorespiratory arrest. Patients had impaired scores on memory, language and academic attainment. Memory impairments were associated with smaller hippocampi, thalami, and striatum. Lower academic attainment correlated with reduced fractional anisotropy of the superior cerebellar peduncle.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Updating the striatal-pallidal wiring diagram

The striatal and pallidal complexes are basal ganglia structures that orchestrate learning and execution of flexible behavior. Models of how the basal ganglia subserve these functions have evolved considerably, and the advent of optogenetic and molecular tools has shed light on the heterogeneity of subcircuits within these pathways. However, a synthesis of how molecularly diverse neurons integrate into existing models of basal ganglia function is lacking. Here, we provide an overview of the neurochemical and molecular diversity of striatal and pallidal neurons and synthesize recent circuit connectivity studies in rodents that takes this diversity into account. We also highlight anatomical organizational principles that distinguish the dorsal and ventral basal ganglia pathways in rodents. Future work integrating the molecular and anatomical properties of striatal and pallidal subpopulations may resolve controversies regarding basal ganglia network function.

High intensity interval training exercise increases dopamine D2 levels and modulates brain dopamine signaling

Background

Previous research has outlined the health benefits of exercise including its therapeutic potential for substance use disorders (SUD). These data have already been utilized and it is now common to find exercise as part of SUD treatment and relapse prevention programs. However, we need to better understand different exercise regimens and determine which would be the most beneficial for SUDs. Recently, high intensity interval training (HIIT) has gained attention in comparison with aerobic and resistance exercise. Little is known regarding the neurobiological mechanisms of HIIT, including its effects on dopamine signaling and receptor levels in the brain. The present study examined the effects of chronic HIIT exercise on dopamine signaling as measured by dopamine type 1-like receptor (D1R)-like, dopamine type 2-like receptor (D2R)-like, and tyrosine hydroxylase (TH) quantification in the brains of male and female rats as measured by [3H] SCH 23390 and [3H] spiperone autoradiography, and TH-immunoreactive optical density values.

Methods

Rats were separated in two groups: sedentary and HIIT exercise. Exercise was on a treadmill for 30 min daily (10 3 min cycles) for six weeks with progressive speed increased up to 0.8 mph (21.5 m/min).

Results

Results showed for D2R-like binding, a significant effect across the ventral caudate putamen (V CPU) between sexes, such that mean D2R-like binding was 14% greater for males than females. In the nucleus accumbens shell (Nac Shell), the HIIT Exercise rats showed 16% greater D2R-like binding as compared to the sedentary rats. No significant effects of HIIT exercise were found across groups for brain D1R-like binding levels or TH expression.

Conclusion

These results suggest that HIIT exercise can modulate dopamine signaling by way of increased D2R. These findings support the premise that HIIT exercise plays an important role in dopamine signaling and, may provide a potential mechanism for how HIIT exercise can impact the brain and behavior.

Prefrontal transcranial magnetic stimulation boosts response vigour during reinforcement learning in healthy adults

A 10-Hz repetitive transcranial magnetic stimulation to the left dorsal lateral prefrontal cortex has been shown to increase dopaminergic activity in the dorsal striatum, a region strongly implicated in reinforcement learning. However, the behavioural influence of this effect remains largely unknown. We tested the causal effects of 10-Hz stimulation on behavioural and computational characteristics of reinforcement learning. A total of 40 healthy individuals were randomized into active and sham (placebo) stimulation groups. Each participant underwent one stimulation session (1500 pulses) in which stimulation was applied over the left dorsal lateral prefrontal cortex using a robotic arm. Participants then completed a reinforcement learning task sensitive to striatal dopamine functioning. Participants' choices were modelled using a reinforcement learning model (Q-learning) that calculates separate learning rates associated with positive and negative reward prediction errors. Subjects receiving active stimulation exhibited increased reward rate (number of correct responses per second of task activity) compared with those in sham. Computationally, although no group differences were observed, the active group displayed a higher learning rate for correct trials (αG) compared with incorrect trials (αL). Finally, when tested with novel pairs of stimuli, the active group displayed extremely fast reaction times, and a trend towards a higher reward rate. This study provided specific behavioural and computational accounts of altered striatal-mediated behaviour, particularly response vigour, induced by a proposed increase of dopamine activity by 10-Hz stimulation to the left dorsal lateral prefrontal cortex. Together, these findings bolster the use of repetitive transcranial magnetic stimulation to target neurocognitive disturbances attributed to the dysregulation of dopaminergic-striatal circuits.

Computational reinforcement learning, reward (and punishment), and dopamine in psychiatric disorders

In the DSM-5, psychiatric diagnoses are made based on self-reported symptoms and clinician-identified signs. Though helpful in choosing potential interventions based on the available regimens, this conceptualization of psychiatric diseases can limit basic science investigation into their underlying causes. The reward prediction error (RPE) hypothesis of dopamine neuron function posits that phasic dopamine signals encode the difference between the rewards a person expects and experiences. The computational framework from which this hypothesis was derived, temporal difference reinforcement learning (TDRL), is largely focused on reward processing rather than punishment learning. Many psychiatric disorders are characterized by aberrant behaviors, expectations, reward processing, and hypothesized dopaminergic signaling, but also characterized by suffering and the inability to change one's behavior despite negative consequences. In this review, we provide an overview of the RPE theory of phasic dopamine neuron activity and review the gains that have been made through the use of computational reinforcement learning theory as a framework for understanding changes in reward processing. The relative dearth of explicit accounts of punishment learning in computational reinforcement learning theory and its application in neuroscience is highlighted as a significant gap in current computational psychiatric research. Four disorders comprise the main focus of this review: two disorders of traditionally hypothesized hyperdopaminergic function, addiction and schizophrenia, followed by two disorders of traditionally hypothesized hypodopaminergic function, depression and post-traumatic stress disorder (PTSD). Insights gained from a reward processing based reinforcement learning framework about underlying dopaminergic mechanisms and the role of punishment learning (when available) are explored in each disorder. Concluding remarks focus on the future directions required to characterize neuropsychiatric disorders with a hypothesized cause of underlying dopaminergic transmission.

What is the contribution of voluntary and reflex processes to sensorimotor control of balance?

The contribution to balance of spinal and transcortical processes including the long-latency reflex is well known. The control of balance has been modelled previously as a continuous, state feedback controller representing, long-latency reflexes. However, the contribution of slower, variable delay processes has not been quantified. Compared with fixed delay processes (spinal, transcortical), we hypothesize that variable delay processes provide the largest contribution to balance and are sensitive to historical context as well as current states. Twenty-two healthy participants used a myoelectric control signal from their leg muscles to maintain balance of their own body while strapped to an actuated, inverted pendulum. We study the myoelectric control signal (u) in relation to the independent disturbance (d) comprising paired, discrete perturbations of varying inter-stimulus-interval (ISI). We fit the closed loop response, u from d, using one linear and two non-linear non-parametric (many parameter) models. Model M1 (ARX) is a generalized, high-order linear-time-invariant (LTI) process with fixed delay. Model M1 is equivalent to any parametric, closed-loop, continuous, linear-time-invariant (LTI), state feedback model. Model M2, a single non-linear process (fixed delay, time-varying amplitude), adds an optimized response amplitude to each stimulus. Model M3, two non-linear processes (one fixed delay, one variable delay, each of time-varying amplitude), add a second process of optimized delay and optimized response amplitude to each stimulus. At short ISI, the myoelectric control signals deviated systematically both from the fixed delay LTI process (M1), and also from the fixed delay, time-varying amplitude process (M2) and not from the two-process model (M3). Analysis of M3 (all fixed delay and variable delay response amplitudes) showed the variable (compared with fixed) delay process 1) made the largest contribution to the response, 2) exhibited refractoriness (increased delay related to short ISI) and 3) was sensitive to stimulus history (stimulus direction 2 relative to stimulus 1). For this whole-body balance task and for these impulsive stimuli, non-linear processes at variable delay are central to control of balance. Compared with fixed delay processes (spinal, transcortical), variable delay processes provided the largest contribution to balance and were sensitive to historical context as well as current states.

Cerebellar and basal ganglia motor network predicts trait depression and hyperactivity

In the human brain, the cerebellum (CB) and basal ganglia (BG) are implicated in cognition-, emotion-, and motor-related cortical processes and are highly interconnected, both to cortical regions via separate, trans-thalamic pathways and to each other via subcortical disynaptic pathways. We previously demonstrated a distinction between cognitive and motor CB-BG networks (CCBN, MCBN, respectively) as it relates to cortical network integration in healthy young adults, suggesting the subcortical networks separately support cortical networks. The CB and BG are also implicated in the pathophysiology of schizophrenia, Parkinson's, and compulsive behavior; thus, integration within subcortical CB-BG networks may be related to transdiagnostic symptomology. Here, we asked whether CCBN or MCBN integration predicted Achenbach Self-Report scores for anxiety, depression, intrusive thoughts, hyperactivity and inactivity, and cognitive performance in a community sample of young adults. We computed global efficiency for each CB-BG network and 7 canonical resting-state networks for all right-handed participants in the Human Connectome Project 1200 release with a complete set of preprocessed resting-state functional MRI data (N = 783). We used multivariate regression to control for substance abuse and age, and permutation testing with exchangeability blocks to control for family relationships. MCBN integration negatively predicted depression and hyperactivity, and positively predicted cortical network integration. CCBN integration predicted cortical network integration (except for the emotional network) and marginally predicted a positive relationship with hyperactivity, indicating a potential dichotomy between cognitive and motor CB-BG networks and hyperactivity. These results highlight the importance of CB-BG interactions as they relate to motivation and symptoms of depression.

Individualized Functional Subnetworks Connect Human Striatum and Frontal Cortex

The striatum and cerebral cortex are interconnected via multiple recurrent loops that play a major role in many neuropsychiatric conditions. Primate corticostriatal connections can be precisely mapped using invasive tract-tracing. However, noninvasive human research has not mapped these connections with anatomical precision, limited in part by the practice of averaging neuroimaging data across individuals. Here we utilized highly sampled resting-state functional connectivity MRI for individual-specific precision functional mapping (PFM) of corticostriatal connections. We identified ten individual-specific subnetworks linking cortex-predominately frontal cortex-to striatum, most of which converged with nonhuman primate tract-tracing work. These included separable connections between nucleus accumbens core/shell and orbitofrontal/medial frontal gyrus; between anterior striatum and dorsomedial prefrontal cortex; between dorsal caudate and lateral prefrontal cortex; and between middle/posterior putamen and supplementary motor/primary motor cortex. Two subnetworks that did not converge with nonhuman primates were connected to cortical regions associated with human language function. Thus, precision subnetworks identify detailed, individual-specific, neurobiologically plausible corticostriatal connectivity that includes human-specific language networks.

Dissecting Motor and Cognitive Component Processes of a Finger-Tapping Task With Hybrid Dopamine Positron Emission Tomography and Functional Magnetic Resonance Imaging

Striatal dopamine is involved in facilitation of motor action as well as various cognitive and emotional functions. Positron emission tomography (PET) is the primary imaging method used to investigate dopamine function in humans. Previous PET studies have shown striatal dopamine release during simple finger tapping in both the putamen and the caudate. It is likely that dopamine release in the putamen is related to motor processes while dopamine release in the caudate could signal sustained cognitive component processes of the task, but the poor temporal resolution of PET has hindered firm conclusions. In this study we simultaneously collected [11C]Raclopride PET and functional Magnetic Resonance Imaging (fMRI) data while participants performed finger tapping, with fMRI being able to isolate activations related to individual tapping events. The results revealed fMRI-PET overlap in the bilateral putamen, which is consistent with a motor component process. Selective PET responses in the caudate, ventral striatum, and right posterior putamen, were also observed but did not overlap with fMRI responses to tapping events, suggesting that these reflect non-motor component processes of finger tapping. Our findings suggest an interplay between motor and non-motor-related dopamine release during simple finger tapping and illustrate the potential of hybrid PET-fMRI in revealing distinct component processes of cognitive functions.

Diverse midbrain dopaminergic neuron subtypes and implications for complex clinical symptoms of Parkinson's disease

Parkinson’s disease (PD), the most common degenerative movement disorder, is clinically manifested with various motor and non-motor symptoms. Degeneration of midbrain substantia nigra pas compacta (SNc) dopaminergic neurons (DANs) is generally attributed to the motor syndrome. The underlying neuronal mechanisms of non-motor syndrome are largely unexplored. Besides SNc, midbrain ventral tegmental area (VTA) DANs also produce and release dopamine and modulate movement, reward, motivation, and memory. Degeneration of VTA DANs also occurs in postmortem brains of PD patients, implying an involvement of VTA DANs in PD-associated non-motor symptoms. However, it remains to be established that there is a distinct segregation of different SNc and VTA DAN subtypes in regulating different motor and non-motor functions, and that different DAN subpopulations are differentially affected by normal ageing or PD. Traditionally, the distinction among different DAN subtypes was mainly based on the location of cell bodies and axon terminals. With the recent advance of single cell RNA sequencing technology, DANs can be readily classified based on unique gene expression profiles. A combination of specific anatomic and molecular markers shows great promise to facilitate the identification of DAN subpopulations corresponding to different behavior modules under normal and disease conditions. In this review, we first summarize the recent progress in characterizing genetically, anatomically, and functionally diverse midbrain DAN subtypes. Then, we provide perspectives on how the preclinical research on the connectivity and functionality of DAN subpopulations improves our current understanding of cell-type and circuit specific mechanisms of the disease, which could be critically informative for designing new mechanistic treatments.

Corticostriatal Regulation of Language Functions

The role of corticostriatal circuits in language functions is unclear. In this review, we consider evidence from language learning, syntax, and controlled language production and comprehension tasks that implicate various corticostriatal circuits. Converging evidence from neuroimaging in healthy individuals, studies in populations with subcortical dysfunction, pharmacological studies, and brain stimulation suggests a domain-general regulatory role of corticostriatal systems in language operations. The role of corticostriatal systems in language operations identified in this review is likely to reflect a broader function of the striatum in responding to uncertainty and conflict which demands selection, sequencing, and cognitive control. We argue that this role is dynamic and varies depending on the degree and form of cognitive control required, which in turn will recruit particular corticostriatal circuits and components organised in a cognitive hierarchy.

Accumbens Cholinergic Interneurons Mediate Cue-Induced Nicotine Seeking and Associated Glutamatergic Plasticity

Nicotine, the primary addictive substance in tobacco, is widely abused. Relapse to cues associated with nicotine results in increased glutamate release within nucleus accumbens core (NAcore), modifying synaptic plasticity of medium spiny neurons (MSNs), which contributes to reinstatement of nicotine seeking. However, the role of cholinergic interneurons (ChIs) within the NAcore in mediating these neurobehavioral processes is unknown. ChIs represent less than 1% of the accumbens neuronal population and are activated during drug seeking and reward-predicting events. Thus, we hypothesized that ChIs may play a significant role in mediating glutamatergic plasticity that underlies nicotine-seeking behavior. Using chemogenetics in transgenic rats expressing Cre under the control of the choline acetyltransferase (ChAT) promoter, ChIs were bidirectionally manipulated before cue-induced reinstatement. Following nicotine self-administration and extinction, ChIs were activated or inhibited before a cue reinstatement session. Following reinstatement, whole-cell electrophysiology from NAcore MSNs was used to assess changes in plasticity, measured via AMPA/NMDA (A/N) ratios. Chemogenetic inhibition of ChIs inhibited cued nicotine seeking and resulted in decreased A/N, relative to control animals, whereas activation of ChIs was unaltered, demonstrating that ChI inhibition may modulate plasticity underlying cue-induced nicotine seeking. These results demonstrate that ChI neurons play an important role in mediating cue-induced nicotine reinstatement and underlying synaptic plasticity within the NAcore.

Brain features of nearly drug-naïve female monozygotic twins with first-episode schizophrenia and the classification accuracy of brain feature patterns: A pilot study

BACKGROUND

Data on differences in brain features between monozygotic (MZ) twins with and without schizophrenia are scarce.

METHODS

We compared brain features of female MZ twins with and without first-episode schizophrenia and healthy controls (n = 20 each). Voxel-based morphometry and tract-based spatial statistics were used to analyze differences in brain structure. Whole-brain effective connectivity (EC) and functional connectivity (FC) networks were constructed using resting-state functional magnetic resonance imaging (rs-fMRI) data.

RESULTS

Female twins with schizophrenia exhibited abnormal gray matter volume (GMV) in the basal ganglia and prefrontal and parietal cortices, impairments in the arcuate fasciculus, and significant disruptions (primarily decreases) in nine EC networks. They exhibited rs-EC alterations involving the limbic areas and subcortex. Combined rs-EC and rs-FC data distinguished twins with first-episode schizophrenia with high accuracy. Combined consideration of structural and functional features enabled the distinction of female MZ twins with schizophrenia from those without schizophrenia and healthy controls with 100% accuracy.

CONCLUSIONS

Female MZ twins with schizophrenia exhibited increased GMV, white matter impairment, and disruptions in EC and FC networks. The combination of rs-EC + rs-FC data could distinguish female twins with schizophrenia from twins without schizophrenia and healthy controls with 97.4% accuracy, and the addition of structural brain features yielded a 100% accuracy rate. These findings may provide pivotal insight for further study of the mechanisms underlying schizophrenia.

Episodic memory governs choices: An RNN-based reinforcement learning model for decision-making task

Typical methods to study cognitive function are to record the electrical activities of animal neurons during the training of animals performing behavioral tasks. A key problem is that they fail to record all the relevant neurons in the animal brain. To alleviate this problem, we develop an RNN-based Actor-Critic framework, which is trained through reinforcement learning (RL) to solve two tasks analogous to the monkeys' decision-making tasks. The trained model is capable of reproducing some features of neural activities recorded from animal brain, or some behavior properties exhibited in animal experiments, suggesting that it can serve as a computational platform to explore other cognitive functions. Furthermore, we conduct behavioral experiments on our framework, trying to explore an open question in neuroscience: which episodic memory in the hippocampus should be selected to ultimately govern future decisions. We find that the retrieval of salient events sampled from episodic memories can effectively shorten deliberation time than common events in the decision-making process. The results indicate that salient events stored in the hippocampus could be prioritized to propagate reward information, and thus allow decision-makers to learn a strategy faster.

Phenotypic evaluation of a childhood-onset parkinsonism-dystonia mouse model with inherent postural abnormalities

Mouse models that replicate facets of human neurological diseases are often used at the pre-clinical stage to better understand the underlying mechanisms of a disease and test the target engagement of potential therapeutic interventions. We recently characterized a mouse model of childhood-onset parkinsonism-dystonia, a disease caused by a homozygous loss-of-function mutation in the SLC39A14 gene. The disease manifests itself phenotypically by impairments in locomotor behaviour and postural abnormalities. Our initial characterization of the model revealed that the Slc39a14^-/- mice showed altered Mn homeostasis and compromised locomotor performance in vertical pole-descending, horizontal beam-traversing, and rotarod tests (Jenkitkasemwong et al., 2018). However, some of the mice also displayed torticollis and Straub tail. In this study, we investigated whether these postural abnormalities affected the performance in the above motility tests and consequently, biased and compromised the external validity of reported abnormal locomotor profiles. Our analyses showed that the Slc39a14^-/- mice displaying torticollis and/or Straub tail had tests scores comparable to scores of their counterparts that never displayed these postural abnormalities. The z-score general index of performance revealed that the Slc39a14^-/- model presents a complex pathological motor phenotype relevant to the complexity of phenotypes identified in childhood-onset parkinsonism-dystonia.

Dopamine D2 receptor signaling on iMSNs is required for initiation and vigor of learned actions

Striatal dopamine D2 receptors (D2Rs) are important for motor output. Selective deletion of D2Rs from indirect pathway-projecting medium spiny neurons (iMSNs) impairs locomotor activities in a task-specific manner. However, the role of D2Rs in the initiation of motor actions in reward seeking and taking is not fully understood, and there is little information about how receptors contribute under different task demands and with different outcome types. The iMSN-D2Rs modulate neuronal activity and synaptic transmission, exerting control on circuit functions that may play distinct roles in action learning and performance. Selective deletion of D2Rs on iMSNs resulted in slower action initiation and response rate in an instrumental conditioning task, but only when performance demand was increased. The iMSN-Drd2KO mice were also slower to initiate swimming in a T-maze procedural learning task but were unimpaired in cognitive function and behavioral flexibility. In contrast, in a Pavlovian discrimination learning task, iMSN-Drd2KO mice exhibited normal acquisition and extinction of rewarded responding. The iMSN-Drd2KO mice showed performance deficits at all phases of rotarod skill learning. These findings reveal that dopamine modulation through iMSN-D2Rs influences the ability to self-initiate actions, as well as the willingness and/or vigor with which these responses are performed. However, these receptors seem to have little influence on simple associative learning or on stimulus-driven responding. The loss of normal D2R roles may contribute to disorders in which impaired dopamine signaling leads to hypokinesia or impaired initiation of specific voluntary actions.

The effect of feedback novelty on neural correlates of feedback processing

It has been suggested that stimulus novelty itself can be rewarding and recent evidence suggests that novelty processing and reward processing share common neural mechanisms. For feedback processing, this can be beneficial as well as detrimental: If novelty lends a rewarding characteristic to a stimulus, then this should particularly decrease the impact of negative feedback. The present study investigated whether such an effect of feedback novelty on feedback processing is reflected in electrophysiological markers of reinforcement learning (feedback-related negativity, FRN) and feedback processing (feedback-P300) in a simple decision-making task. In this task, participants had to chose between two stimuli in a learning trial followed by a novel or a familiar feedback stimulus. Learning from feedback allowed them to optimize their payoff in a later test trial. As expected, we found that the FRN effect, i.e. the difference between the FRN amplitudes after negative and positive feedback, was reduced for novel compared to familiar feedback stimuli. In addition, the amplitude of the feedback-P300 was decreased by feedback novelty, both for the anterior P3a and the posterior P3b. Together, these results indicate that feedback novelty can affect feedback processing as reflected by feedback-related brain activity.

The role of dopamine pharmacotherapy and addiction-like behaviors in Parkinson's disease

Addictions involve a spectrum of behaviors that encompass features of impulsivity and compulsivity, herein referred to as impulsive-compulsive spectrum disorders (ICSDs). The etiology of ICSDs likely involves a complex interplay among neurobiological, psychological and social risk factors. Neurobiological risk factors include the status of the neuroanatomical circuits that govern ICSDs. These circuits can be altered by disease, as well as exogenous influences such as centrally-acting pharmacologics. The 'poster child' for this scenario is Parkinson's disease (PD) medically managed by pharmacological treatments. PD is a progressive neurodegenerative disease that involves a gradual loss of dopaminergic neurons largely within nigrostriatal projections. Replacement therapy includes dopamine receptor agonists that directly activate postsynaptic dopamine receptors (bypassing the requirement for functioning presynaptic terminals). Some clinically useful dopamine agonists, e.g., pramipexole and ropinirole, exhibit high affinity for the D2/D3 receptor subtypes. These agonists provide excellent relief from PD motor symptoms, but some patients exhibit debilitating ICSD. Teasing out the neuropsychiatric contribution of PD-associated pathology from the drugs used to treat PD motor symptoms is challenging. In this review, we posit that modern clinical and preclinical research converge on the conclusion that dopamine replacement therapy can mediate addictions in PD and other neurological disorders. We provide five categories of evidences that align with this position: (i) ICSD prevalence is greater with D2/D3 receptor agonist therapy vs PD alone. (ii) Capacity of dopamine replacement therapy to produce addiction-like behaviors is independent of disease for which the therapy is being provided. (iii) ICSD-like behaviors are recapitulated in laboratory rats with and without PD-like pathology. (iv) Behavioral pathology co-varies with drug exposure. (v) ICSD Features of ICSDs are consistent with agonist pharmacology and neuroanatomical substrates of addictions. Considering the underpinnings of ICSDs in PD should not only help therapeutic decision-making in neurological disorders, but also apprise ICSDs in general.

Mathematical modeling and parameter estimation of levodopa motor response in patients with parkinson disease

Parkinson disease (PD) is characterized by a clear beneficial motor response to levodopa (LD) treatment. However, with disease progression and longer LD exposure, drug-related motor fluctuations usually occur. Recognition of the individual relationship between LD concentration and its effect may be difficult, due to the complexity and variability of the mechanisms involved. This work proposes an innovative procedure for the automatic estimation of LD pharmacokinetics and pharmacodynamics parameters, by a biologically-inspired mathematical model. An original issue, compared with previous similar studies, is that the model comprises not only a compartmental description of LD pharmacokinetics in plasma and its effect on the striatal neurons, but also a neurocomputational model of basal ganglia action selection. Parameter estimation was achieved on 26 patients (13 with stable and 13 with fluctuating LD response) to mimic plasma LD concentration and alternate finger tapping frequency along four hours after LD administration, automatically minimizing a cost function of the difference between simulated and clinical data points. Results show that individual data can be satisfactorily simulated in all patients and that significant differences exist in the estimated parameters between the two groups. Specifically, the drug removal rate from the effect compartment, and the Hill coefficient of the concentration-effect relationship were significantly higher in the fluctuating than in the stable group. The model, with individualized parameters, may be used to reach a deeper comprehension of the PD mechanisms, mimic the effect of medication, and, based on the predicted neural responses, plan the correct management and design innovative therapeutic procedures.

Respiratory regulation & interactions with neuro-cognitive circuitry

It is increasingly being recognized that active control of breathing - a key aspect of ancient Vedic meditative practices, can relieve stress and anxiety and improve cognition. However, the underlying mechanisms of respiratory modulation of neurophysiology are just beginning to be elucidated. Research shows that brainstem circuits involved in the motor control of respiration receive input from and can directly modulate activity in subcortical circuits, affecting emotion and arousal. Meanwhile, brain regions involved in the sensory aspects of respiration, such as the olfactory bulb, are like-wise linked with wide-spread brain oscillations; and perturbing olfactory bulb activity can significantly affect both mood and cognition. Thus, via both motor and sensory pathways, there are clear mechanisms by which brain activity is entrained to the respiratory cycle. Here, we review evidence gathered across multiple species demonstrating the links between respiration, entrainment of brain activity and functional relevance for affecting mood and cognition. We also discuss further linkages with cardiac rhythms, and the potential translational implications for biorhythm monitoring and regulation in neuropsychiatric disorders.

Role of dopamine D1-like and D2-like receptors in the activation of ingestive behaviour in thirsty rats licking for water

RATIONALE

Analysis of lick pattern for sucrose and NaCl and of the forced swimming response after dopamine antagonist administration led us to suggest that dopamine on D₁-like receptors is involved in behavioural activation, and the level of activation is "reboosted" on the basis of an evaluation process involving D₂-like receptors. Although some studies investigated licking microstructure for water after dopamine antagonists, the within-session time course of their effect was never investigated.

OBJECTIVES

The aims of this study were to further investigate the role of dopamine receptors in the mechanisms governing water ingestion, focussing on the within-session time course of the microstructure parameters, and to test the proposed hypothesis.

MATERIALS AND METHODS

The effects of the dopamine D₁-like receptor antagonist SCH 23390 (0.01-0.04 mg/kg) and of the dopamine D₂-like receptor antagonist raclopride (0.025-0.25 mg/kg) on licking microstructure for water were examined in 20-h water-deprived rats in 30-min sessions.

RESULTS

As previously observed with sucrose and NaCl, SCH 23390 reduced licking by reducing burst number, suggesting reduced behavioural activation. Moreover, it resulted in an increased burst size. Raclopride reduced the size of licking bursts, while their number was either increased or decreased depending on the dose.

CONCLUSION

The results support the suggestion that D₁ receptors are involved in behavioural activation and D₂ receptors are involved in a related evaluation process. Within the framework of the proposed hypothesis, the increased burst size after D₁-like receptor blockade might be interpreted as a pro-hedonic effect consequent to the increased cost of the activation of the licking response.

Spontaneous eye blink rate predicts individual differences in exploration and exploitation during reinforcement learning

Spontaneous eye blink rate (sEBR) has been linked to striatal dopamine function and to how individuals make value-based choices after a period of reinforcement learning (RL). While sEBR is thought to reflect how individuals learn from the negative outcomes of their choices, this idea has not been tested explicitly. This study assessed how individual differences in sEBR relate to learning by focusing on the cognitive processes that drive RL. Using Bayesian latent mixture modelling to quantify the mapping between RL behaviour and its underlying cognitive processes, we were able to differentiate low and high sEBR individuals at the level of these cognitive processes. Further inspection of these cognitive processes indicated that sEBR uniquely indexed explore-exploit tendencies during RL: lower sEBR predicted exploitative choices for high valued options, whereas higher sEBR predicted exploration of lower value options. This relationship was additionally supported by a network analysis where, notably, no link was observed between sEBR and how individuals learned from negative outcomes. Our findings challenge the notion that sEBR predicts learning from negative outcomes during RL, and suggest that sEBR predicts individual explore-exploit tendencies. These then influence value sensitivity during choices to support successful performance when facing uncertain reward.

White Matter Hyperintensities Relate to Basal Ganglia Functional Connectivity and Memory Performance in aMCI and SVMCI

Cerebral small vessel diseases play a crucial role in both vascular and non-vascular dementias. The location of white matter hyperintensities (WMHs), a neuroimaging marker of cerebral small vessel disease, has been found to vary between different types of dementias, and those in the basal ganglia (BG) have been particularly associated with vascular cognitive impairment (VCI). However, anatomical variation of WMHs across BG nuclei and its effect on brain network dysconnectivity has not been clearly elucidated. The study sample consisted of 40 patients with amnestic mild cognitive impairment (aMCI), 40 with subcortical vascular MCI (SVMCI), and 40 healthy control subjects. We examined the volume of WMH using T2-weighted magnetic resonance imaging. We also assessed the disturbances in BG-cortical communication by measuring resting-state functional connectivity (rsFC) from the functional magnetic resonance imaging signal. WMHs were more pronounced in the SVMCI group particularly in the caudate regions. In SVMCI patients, while higher WMHs in the dorsal caudate correlated with weaker FC with executive control regions and worse immediate recall performance, WMHs in the ventral caudate were associated with weaker FC with anterior default mode regions and worse delayed recall performance. In contrast, in aMCI patients, BG WMHs were not correlated with their changes in functional connectivity changes, which showed weaker connectivity with almost all BG structures, rather than restricting to specific BG subdivisions as observed in the SVMCI group. Our findings demonstrate that heterogeneously distributed BG WMHs are associated with changes in functional network interactions and verbal episodic memory performance only in SVMCI patients, which establishes a link between cerebrovascular-related structural abnormality, functional integrity of BG circuits, and episodic memory impairments in SVMCI, and may reflect a differential role of the cerebrovascular pathology in disrupting network-level communications and cognition between Alzheimer's and subcortical vascular dementia.

Humans strategically shift decision bias by flexibly adjusting sensory evidence accumulation

Decision bias is traditionally conceptualized as an internal reference against which sensory evidence is compared. Instead, we show that individuals implement decision bias by shifting the rate of sensory evidence accumulation toward a decision bound. Participants performed a target detection task while we recorded EEG. We experimentally manipulated participants' decision criterion for reporting targets using different stimulus-response reward contingencies, inducing either a liberal or a conservative bias. Drift diffusion modeling revealed that a liberal strategy biased sensory evidence accumulation toward target-present choices. Moreover, a liberal bias resulted in stronger midfrontal pre-stimulus 2-6 Hz (theta) power and suppression of pre-stimulus 8-12 Hz (alpha) power in posterior cortex. Alpha suppression in turn was linked to the output activity in visual cortex, as expressed through 59-100 Hz (gamma) power. These findings show that observers can intentionally control cortical excitability to strategically bias evidence accumulation toward the decision bound that maximizes reward.

Dopaminergic Modulation of Synaptic Plasticity, Its Role in Neuropsychiatric Disorders, and Its Computational Modeling

Neuromodulators modify intrinsic characteristics of the nervous system in order to reconfigure the functional properties of neural circuits. This reconfiguration is crucial for the flexibility of the nervous system to respond on an input-modulated basis. Such a functional rearrangement is realized by modification of intrinsic properties of the neural circuits including synaptic interactions. Dopamine is an important neuromodulator involved in motivation and stimulus-reward learning process, and adjusts synaptic dynamics in multiple time scales through different pathways. The modification of synaptic plasticity by dopamine underlies the change in synaptic transmission and integration mechanisms, which affects intrinsic properties of the neural system including membrane excitability, probability of neurotransmitters release, receptors’ response to neurotransmitters, protein trafficking, and gene transcription. Dopamine also plays a central role in behavioral control, whereas its malfunction can cause cognitive disorders. Impaired dopamine signaling is implicated in several neuropsychiatric disorders such as Parkinson’s disease, drug addiction, schizophrenia, attention-deficit/hyperactivity disorder, obsessive-compulsive disorder and Tourette’s syndrome. Therefore, dopamine plays a crucial role in the nervous system, where its proper modulation of neural circuits may enhance plasticity-related procedures, but disturbances in dopamine signaling might be involved in numerous neuropsychiatric disorders. In recent years, several computational models are proposed to formulate the involvement of dopamine in synaptic plasticity or neuropsychiatric disorders and address their connection based on the experimental findings.

Abnormal phasic activity in saliency network, motor areas, and basal ganglia in Parkinson's disease during rhythm perception

Behavioral studies indicate that persons with Parkinson's disease have complexity dependent problems with the discrimination of auditory rhythms. Furthermore, neuroimaging studies show that rhythm processing activates many brain areas that overlap with areas affected by Parkinson's disease (PD). This study sought to investigate the neural correlates of rhythm processing in PD and healthy controls, with a particular focus on rhythmic complexity. We further aimed to investigate differences in brain activation during initial phases of rhythm processing. Functional magnetic resonance imaging was used to scan 15 persons with Parkinson's disease and 15 healthy controls while they listened to musical rhythms with two different levels of complexity. Rhythmic complexity had no significant effect on brain activations, but patients and controls showed differences in areas related to temporal auditory processing, notably bilateral planum temporale and inferior parietal lobule. We found indications of a particular sequential or phasic activation pattern of brain activity, where activity in caudate nucleus in the basal ganglia was time‐displaced by activation in the saliency network—comprised of anterior cingulate cortex and bilateral anterior insula—and cortical and subcortical motor areas, during the initial phases of listening to rhythms. We relate our findings to core PD pathology, and discuss the overall, rhythm processing related hyperactivity in PD as a possible dysfunction in specific basal ganglia mechanisms, and the phasic activation pattern in PD as a reflection of a lack of preparatory activation of task‐relevant brain networks for rhythm processing in PD.

A competitive model for striatal action selection

The direct and indirect pathway striatal medium spiny neurons (dMSNs and iMSNs) have long been linked to action selection, but the precise roles of these neurons in this process remain unclear. Here, we review different models of striatal pathway function, focusing on the classic "go/no-go" model which posits that dMSNs facilitate movement while iMSNs inhibit movement, and the "complementary" model, which argues that dMSNs facilitate the selection of specific actions while iMSNs inhibit potentially conflicting actions. We discuss the merits and shortcomings of these models and propose a "competitive" model to explain the contribution of these two pathways to behavior. The "competitive" model argues that rather than inhibiting conflicting actions, iMSNs are tuned to the same actions that dMSNs facilitate, and the two populations "compete" to determine the animal's behavioral response. This model provides a theoretical explanation for how these pathways work together to select actions. In addition, it provides a link between action selection and behavioral reinforcement, via modulating synaptic strength at inputs onto dMSNs and iMSNs. Finally, this model makes predictions about how imbalances in the activity of these pathways may underlie behavioral traits associated with psychiatric disorders. Understanding the roles of these striatal pathways in action selection may help to clarify the neuronal mechanisms of decision-making under normal and pathological conditions.

A Preliminary High-Definition Fiber Tracking Study of the Executive Control Network in Blast-Induced Traumatic Brain Injury

Blast-induced traumatic brain injury (bTBI) is common in veterans of the Iraq- and Afghanistan-era conflicts. However, the typical subtlety of neural alterations and absence of definitive biomarkers impede clinical detection on conventional imaging. This preliminary study examined the structure and functional correlates of executive control network (ECN) white matter in veterans to investigate the clinical utility of using high-definition fiber tracking (HDFT) to detect chronic bTBI. Demographically similar male veterans (N = 38) with and without bTBI (ages 24 to 50 years) completed standardized neuropsychological testing and magnetic resonance imaging. Quantitative HDFT metrics of subcortical-dorsolateral prefrontal cortex (DLPFC) tracts were derived. Moderate-to-large group effects were observed on HDFT metrics. Relative to comparisons, bTBI demonstrated elevated quantitative anisotropy (QA) and reduced right hemisphere volume of all examined tracts, and reduced fiber count and increased generalized fractional anisotropy in the right DLPFC-putamen tract and DLPFC-thalamus, respectively. The Group × Age interaction effect on DLPFC-caudate tract volume was large; age negatively related to volume in the bTBI group, but not comparison group. Groups performed similarly on the response inhibition measure. Performance (reaction time and commission errors) robustly correlated with HDFT tract metrics (QA and tract volume) in the comparison group, but not bTBI group. Results support anomalous density and integrity of ECN connectivity, particularly of the right DLPFC-putamen pathway, in bTBI. Results also support exacerbated aging in veterans with bTBI. Similar ECN function despite anomalous microstructure could reflect functional compensation in bTBI, although alternate interpretations are explored.

A Multilevel Computational Characterization of Endophenotypes in Addiction

Addiction is characterized by a profound intersubject (phenotypic) variability in the expression of addictive symptomatology and propensity to relapse following treatment. However, laboratory investigations have primarily focused on common neural substrates in addiction and have not yet been able to identify mechanisms that can account for the multifaceted phenotypic behaviors reported in the literature. To fill this knowledge gap theoretically, here we simulated phenotypic variations in addiction symptomology and responses to putative treatments, using both a neural model, based on cortico-striatal circuit dynamics, and an algorithmic model of reinforcement learning (RL). These simulations rely on the widely accepted assumption that both the ventral, model-based, goal-directed system and the dorsal, model-free, habitual system are vulnerable to extra-physiologic dopamine reinforcements triggered by addictive rewards. We found that endophenotypic differences in the balance between the two circuit or control systems resulted in an inverted-U shape in optimal choice behavior. Specifically, greater unbalance led to a higher likelihood of developing addiction and more severe drug-taking behaviors. Furthermore, endophenotypes with opposite asymmetrical biases among cortico-striatal circuits expressed similar addiction behaviors, but responded differently to simulated treatments, suggesting personalized treatment development could rely on endophenotypic rather than phenotypic differentiations. We propose our simulated results, confirmed across neural and algorithmic levels of analysis, inform on a fundamental and, to date, neglected quantitative method to characterize clinical heterogeneity in addiction.

The Basal Ganglia: More than just a switching device

The basal ganglia consist of a variety of subcortical nuclei engaged in motor control and executive functions, such as motor learning, behavioral control, and emotion. The striatum, a major basal ganglia component, is particularly useful for cognitive planning of purposive motor acts owing to its structural features and the neuronal circuitry established with the cerebral cortex. Recent data indicate emergent functions played by the striatum. Indeed, cortico-striatal circuits carrying motor information are paralleled by circuits originating from associative and limbic territories, which are functionally integrated in the striatum. Functional integration between brain areas is achieved through patterns of coherent activity. Coherence belonging to cortico-basal ganglia circuits is also present in Parkinson's disease patients. Excessive synchronization occurring in this pathology is reduced by dopaminergic therapies. The mechanisms through which the dopaminergic effects may be addressed are the object of several ongoing investigations. Overall, the bulk of data reported in recent years has provided new vistas concerning basal ganglia role in the organization and control of movement and behavior, both in physiological and pathological conditions. In this review, basal ganglia functions involved in the organization of main movement categories and behaviors are critically discussed. Comparatively, the multiplicity of Parkinson's disease symptomatology is also revised.

A neurocomputational investigation of reinforcement-based decision making as a candidate latent vulnerability mechanism in maltreated children

Alterations in reinforcement-based decision making may be associated with increased psychiatric vulnerability in children who have experienced maltreatment. A probabilistic passive avoidance task and a model-based functional magnetic resonance imaging analytic approach were implemented to assess the neurocomputational components underlying decision making: (a) reinforcement expectancies (the representation of the outcomes associated with a stimulus) and (b) prediction error signaling (the ability to detect the differences between expected and actual outcomes). There were three main findings. First, the maltreated group (n = 18; mean age = 13), relative to nonmaltreated peers (n = 19; mean age = 13), showed decreased activity during expected value processing in a widespread network commonly associated with reinforcement expectancies representation, including the striatum (especially the caudate), the orbitofrontal cortex, and medial temporal structures including the hippocampus and insula. Second, consistent with previously reported hyperresponsiveness to negative cues in the context of childhood abuse, the maltreated group showed increased prediction error signaling in the middle cingulate gyrus, somatosensory cortex, superior temporal gyrus, and thalamus. Third, the maltreated group showed increased activity in frontodorsal regions and in the putamen during expected value representation. These findings suggest that early adverse environments disrupt the development of decision-making processes, which in turn may compromise psychosocial functioning in ways that increase latent vulnerability to psychiatric disorder.

Striatal Local Circuitry: A New Framework for Lateral Inhibition

This Perspective will examine the organization of intrastriatal circuitry, review recent findings in this area, and discuss how the pattern of connectivity between striatal neurons might give rise to the behaviorally observed synergism between the direct/indirect pathway neurons. The emphasis of this Perspective is on the underappreciated role of lateral inhibition between striatal projection cells in controlling neuronal firing and shaping the output of this circuit. We review some classic studies in combination with more recent anatomical and functional findings to lay out a framework for an updated model of the intrastriatal lateral inhibition, where we explore its contribution to the formation of functional units of processing and the integration and filtering of inputs to generate motor patterns and learned behaviors.

Evidence for a task-dependent switch in subthalamo-nigral basal ganglia signaling

Basal ganglia (BG) can either facilitate or inhibit movement through excitatory and inhibitory pathways; however whether these opposing signals are dynamically regulated during healthy behavior is not known. Here, we present compelling neurophysiological evidence from three complimentary experiments in non-human primates, indicating task-specific changes in tonic BG pathway weightings during saccade behavior with different cognitive demands. First, simultaneous local field potential recording in the subthalamic nucleus (STN; BG input) and substantia nigra pars reticulata (SNr; BG output) reveals task-dependent shifts in subthalamo-nigral signals. Second, unilateral electrical stimulation of the STN, SNr, and caudate nucleus results in strikingly different saccade directionality and latency biases across the BG. Third, a simple artificial neural network representing canonical BG signaling pathways suggests that pathway weightings can be altered by cortico-BG input activation. Overall, inhibitory pathways (striato-pallidal-subthalamo-nigral) dominate during goal-driven behavior with instructed rewards, while facilitatory pathways (striato-nigral and subthalamo-pallidal-nigral) dominate during unconstrained (free reward) conditions.

Basal ganglia can both facilitate or inhibit movement through excitatory and inhibitory pathways; however whether these opposing signals are dynamically regulated during behavior is not known. Here the authors use multinucleus LFP recordings and electrical microstimulation in monkeys performing saccade based tasks to show task specific changes in the tonic weighting of these pathways.

Visuo-manual tracking: does intermittent control with aperiodic sampling explain linear power and non-linear remnant without sensorimotor noise?

KEY POINTS

A human controlling an external system is described most easily and conventionally as linearly and continuously translating sensory input to motor output, with the inevitable output remnant, non-linearly related to the input, attributed to sensorimotor noise. Recent experiments show sustained manual tracking involves repeated refractoriness (insensitivity to sensory information for a certain duration), with the temporary 200-500 ms periods of irresponsiveness to sensory input making the control process intrinsically non-linear. This evidence calls for re-examination of the extent to which random sensorimotor noise is required to explain the non-linear remnant. This investigation of manual tracking shows how the full motor output (linear component and remnant) can be explained mechanistically by aperiodic sampling triggered by prediction error thresholds. Whereas broadband physiological noise is general to all processes, aperiodic sampling is associated with sensorimotor decision making within specific frontal, striatal and parietal networks; we conclude that manual tracking utilises such slow serial decision making pathways up to several times per second.

ABSTRACT

The human operator is described adequately by linear translation of sensory input to motor output. Motor output also always includes a non-linear remnant resulting from random sensorimotor noise from multiple sources, and non-linear input transformations, for example thresholds or refractory periods. Recent evidence showed that manual tracking incurs substantial, serial, refractoriness (insensitivity to sensory information of 350 and 550 ms for 1st and 2nd order systems respectively). Our two questions are: (i) What are the comparative merits of explaining the non-linear remnant using noise or non-linear transformations? (ii) Can non-linear transformations represent serial motor decision making within the sensorimotor feedback loop intrinsic to tracking? Twelve participants (instructed to act in three prescribed ways) manually controlled two systems (1st and 2nd order) subject to a periodic multi-sine disturbance. Joystick power was analysed using three models, continuous-linear-control (CC), continuous-linear-control with calculated noise spectrum (CCN), and intermittent control with aperiodic sampling triggered by prediction error thresholds (IC). Unlike the linear mechanism, the intermittent control mechanism explained the majority of total power (linear and remnant) (77-87% vs. 8-48%, IC vs. CC). Between conditions, IC used thresholds and distributions of open loop intervals consistent with, respectively, instructions and previous measured, model independent values; whereas CCN required changes in noise spectrum deviating from broadband, signal dependent noise. We conclude that manual tracking uses open loop predictive control with aperiodic sampling. Because aperiodic sampling is inherent to serial decision making within previously identified, specific frontal, striatal and parietal networks we suggest that these structures are intimately involved in visuo-manual tracking.

Learning From One's Mistakes: A Dual Role for the Rostromedial Tegmental Nucleus in the Encoding and Expression of Punished Reward Seeking

BACKGROUND

Psychiatric disorders such as addiction and mania are marked by persistent reward seeking despite highly negative or aversive outcomes, but the neural mechanisms underlying this aberrant decision making are unknown. The recently identified rostromedial tegmental nucleus (RMTg) encodes a wide variety of aversive stimuli and sends robust inhibitory projections to midbrain dopamine neurons, leading to the hypothesis that the RMTg provides a brake to reward signaling in response to aversive costs.

METHODS

To test the role of the RMTg in punished reward seeking, adult male Sprague Dawley rats were tested in several cost-benefit decision tasks after excitotoxic lesions of the RMTg or temporally specific optogenetic inhibition of RMTg efferents in the ventral tegmental area.

RESULTS

RMTg lesions drastically impaired the ability of foot shock to suppress operant responding for food. Optogenetic inhibition showed that this resistance to punishment was due in part to RMTg activity at the precise moment of shock delivery and was mediated by projections to the ventral tegmental area, which is consistent with an aversive "teaching signal" role for the RMTg during encoding of the aversive event. We observed a similar resistance to punishment when the RMTg was selectively inhibited immediately prior to the operant lever press, which is consistent with a second distinct role for the RMTg during action selection. These effects were not attributable to RMTg effects on learning rate, locomotion, shock sensitivity, or perseveration.

CONCLUSIONS

The RMTg has two strong and dissociable roles during both encoding and recall of aversive consequences of behavior.

Closed-Loop Modulation of the Pathological Disorders of the Basal Ganglia Network

A generalized predictive closed-loop control strategy to improve the basal ganglia activity patterns in Parkinson's disease (PD) is explored in this paper. Based on system identification, an input-output model is established to reveal the relationship between external stimulation and neuronal responses. The model contributes to the implementation of the generalized predictive control (GPC) algorithm that generates the optimal stimulation waveform to modulate the activities of neuronal nuclei. By analyzing the roles of two critical control parameters within the GPC law, optimal closed-loop control that has the capability of restoring the normal relay reliability of the thalamus with the least stimulation energy expenditure can be achieved. In comparison with open-loop deep brain stimulation and traditional static control schemes, the generalized predictive closed-loop control strategy can optimize the stimulation waveform without requiring any particular knowledge of the physiological properties of the system. This type of closed-loop control strategy generates an adaptive stimulation waveform with low energy expenditure with the potential to improve the treatments for PD.

Dopamine Selectively Modulates the Outcome of Learning Unnatural Action-Valence Associations

Learning the contingencies between stimulus, action, and outcomes is disrupted in disorders associated with altered dopamine (DA) function in the BG, such as Parkinson disease (PD). Although the role of DA in learning to act has been extensively investigated in PD, the role of DA in "learning to withhold" (or inhibit) action to influence outcomes is not as well understood. The current study investigated the role of DA in learning to act or to withhold action to receive rewarding, or avoid punishing outcomes, in patients with PD tested "off" and "on" dopaminergic medication (n = 19) versus healthy controls (n = 30). Participants performed a reward-based learning task that orthogonalized action and outcome valence (action-reward, inaction-reward, action-punishment, inaction-punishment). We tested whether DA would bias learning toward action, toward reward, or to particular action-outcome interactions. All participants demonstrated inherent learning biases preferring action with reward and inaction to avoid punishment, and this was unaffected by medication. Instead, DA produced a complex modulation of learning less natural action-outcome associations. "Off" DA medication, patients demonstrated impairments in learning to withhold action to gain reward, suggesting a difficulty to overcome a bias toward associating inaction with punishment avoidance. On DA medication, these patterns changed, and patients showed a reduced ability to learn to act to avoid punishment, indicating a bias toward action and reward. The current findings suggest that DA in PD has a complex influence on the formation of action-outcome associations, particularly those involving less natural linkages between action and outcome valence.

Corticostriatal circuitry

Corticostriatal connections play a central role in developing appropriate goal-directed behaviors, including the motivation and cognition to develop appropriate actions to obtain a specific outcome. The cortex projects to the striatum topographically. Thus, different regions of the striatum have been associated with these different functions: the ventral striatum with reward; the caudate nucleus with cognition; and the putamen with motor control. However, corticostriatal connections are more complex, and interactions between functional territories are extensive. These interactions occur in specific regions in which convergence of terminal fields from different functional cortical regions are found. This article provides an overview of the connections of the cortex to the striatum and their role in integrating information across reward, cognitive, and motor functions. Emphasis is placed on the interface between functional domains within the striatum.

The nature of working memory gating in Parkinson's disease: A multi-domain signal detection examination

Distractions are ubiquitous; our brains are inundated with task-irrelevant information. Thus, to remember successfully, one must actively maintain relevant information and prevent distraction from entering working memory. Researchers suggest the basal ganglia-prefrontal pathways are vital to this process by acting as a working memory gate. Using Parkinson's disease as a model of frontostriatal functioning and with signal detection analyses, the present study aims to better characterize the contribution of frontostriatal pathways of this gating process and to determine how it operates across multiple domains. To achieve this, Parkinson's disease patients and healthy controls completed verbal and spatial working memory tasks consisting of three conditions: low-load without distraction; low-load with distraction; and high-load without distraction. Patients were tested both ON and OFF dopaminergic medication, allowing for assessment of the contribution of dorsal and ventral frontostriatal pathways. The results demonstrate that when medication is withheld, Parkinson's patients have a response bias to answer "NO" across all conditions and domains, supporting our hypothesis that the basal ganglia-prefrontal pathways allow or prevent updates of working memory. Contrastingly, medication status affects d' in the distraction condition but not in the high- or low-load conditions. We attribute this to stimulus valuation processes that were impaired by dopaminergic medication overdosing the ventral pathway. These findings are both consistent with the hypothesis that the working memory gate filters spatial and verbal information before it enters into the working memory system, adding support for the gate being a domain-general mechanism of the central executive.