Computational models of the basal ganglia

Neural correlates of psychodynamic and non-psychodynamic therapies in different clinical populations through fMRI: A meta-analysis and systematic review

Background

The COVID-19 pandemic has exacerbated the ongoing crisis in psychiatric and psychological care, contributing to what we have identified as a new psychological and psychiatric pandemic. Psychotherapy is an effective method for easing the psychological suffering experienced also by the various impacts of COVID-19. This treatment can be examined from a neurological perspective, through the application of brain imaging techniques. Specifically, the meta-analysis of imaging studies can aid in expanding researchers' understanding of the many beneficial applications of psychotherapy.

Objectives

We examined the functional brain changes accompanying different mental disorders with functional Magnetic Resonance Imaging (fMRI), through a meta-analysis, and systematic review in order to better understand the general neural mechanism involved in psychotherapy and the potential neural difference between psychodynamic and non-psychodynamic approaches.

Data sources

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were employed for our systematic review and meta-analysis. We conducted a computer-based literature search, following the Population, Intervention, Comparison and Outcomes (PICO) approach, to retrieve all published articles in English regarding the above-described topics from PubMed (MEDLINE), Scopus, and Web of Science.

Study eligibility criteria participants and interventions

We combined terms related to psychotherapy and fMRI: ("psychotherapy" [All Fields] OR "psychotherapy" [MeSH Terms] OR "psychotherapy" [All Fields] OR "psychotherapies" [All Fields] OR "psychotherapy s" [All Fields]) AND ("magnetic resonance imaging" [MeSH Terms]) OR ("magnetic"[All Fields] AND "resonance"[All Fields] AND "imaging"[All Fields]) OR ("magnetic resonance imaging"[All Fields] OR "fmri"[All Fields]). We considered (1) whole brain fMRI studies; (2) studies in which participants have been involved in a clinical trial with psychotherapy sessions, with pre/post fMRI; (3) fMRI results presented in coordinate-based (x, y, and z) in MNI or Talairach space; (4) presence of neuropsychiatric patients. The exclusion criteria were: (1) systematic review or meta-analysis; (2) behavioral study; (3) single-case MRI or fMRI study; and (4) other imaging techniques (i.e., PET, SPECT) or EEG.

Results

After duplicates removal and assessment of the content of each published study, we included 38 sources. The map including all studies that assessed longitudinal differences in brain activity showed two homogeneous clusters in the left inferior frontal gyrus, and caudally involving the anterior insular cortex (p < 0.0001, corr.). Similarly, studies that assessed psychotherapy-related longitudinal changes using emotional or cognitive tasks (TASK map) showed a left-sided homogeneity in the anterior insula (p < 0.000) extending to Broca's area of the inferior frontal gyrus (p < 0.0001) and the superior frontal gyrus (p < 0.0001). Studies that applied psychodynamic psychotherapy showed Family-Wise Error (FWE) cluster-corrected (p < 0.05) homogeneity values in the right superior and inferior frontal gyri, with a small cluster in the putamen. No FWE-corrected homogeneity foci were observed for Mindful- based and cognitive behavioral therapy psychotherapy. In both pre- and post-therapy results, studies showed two bilateral clusters in the dorsal anterior insulae (p = 0.00001 and p = 0.00003, respectively) and involvement of the medial superior frontal gyrus (p = 0.0002).

Limitations

Subjective experiences, such as an individual's response to therapy, are intrinsically challenging to quantify as objective, factual realities. Brain changes observed both pre- and post-therapy could be related to other factors, not necessary to the specific treatment received. Therapeutic modalities and study designs are generally heterogeneous. Differences exist in sample characteristics, such as the specificity of the disorder and number and duration of sessions. Moreover, the sample size is relatively small, particularly due to the paucity of studies in this field and the little contribution of PDT.

Conclusions and implications of key findings

All psychological interventions seem to influence the brain from a functional point of view, showing their efficacy from a neurological perspective. Frontal, prefrontal regions, insular cortex, superior and inferior frontal gyrus, and putamen seem involved in these neural changes, with the psychodynamic more linked to the latter three regions.

A loop-based neural architecture for structured behavior encoding and decoding

We present a new type of artificial neural network that generalizes on anatomical and dynamical aspects of the mammal brain. Its main novelty lies in its topological structure which is built as an array of interacting elementary motifs shaped like loops. These loops come in various types and can implement functions such as gating, inhibitory or executive control, or encoding of task elements to name a few. Each loop features two sets of neurons and a control region, linked together by non-recurrent projections. The two neural sets do the bulk of the loop's computations while the control unit specifies the timing and the conditions under which the computations implemented by the loop are to be performed. By functionally linking many such loops together, a neural network is obtained that may perform complex cognitive computations. To demonstrate the potential offered by such a system, we present two neural network simulations. The first illustrates the structure and dynamics of a single loop implementing a simple gating mechanism. The second simulation shows how connecting four loops in series can produce neural activity patterns that are sufficient to pass a simplified delayed-response task. We also show that this network reproduces electrophysiological measurements gathered in various regions of the brain of monkeys performing similar tasks. We also demonstrate connections between this type of neural network and recurrent or long short-term memory network models, and suggest ways to generalize them for future artificial intelligence research.

Evidence for a task-dependent switch in subthalamo-nigral basal ganglia signaling

Basal ganglia (BG) can either facilitate or inhibit movement through excitatory and inhibitory pathways; however whether these opposing signals are dynamically regulated during healthy behavior is not known. Here, we present compelling neurophysiological evidence from three complimentary experiments in non-human primates, indicating task-specific changes in tonic BG pathway weightings during saccade behavior with different cognitive demands. First, simultaneous local field potential recording in the subthalamic nucleus (STN; BG input) and substantia nigra pars reticulata (SNr; BG output) reveals task-dependent shifts in subthalamo-nigral signals. Second, unilateral electrical stimulation of the STN, SNr, and caudate nucleus results in strikingly different saccade directionality and latency biases across the BG. Third, a simple artificial neural network representing canonical BG signaling pathways suggests that pathway weightings can be altered by cortico-BG input activation. Overall, inhibitory pathways (striato-pallidal-subthalamo-nigral) dominate during goal-driven behavior with instructed rewards, while facilitatory pathways (striato-nigral and subthalamo-pallidal-nigral) dominate during unconstrained (free reward) conditions.

Basal ganglia can both facilitate or inhibit movement through excitatory and inhibitory pathways; however whether these opposing signals are dynamically regulated during behavior is not known. Here the authors use multinucleus LFP recordings and electrical microstimulation in monkeys performing saccade based tasks to show task specific changes in the tonic weighting of these pathways.

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

The brain enables animals to behaviorally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviors are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and post-synaptic activity, receptor type, and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson's disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

New Models of Frontal-Subcortical Skeletomotor Circuit Pathology in Tardive Dyskinesia

Tardive dyskinesia (TD) is a hyperkinetic movement disorder that can occur as a side effect of treatment with antipsychotic medications. Because antipsychotics block the D2 family of dopamine receptors in the striatum, it has long been suspected this blockade contributes to the development of TD. Specifically, increased sensitivity of the dopamine receptors following chronic blockade has been thought to result in abnormal functioning of the frontal-subcortical (FSC) skeletomotor circuit and the symptoms of TD. However, this hypothesis remains unproven. In recent years, substantial research has focused on the basal ganglia and FSC circuits. This research has resulted in the development of the focused selection model of skeletomotor circuit function. This hypothesis provides a compelling model of neurocircuit abnormalities in TD. A greater understanding of the neuropathology of TD may lead to the development of better treatment and prevention strategies for this disorder. Furthermore, this information may contribute to a more complete understanding of normal skeletomotor circuit function and the role of circuit pathology in numerous neuropsychiatric conditions.

The basal ganglia, the ideal machinery for the cost-benefit analysis of action plans

Basal ganglia dysfunction causes profound movement disorders, often attributed to imbalance between direct and indirect pathway activity in the sensorimotor basal ganglia. In the classical view, the direct pathway facilitates movements, whereas the indirect pathway inhibits movements. However, the recent finding of co-activation of the two pathways during movement challenges this view. Reconciling the new finding with the body of evidence supporting the classical view, this perspective proposes that the direct pathway computes the expected benefits of motor plans entering the basal ganglia, while the indirect pathway computes their expected costs. Thus, basal ganglia output combining the two pathway signals in a subtraction manner weighs benefits against costs, and endorses the plan with the best prospective outcome via feedback projections to the cortex. The cost-benefit model, while retaining the antagonistic roles of the two pathways for movements, requires co-activation of the two pathways during movement as both benefit and cost are computed for every movement. The cost-benefit model, though simple, accounts for a number of confounding results, and generates new focus for future research with testable predictions.

Action selection performance of a reconfigurable basal ganglia inspired model with Hebbian-Bayesian Go-NoGo connectivity

Several studies have shown a strong involvement of the basal ganglia (BG) in action selection and dopamine dependent learning. The dopaminergic signal to striatum, the input stage of the BG, has been commonly described as coding a reward prediction error (RPE), i.e., the difference between the predicted and actual reward. The RPE has been hypothesized to be critical in the modulation of the synaptic plasticity in cortico-striatal synapses in the direct and indirect pathway. We developed an abstract computational model of the BG, with a dual pathway structure functionally corresponding to the direct and indirect pathways, and compared its behavior to biological data as well as other reinforcement learning models. The computations in our model are inspired by Bayesian inference, and the synaptic plasticity changes depend on a three factor Hebbian–Bayesian learning rule based on co-activation of pre- and post-synaptic units and on the value of the RPE. The model builds on a modified Actor-Critic architecture and implements the direct (Go) and the indirect (NoGo) pathway, as well as the reward prediction (RP) system, acting in a complementary fashion. We investigated the performance of the model system when different configurations of the Go, NoGo, and RP system were utilized, e.g., using only the Go, NoGo, or RP system, or combinations of those. Learning performance was investigated in several types of learning paradigms, such as learning-relearning, successive learning, stochastic learning, reversal learning and a two-choice task. The RPE and the activity of the model during learning were similar to monkey electrophysiological and behavioral data. Our results, however, show that there is not a unique best way to configure this BG model to handle well all the learning paradigms tested. We thus suggest that an agent might dynamically configure its action selection mode, possibly depending on task characteristics and also on how much time is available.

Conditional routing of information to the cortex: a model of the basal ganglia's role in cognitive coordination

The basal ganglia play a central role in cognition and are involved in such general functions as action selection and reinforcement learning. Here, we present a model exploring the hypothesis that the basal ganglia implement a conditional information-routing system. The system directs the transmission of cortical signals between pairs of regions by manipulating separately the selection of sources and destinations of information transfers. We suggest that such a mechanism provides an account for several cognitive functions of the basal ganglia. The model also incorporates a possible mechanism by which subsequent transfers of information control the release of dopamine. This signal is used to produce novel stimulus-response associations by internalizing transferred cortical representations in the striatum. We discuss how the model is related to production systems and cognitive architectures. A series of simulations is presented to illustrate how the model can perform simple stimulus-response tasks, develop automatic behaviors, and provide an account of impairments in Parkinson's and Huntington's diseases.

Dealing with the devil in the detail - some thoughts about the next model of the basal ganglia

Any simplified description of a brain system that survives 20 years is bound to have some problems. Twenty years of research should have added details that were not known when the now classical description of the basal ganglia circuitry was new. Some examples of such new information about the parts of the original model include: All of these details make it difficult to cling to the model as it was presented, but which of the new information will support a new synthesis? Do we have a coherent replacement for those powerfully heuristic ideas put forward by Albin Young and Penny in 1989? Perhaps in the quantitative anatomical detail there is a plan that may support a new theory. If all of the cortex is not accessible to every striatal cell. If dopamine influences many spines rather than the one it contacts. If we stop looking for a mechanical control system for movement in the basal ganglia and begin to investigate the predictive properties of 'reinforcement learning'. Then maybe a new functional model will emerge that is less tied to anatomical and physiological simplification, but perhaps will still allow us to engineer therapeutic strategies to help patients with movement disorders.

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain.

The emergence of saliency and novelty responses from Reinforcement Learning principles

Recent attempts to map reward-based learning models, like Reinforcement Learning [Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An introduction. Cambridge, MA: MIT Press], to the brain are based on the observation that phasic increases and decreases in the spiking of dopamine-releasing neurons signal differences between predicted and received reward [Gillies, A., & Arbuthnott, G. (2000). Computational models of the basal ganglia. Movement Disorders, 15(5), 762-770; Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27]. However, this reward-prediction error is only one of several signals communicated by that phasic activity; another involves an increase in dopaminergic spiking, reflecting the appearance of salient but unpredicted non-reward stimuli [Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4-6), 495-506; Horvitz, J. C. (2000). Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience, 96(4), 651-656; Redgrave, P., & Gurney, K. (2006). The short-latency dopamine signal: A role in discovering novel actions? Nature Reviews Neuroscience, 7(12), 967-975], especially when an organism subsequently orients towards the stimulus [Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27]. To explain these findings, Kakade and Dayan [Kakade, S., & Dayan, P. (2002). Dopamine: Generalization and bonuses. Neural Networks, 15(4-6), 549-559.] and others have posited that novel, unexpected stimuli are intrinsically rewarding. The simulation reported in this article demonstrates that this assumption is not necessary because the effect it is intended to capture emerges from the reward-prediction learning mechanisms of Reinforcement Learning. Thus, Reinforcement Learning principles can be used to understand not just reward-related activity of the dopaminergic neurons of the basal ganglia, but also some of their apparently non-reward-related activity.

Where neuroscience and dynamic system theory meet autonomous robotics: a contracting basal ganglia model for action selection

Action selection, the problem of choosing what to do next, is central to any autonomous agent architecture. We use here a multi-disciplinary approach at the convergence of neuroscience, dynamical system theory and autonomous robotics, in order to propose an efficient action selection mechanism based on a new model of the basal ganglia. We first describe new developments of contraction theory regarding locally projected dynamical systems. We exploit these results to design a stable computational model of the cortico-baso-thalamo-cortical loops. Based on recent anatomical data, we include usually neglected neural projections, which participate in performing accurate selection. Finally, the efficiency of this model as an autonomous robot action selection mechanism is assessed in a standard survival task. The model exhibits valuable dithering avoidance and energy-saving properties, when compared with a simple if-then-else decision rule.

Neuroinformatics and modeling of the basal ganglia: bridging pharmacology and physiology

The subthalamic nucleus (STN) is the primary target for the chronic deep brain stimulation treatment of Parkinson's disease. STN neurons exhibit a variety of characteristic properties that may play a key role in the overall population response to deep brain stimulation. Neuroinformatics techniques, in particular computational modeling, provide a method of bringing together pharmacological phenomena, such as the loss of dopamine, with electrophysiological characteristics. Developing accurate models of STN neurons plays an important part in the process of uncovering the link between the changes in STN pharmacology, physiology and synaptic input that occurs with Parkinson's disease and the effectiveness of treatments targeting the STN. We review a general procedure for developing computational models and present a model of STN neurons that reveals important membrane channel interactions. In particular, changes in these channel interactions under parkinsonian conditions may underlie changes in characteristic physiology, critical in determining the mechanisms of deep brain stimulation.

Modeling parkinsonian circuitry and the DBS electrode. I. Biophysical background and software

Deep brain stimulation (DBS) of the subthalamic nucleus (STN) for Parkinson's disease (PD) has become routine over the past decade, utilizing microelectrode recordings to ensure accurate placement of the stimulating electrodes. The clinical benefits of STN DBS for PD are well documented, but the mechanisms by which DBS achieves these results remain elusive. We have created a closed-form mathematical function of the potential field generated by a typical 4-contact DBS electrode and inserted this function into a computational model designed to simulate individual neurons and neural circuitry of significant portions of the basal ganglia. We present the mathematical function representing the potential field itself and the basis for the neural circuitry modeling in this paper.

A scale-free systems theory of motivation and addiction

Scale-free organizations, characterized by uneven distributions of linkages between nodal elements, describe the structure and function of many life-based complex systems developing under evolutionary pressures. We explore motivated behavior as a scale-free map toward a comprehensive translational theory of addiction. Motivational and behavioral repertoires are reframed as link and nodal element sets, respectively, comprising a scale-free structure. These sets are generated by semi-independent information-processing streams within cortical-striatal circuits that cooperatively provide decision-making and sequential processing functions necessary for traversing maps of motivational links connecting behavioral nodes. Dopamine modulation of cortical-striatal plasticity serves a central-hierarchical mechanism for survival-adaptive sculpting and development of motivational-behavioral repertoires by guiding a scale-free design. Drug-induced dopamine activity promotes drug taking as a highly connected behavioral hub at the expense of natural-adaptive motivational links and behavioral nodes. Conceptualizing addiction as pathological alteration of scale-free motivational-behavioral repertoires unifies neurobiological, neurocomputational and behavioral research while addressing addiction vulnerability in adolescence and psychiatric illness. This model may inform integrative research in defining more effective prevention and treatment strategies for addiction.

A physiologically plausible model of action selection and oscillatory activity in the basal ganglia

The basal ganglia (BG) have long been implicated in both motor function and dysfunction. It has been proposed that the BG form a centralized action selection circuit, resolving conflict between multiple neural systems competing for access to the final common motor pathway. We present a new spiking neuron model of the BG circuitry to test this proposal, incorporating all major features and many physiologically plausible details. We include the following: effects of dopamine in the subthalamic nucleus (STN) and globus pallidus (GP), transmission delays between neurons, and specific distributions of synaptic inputs over dendrites. All main parameters were derived from experimental studies. We find that the BG circuitry supports motor program selection and switching, which deteriorates under dopamine-depleted and dopamine-excessive conditions in a manner consistent with some pathologies associated with those dopamine states. We also validated the model against data describing oscillatory properties of BG. We find that the same model displayed detailed features of both gamma-band (30-80 Hz) and slow (approximately 1 Hz) oscillatory phenomena reported by Brown et al. (2002) and Magill et al. (2001), respectively. Only the parameters required to mimic experimental conditions (e.g., anesthetic) or manipulations (e.g., lesions) were changed. From the results, we derive the following novel predictions about the STN-GP feedback loop: (1) the loop is functionally decoupled by tonic dopamine under normal conditions and recoupled by dopamine depletion; (2) the loop does not show pacemaking activity under normal conditions in vivo (but does after combined dopamine depletion and cortical lesion); (3) the loop has a resonant frequency in the gamma-band.

Computational models simulating electrophysiological activity in the basal ganglia

Modeling of the basal ganglia has played a substantial role in gaining insight into the mechanisms involved in the computational processes performed by this elusive group of nuclei. Models of the basal ganglia have undergone revolutionary changes over the last twenty years due to the rapid accumulation of neuroscientific data. In this chapter, we present distinct modeling approaches that can be used to enhance our understanding of the functional dynamics of information processing within the basal ganglia, and their interactions with the rest of the brain. Specific examples of recently developed models dealing with the analysis of computational processing issues at different structural levels of the basal ganglia are discussed.

The role of intact frontostriatal circuits in error processing

The basal ganglia have been suggested to play a key role in performance monitoring and resulting behavioral adjustments. It is assumed that the integration of prefrontal and motor cortico-striato-thalamo-cortical circuits provides contextual information to the motor anterior cingulate cortex regions to enable their function in performance monitoring. So far, direct evidence is missing, however. We addressed the involvement of frontostriatal circuits in performance monitoring by collecting event-related brain potentials (ERPs) and behavioral data in nine patients with focal basal ganglia lesions and seven patients with lateral prefrontal cortex lesions while they performed a flanker task. In both patient groups, the amplitude of the error-related negativity was reduced, diminishing the difference to the ERPs on correct responses. Despite these electrophysiological abnormalities, most of the patients were able to correct errors. Only in lateral prefrontal cortex patients whose lesions extended into the frontal white matter, disrupting the connections to the motor anterior cingulate cortex and the striatum, were error corrections severely impaired. In sum, the fronto-striato-thalamo-cortical circuits seem necessary for the generation of error-related negativity, even when brain plasticity has resulted in behavioral compensation of the damage. Thus, error-related ERPs in patients provide a sensitive measure of the integrity of the performance monitoring network.

From brainstem to cortex: computational models of saccade generation circuitry

The brain circuitry of saccadic eye movements, from brainstem to cortex, has been extensively studied during the last 30 years. The wealth of data gathered allowed the conception of numerous computational models. These models proposed descriptions of the putative mechanisms generating this data, and, in turn, made predictions and helped to plan new experiments. In this article, we review the computational models of the five main brain regions involved in saccade generation: reticular formation saccadic burst generators, superior colliculus, cerebellum, basal ganglia and premotor cortical areas. We present the various topics these models are concerned with: location of the feedback loop, multimodal saccades, long-term adaptation, on the fly trajectory correction, strategy and metrics selection, short-term spatial memory, transformations between retinocentric and craniocentric reference frames, sequence learning, to name the principle ones. Our objective is to provide a global view of the whole system. Indeed, narrowing too much the modelled areas while trying to explain too much data is a recurrent problem that should be avoided. Moreover, beyond the multiple research topics remaining to be solved locally, questions regarding the operation of the whole structure can now be addressed by building on the existing models.

Excitatory GABAergic Effects in Striatal Projection Neurons

The ability of synaptically released GABA to facilitate action potential generation in striatal projection neurons was studied in brain slices using current-clamp, gramicidin-perforated whole cell recordings. Evoked GABAergic postsynaptic potentials (PSPs) were pharmacologically isolated with ionotropic glutamate receptor antagonists. Subthreshold depolarizing current injections were paired with GABAergic PSPs at different intervals. GABAergic PSPs were able to convert current injection-induced depolarizations from subthreshold to suprathreshold, but only when they preceded the current injection by an appropriate interval; accordingly, action potentials were observed 4–140 ms after the onset of the GABAergic PSP, and their likelihood was maximal after 50–60 ms. The GABAergic excitatory effects were fully blocked by the GABAA receptor antagonist bicuculline. Appropriately timed GABA PSPs decreased the time taken by current injections to depolarize projection neurons, causing an apparent reduction in the spike threshold. In control solution, the ability of evoked PSPs (comprising both glutamatergic and GABAergic components) to reach spike threshold was often impaired by bicuculline. We conclude that GABAergic PSPs can exert excitatory effects on projection neurons and that this ability crucially depends on the timing between the GABAergic event and a concomitant depolarizing input.

A robot model of the basal ganglia: Behavior and intrinsic processing

The existence of multiple parallel loops connecting sensorimotor systems to the basal ganglia has given rise to proposals that these nuclei serve as a selection mechanism resolving competitions between the alternative actions available in a given context. A strong test of this hypothesis is to require a computational model of the basal ganglia to generate integrated selection sequences in an autonomous agent, we therefore describe a robot architecture into which such a model is embedded, and require it to control action selection in a robotic task inspired by animal observations. Our results demonstrate effective action selection by the embedded model under a wide range of sensory and motivational conditions. When confronted with multiple, high salience alternatives, the robot also exhibits forms of behavioral disintegration that show similarities to animal behavior in conflict situations. The model is shown to cast light on recent neurobiological findings concerning behavioral switching and sequencing.

Models of the subthalamic nucleus. The importance of intranuclear connectivity

A coherent set of models is presented that provide novel and testable predictions about the functional role of the subthalamic nucleus (STN) in the basal ganglia. The STN is emerging as an important target for novel therapeutic strategies for the alleviation of Parkinsonian type symptoms [Lancet 345 (1995) 91; Science 249 (1990) 1436]. Computational and mathematical models based on the properties of the STN and its interactions are reviewed. These models focus on core anatomical and physiological data that span many levels. By assessing models of anatomy, dynamic network models, and a detailed model of a recent pharmacological experiment, we can expose the primary modes of STN function and highlight their underlying properties. We show that the presence of functional interactions between STN projection neurons is critical in defining its behaviour and how it interacts with other basal ganglia nuclei. Pulses or switch-like activity patterns emerge in the models as a consequence of these local interactions. Furthermore, the models demonstrate that this behaviour can break down under abnormal conditions resulting in low frequency bursting oscillations. Such oscillations may play a role in symptoms of Parkinson's disease.

Activation of conflicting responses in Parkinson's disease: evidence for degrading and facilitating effects on response time

Response selection often occurs in a context of competition among conflicting responses. According to recent models, the basal ganglia may play an integral role in resolving this competition by focusing the selection and inhibition of responses. We hypothesized that basal ganglia dysfunction produced by Parkinson's disease (PD) disrupts selection among conflicting responses. Using a version of the Eriksen flanker task, we tested the specific prediction that individuals with PD would experience greater response interference when distractors in the visual field activate a response that conflicts with the target response. In addition, we investigated whether greater response interference induced by these distractors could actually reduce normal response time costs in PD when the task required production of the response opposite the target. Compared to 16 healthy controls (HC), 16 individuals with PD showed an exacerbated slowing when target and distracting stimuli corresponded to conflicting responses. No group differences occurred when targets and distractors corresponded to the same response. Furthermore, the slowing induced by the distractors was reduced in both groups, but more so in PD, when execution of a response opposite the target response (i.e. incompatible response) was required. Moreover, among individuals with PD, the magnitude of the interference produced by the distractors was related to clinical ratings of bradykinesia. These findings are consistent with the hypothesis that basal ganglia dysfunction due to Parkinson's disease disrupts processes that resolve response conflict.

Computational models of the basal ganglia: from robots to membranes

With the rapid accumulation of neuroscientific data comes a pressing need to develop models that can explain the computational processes performed by the basal ganglia. Relevant biological information spans a range of structural levels, from the activity of neuronal membranes to the role of the basal ganglia in overt behavioural control. This viewpoint presents a framework for understanding the aims, limitations and methods for testing of computational models across all structural levels. We identify distinct modelling strategies that can deliver important and complementary insights into the nature of problems the basal ganglia have evolved to solve, and describe methods that are used to solve them.

Identification of prosodic attitudes by a temporal recurrent network

Human speakers modulate the fundamental frequency (F0) of their utterances in order to express different 'prosodic' attitudes such as surprise or curiosity. How are these prosodic attitudes then decoded? The current research addresses the issue of how the temporal structure of F0 can be used in order to discriminate between prosodic attitudes in natural language using a temporal recurrent neural network (TRN) that was initially developed to simulate the neurophysiology of the primate frontostriatal system. In the TRN, a recurrent network of leaky integrator neurons encodes a continuous trajectory of internal states that characterizes the input sequence. The input to the model is a population coding of the continuous, time-varying values of the fundamental frequency (F0) of natural language sentences. We expose the model to an experiment based on one in which human subjects were required to discriminate between different prosodic attitudes (surprise, exclamation, question, etc.). After training, the model discriminates between six prosodic attitudes in new sentences at 82.52% correct, compared to 72.8% correct for human subjects. These results reveal (1) that F0 provides relevant information for prosodic attitude discrimination, and (2) that the TRN demonstrates a categorical sensitivity to this information that can be used for classifying new sentences.

Pathophysiology of obsessive–compulsive disorder

Obsessive-compulsive disorder (OCD) is characterized by repetitive intrusive thoughts and compulsive time-consuming behaviors classified into three to five distinct symptom dimensions including: (1) aggressive/somatic obsessions with checking compulsions; (2) contamination concerns with washing compulsions; (3) symmetry obsessions with counting/ordering compulsions; (4) hoarding obsessions with collecting compulsions; and (5) sexual/religious concerns. Phenomenologically, OCD could be thought of as the irruption of internal signals centered on the erroneous perception that "something is wrong" in a specific situation. This generates severe anxiety, leading to recurrent behaviors aimed at reducing the emotional tension. In this paper, we examine how the abnormalities in brain activity reported in OCD can be interpreted in the light of physiology after consideration of various approaches (phenomenology, neuropsychology, neuroimmunology and neuroimagery) that contribute to proposing the central role of several cortical and subcortical regions, especially the orbitofrontal cortex (OFC), the anterior cingulate cortex (ACC), the dorsolateral prefrontal cortex (DLPC), the head of the caudate nucleus and the thalamus. The OFC is involved in the significance attributed to the consequences of action, thereby subserving decision-making, whereas the ACC is particularly activated in situations in which there are conflicting options and a high likelihood of making an error. The DLPC plays a critical part in the cognitive processing of relevant information. This cortical information is then integrated by the caudate nucleus, which controls behavioral programs. A dysfunction of these networks at one or several stages will result in the emergence and maintenance of repetitive thoughts and characteristic OCD behavior.

Information processing, dimensionality reduction and reinforcement learning in the basal ganglia

Modeling of the basal ganglia has played a major role in our understanding of this elusive group of nuclei. Models of the basal ganglia have undergone evolutionary and revolutionary changes over the last 20 years, as new research in the fields of anatomy, physiology and biochemistry of these nuclei has yielded new information. Early models dealt with a single pathway through the nuclei and focused on the nature of the processing performed within it, convergence of information versus parallel processing of information. Later, the Albin-DeLong "box-and-arrow" model characterized the inter-nuclei interaction as multiple pathways while maintaining a simplistic scalar representation of the nuclei themselves. This model made a breakthrough by providing key insights into the behavior of these nuclei in hypo- and hyper-kinetic movement disorders. The next generation of models elaborated the intra-nuclei interactions and focused on the role of the basal ganglia in action selection and sequence generation which form the most current consensus regarding basal ganglia function in both normal and pathological conditions. However, new findings challenge these models and point to a different neural network approach to information processing in the basal ganglia. Here, we take an in-depth look at the reinforcement driven dimensionality reduction (RDDR) model which postulates that the basal ganglia compress cortical information according to a reinforcement signal using optimal extraction methods. The model provides new insights and experimental predictions on the computational capacity of the basal ganglia and their role in health and disease.

A BASAL GANGLIA INSPIRED MODEL OF ACTION SELECTION EVALUATED IN A ROBOTIC SURVIVAL TASK

The basal ganglia system has been proposed as a possible neural substrate for action selection in the vertebrate brain. We describe a robotic implementation of a model of the basal ganglia and demonstrate the capacity of this system to generate adaptive switching between several acts when embedded in a robot that has to "survive" in a laboratory environment. A comparison between this brain-inspired selection mechanism and classical "winner-takes-all" selection highlights some adaptive properties specific to the model, such as avoidance of dithering and energy-saving. These properties derive, in part, from the capacity of simulated basal ganglia-thalamo-cortical loops to generate appropriate "behavioral persistence".

Rana computatrix to human language: towards a computational neuroethology of language evolution

Walter's Machina speculatrix inspired the name Rana computatrix for a family of models of visuomotor coordination in the frog, which contributed to the development of computational neuroethology. We offer here an 'evolutionary' perspective on models in the same tradition for rat, monkey and human. For rat, we show how the frog-like taxon affordance model provides a basis for the spatial navigation mechanisms that involve the hippocampus and other brain regions. For monkey, we recall two models of neural mechanisms for visuomotor coordination. The first, for saccades, shows how interactions between the parietal and frontal cortex augment superior colliculus seen as the homologue of frog tectum. The second, for grasping, continues the theme of parieto-frontal interactions, linking parietal affordances to motor schemas in premotor cortex. It further emphasizes the mirror system for grasping, in which neurons are active both when the monkey executes a specific grasp and when it observes a similar grasp executed by others. The model of human-brain mechanisms is based on the mirror-system hypothesis of the evolution of the language-ready brain, which sees the human Broca's area as an evolved extension of the mirror system for grasping.

How is firing activity of substantia nigra cells regulated? Relevance of pattern-code in the basal ganglia

The current model of the basal ganglia (BG) assumes that neurons use a firing rate renewal code for movement computing under normal and pathological conditions. Here, we report nonrenewal firing (neuronal firing is influenced by its own previous activity) in cells of the anesthetized rat's substantia nigra (SN). Both compensatory (short interspike intervals (ISIs) are followed by long ISIs and vice versa) and persistent (short and long ISIs cluster for long time periods) nonrenewal activity was found in 52.6% and 33.8% of SN cells, respectively. A compensatory pattern was found in 77.7% of DA cells, but in only 9.8% of GABA-cells. Conversely, a persistent pattern was observed in 74.6% of GABAergic cells and in only 9.9% of DA cells. These findings indicate two types of nonrenewal firing pattern codes specifically present in SN dopaminergic and GABAergic neurons. Disruption of these patterns may play a role in the pathophysiology of basal ganglia disorders such as Parkinson's disease and dyskinesias.

Firing regulation in dopaminergic cells: effect of the partial degeneration of nigrostriatal system in surviving neurons

Two mechanisms for firing rate regulation were identified in dopaminergic nigrostriatal cells (DA cells), one of a renewal nature which prevents short and long interspike intervals (ISIs) and the other of a no-renewal nature which compensates long ISIs with short ISIs and vice versa. Renewal regulation was found in 96% of DA cells and less frequently in nigrocollicular (63%), nigrothalamic (61%) and nigropeduncular (50%) nigral GABA cells. No-renewal regulation was found in 77% of DA cells, and was only observed in 8% of GABA cells. Thus, most DA cells showed both regulatory mechanisms, which justifies the low variability in their firing rate and the low oscillation of extracellular striatal dopamine previously reported. DA cells surviving a partial degeneration of the nigrostriatal system did not show alterations in their firing rate and burst firing but presented a marked disturbance for no-renewal regulation. Under these conditions, small fluctuations in firing rate are not compensated for in time, which could be one of the factors responsible for the motor fluctuations often observed in advanced Parkinson's disease.

Simulating closed- and open-loop voluntary movement: a nonlinear control-systems approach

In many recent human motor control models, including feedback-error learning and adaptive model theory (AMT), feedback control is used to correct errors while an inverse model is simultaneously tuned to provide accurate feedforward control. This popular and appealing hypothesis, based on a combination of psychophysical observations and engineering considerations, predicts that once the tuning of the inverse model is complete the role of feedback control is limited to the correction of disturbances. This hypothesis was tested by looking at the open-loop behavior of the human motor system during adaptation. An experiment was carried out involving 20 normal adult subjects who learned a novel visuomotor relationship on a pursuit tracking task with a steering wheel for input. During learning, the response cursor was periodically blanked, removing all feedback about the external system (i.e., about the relationship between hand motion and response cursor motion). Open-loop behavior was not consistent with a progressive transfer from closed- to open-loop control. Our recently developed computational model of the brain--a novel nonlinear implementation of AMT--was able to reproduce the observed closed- and open-loop results. In contrast, other control-systems models exhibited only minimal feedback control following adaptation, leading to incorrect open-loop behavior. This is because our model continues to use feedback to control slow movements after adaptation is complete. This behavior enhances the internal stability of the inverse model. In summary, our computational model is currently the only motor control model able to accurately simulate the closed- and open-loop characteristics of the experimental response trajectories.

Learning and memory functions of the Basal Ganglia

Although the mammalian basal ganglia have long been implicated in motor behavior, it is generally recognized that the behavioral functions of this subcortical group of structures are not exclusively motoric in nature. Extensive evidence now indicates a role for the basal ganglia, in particular the dorsal striatum, in learning and memory. One prominent hypothesis is that this brain region mediates a form of learning in which stimulus-response (S-R) associations or habits are incrementally acquired. Support for this hypothesis is provided by numerous neurobehavioral studies in different mammalian species, including rats, monkeys, and humans. In rats and monkeys, localized brain lesion and pharmacological approaches have been used to examine the role of the basal ganglia in S-R learning. In humans, study of patients with neurodegenerative diseases that compromise the basal ganglia, as well as research using brain neuroimaging techniques, also provide evidence of a role for the basal ganglia in habit learning. Several of these studies have dissociated the role of the basal ganglia in S-R learning from those of a cognitive or declarative medial temporal lobe memory system that includes the hippocampus as a primary component. Evidence suggests that during learning, basal ganglia and medial temporal lobe memory systems are activated simultaneously and that in some learning situations competitive interference exists between these two systems.

Inhibitory interactions between spiny projection neurons in the rat striatum

The spiny projection neurons are by far the most numerous type of striatal neuron. In addition to being the principal projection neurons of the striatum, the spiny projection neurons also have an extensive network of local axon collaterals by which they make synaptic connections with other striatal projection neurons. However, up to now there has been no direct physiological evidence for functional inhibitory interactions between spiny projection neurons. Here we present new evidence that striatal projection neurons are interconnected by functional inhibitory synapses. To examine the physiological properties of unitary inhibitory postsynaptic potentials (IPSPs), dual intracellular recordings were made from pairs of spiny projection neurons in brain slices of adult rat striatum. Synaptic interactions were found in 9 of 45 pairs of neurons using averages of 200 traces that were triggered by a single presynaptic action potential. In all cases, synaptic interactions were unidirectional, and no bidirectional interactions were detected. Unitary IPSPs evoked by a single presynaptic action potential had a peak amplitude ranging from 157 to 319 microV in different connections (mean: 277 +/- 46 microV, n = 9). The percentage of failures of single action potentials to evoke a unitary IPSP was estimated and ranged from 9 to 63% (mean: 38 +/- 14%, n = 9). Unitary IPSPs were reversibly blocked by bicuculline (n = 4) and had a reversal potential of -62.4 +/- 0.7 mV (n = 5), consistent with GABA-mediated inhibition. The findings of the present study correlate very well with anatomical evidence for local synaptic connectivity between spiny projection neurons and suggest that lateral inhibition plays a significant role in the information processing operations of the striatum.

Actor-critic models of the basal ganglia: new anatomical and computational perspectives

A large number of computational models of information processing in the basal ganglia have been developed in recent years. Prominent in these are actor-critic models of basal ganglia functioning, which build on the strong resemblance between dopamine neuron activity and the temporal difference prediction error signal in the critic, and between dopamine-dependent long-term synaptic plasticity in the striatum and learning guided by a prediction error signal in the actor. We selectively review several actor-critic models of the basal ganglia with an emphasis on two important aspects: the way in which models of the critic reproduce the temporal dynamics of dopamine firing, and the extent to which models of the actor take into account known basal ganglia anatomy and physiology. To complement the efforts to relate basal ganglia mechanisms to reinforcement learning (RL), we introduce an alternative approach to modeling a critic network, which uses Evolutionary Computation techniques to 'evolve' an optimal RL mechanism, and relate the evolved mechanism to the basic model of the critic. We conclude our discussion of models of the critic by a critical discussion of the anatomical plausibility of implementations of a critic in basal ganglia circuitry, and conclude that such implementations build on assumptions that are inconsistent with the known anatomy of the basal ganglia. We return to the actor component of the actor-critic model, which is usually modeled at the striatal level with very little detail. We describe an alternative model of the basal ganglia which takes into account several important, and previously neglected, anatomical and physiological characteristics of basal ganglia-thalamocortical connectivity and suggests that the basal ganglia performs reinforcement-biased dimensionality reduction of cortical inputs. We further suggest that since such selective encoding may bias the representation at the level of the frontal cortex towards the selection of rewarded plans and actions, the reinforcement-driven dimensionality reduction framework may serve as a basis for basal ganglia actor models. We conclude with a short discussion of the dual role of the dopamine signal in RL and in behavioral switching.

Pathophysiology of Parkinson's disease: from clinical neurology to basic neuroscience and back

Parkinson's disease (PD) is characterized by motor and nonmotor (cognitive and limbic) deficits. The motor signs of PD include hypokinetic signs such as akinesia/bradykinesia, rigidity and loss of normal postural reflexes, and hyperkinetic signs such as tremor. Dopamine depletion in the striatum is the hallmark of PD and of its animal models, still the pathophysiology of the parkinsonian symptoms and especially of parkinsonian tremor are under debate. The most extreme hypotheses argue about peripheral versus central nervous system origin, intrinsic cellular oscillator versus network oscillators, and basal ganglia-based pathophysiology versus cerebellar-thalamic based pathophysiology. Recent studies support the view that parkinsonian symptoms are most likely due to abnormal synchronous oscillating neuronal activity within the basal ganglia. Peripheral factors do only play a minor role for the generation, maintenance, and modulation of PD tremor and other signs. The most likely candidates producing these neuronal oscillations are the weakly coupled neural networks of the basal ganglia-thalamo-cortical loops. However, the present evidence supports the view that the basal ganglia loops are influenced by other neuronal structures and systems and that the tuning of these loops by cerebello-thalamic mechanisms and by other modulator neurotransmitter systems entrain the abnormal synchronized oscillations. Neurosurgical procedures, such as lesions or high-frequency stimulation of different parts of the loop, might resume the normal unsynchronized activity of the basal ganglia circuitry, and, therefore, ameliorate the clinical symptoms of Parkinson's disease.

Stepping out of the box: information processing in the neural networks of the basal ganglia

The Albin-DeLong 'box and arrow' model has long been the accepted standard model for the basal ganglia network. However, advances in physiological and anatomical research have enabled a more detailed neural network approach. Recent computational models hold that the basal ganglia use reinforcement signals and local competitive learning rules to reduce the dimensionality of sparse cortical information. These models predict a steady-state situation with diminished efficacy of lateral inhibition and low synchronization. In this framework, Parkinson's disease can be characterized as a persistent state of negative reinforcement, inefficient dimensionality reduction, and abnormally synchronized basal ganglia activity.