The basal ganglia: a vertebrate solution to the selection problem?

The subthalamic nucleus contributes causally to perceptual decision-making in monkeys

The subthalamic nucleus (STN) plays critical roles in the motor and cognitive function of the basal ganglia (BG), but the exact nature of these roles is not fully understood, especially in the context of decision-making based on uncertain evidence. Guided by theoretical predictions of specific STN contributions, we used single-unit recording and electrical microstimulation in the STN of healthy monkeys to assess its causal, computational roles in visual-saccadic decisions based on noisy evidence. The recordings identified subpopulations of STN neurons with distinct task-related activity patterns that related to different theoretically predicted functions. Microstimulation caused changes in behavioral choices and response times that reflected multiple contributions to an "accumulate-to-bound"-like decision process, including modulation of decision bounds and evidence accumulation, and to non-perceptual processes. These results provide new insights into the multiple ways that the STN can support higher brain function.

Bridging flexible goal-directed cognition and consciousness: The Goal-Aligning Representation Internal Manipulation theory

Goal-directed manipulation of internal representations is a key element of human flexible behaviour, while consciousness is commonly associated with higher-order cognition and human flexibility. Current perspectives have only partially linked these processes, thus preventing a clear understanding of how they jointly generate flexible cognition and behaviour. Moreover, these limitations prevent an effective exploitation of this knowledge for technological scopes. We propose a new theoretical perspective that extends our 'three-component theory of flexible cognition' toward higher-order cognition and consciousness, based on the systematic integration of key concepts from Cognitive Neuroscience and AI/Robotics. The theory proposes that the function of conscious processes is to support the alignment of representations with multi-level goals. This higher alignment leads to more flexible and effective behaviours. We analyse here our previous model of goal-directed flexible cognition (validated with more than 20 human populations) as a starting GARIM-inspired model. By bridging the main theories of consciousness and goal-directed behaviour, the theory has relevant implications for scientific and technological fields. In particular, it contributes to developing new experimental tasks and interpreting clinical evidence. Finally, it indicates directions for improving machine learning and robotics systems and for informing real-world applications (e.g., in digital-twin healthcare and roboethics).

Rethinking Movement Disorders

At present, clinical practice and research in movement disorders (MDs) focus on the "normalization" of altered movements. In this review, rather than concentrating on problems and burdens people with MDs undoubtedly have, we highlight their hidden potentials. Starting with current definitions of Parkinson's disease (PD), dystonia, chorea, and tics, we outline that solely conceiving these phenomena as signs of dysfunction falls short of their complex nature comprising both problems and potentials. Such potentials can be traced and understood in light of well-established cognitive neuroscience frameworks, particularly ideomotor principles, and their influential modern derivatives. Using these frameworks, the wealth of data on altered perception-action integration in the different MDs can be explained and systematized using the mechanism-oriented concept of perception-action binding. According to this concept, MDs can be understood as phenomena requiring and fostering flexible modifications of perception-action associations. Consequently, although conceived as being caught in a (trough) state of deficits, given their high flexibility, people with MDs also have high potential to switch to (adaptive) peak activity that can be conceptualized as hidden potentials. Currently, clinical practice and research in MDs are concerned with deficits and thus the "deep and wide troughs," whereas "scattered narrow peaks" reflecting hidden potentials are neglected. To better delineate and utilize the latter to alleviate the burden of affected people, and destigmatize their conditions, we suggest some measures, including computational modeling combined with neurophysiological methods and tailored treatment. © 2024 International Parkinson and Movement Disorder Society.

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.

Simulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia

The vertebrate basal ganglia play an important role in action selection-the resolution of conflicts between alternative motor programs. The effective operation of basal ganglia circuitry is also known to rely on appropriate levels of the neurotransmitter dopamine. We investigated reducing or increasing the tonic level of simulated dopamine in a prior model of the basal ganglia integrated into a robot control architecture engaged in a foraging task inspired by animal behaviour. The main findings were that progressive reductions in the levels of simulated dopamine caused slowed behaviour and, at low levels, an inability to initiate movement. These states were partially relieved by increased salience levels (stronger sensory/motivational input). Conversely, increased simulated dopamine caused distortion of the robot's motor acts through partially expressed motor activity relating to losing actions. This could also lead to an increased frequency of behaviour switching. Levels of simulated dopamine that were either significantly lower or higher than baseline could cause a loss of behavioural integration, sometimes leaving the robot in a 'behavioral trap'. That some analogous traits are observed in animals and humans affected by dopamine dysregulation suggests that robotic models could prove useful in understanding the role of dopamine neurotransmission in basal ganglia function and dysfunction.

Social influence and external feedback control in humans

This article aims to unravel the dynamics of social influence by examining the processes that occur when one person is the target of another's influence. We hypothesized that these processes are part of a feedback loop system in an individual. This loop involves the situation (input), a goal state (reference), a comparator, a selection mechanism, a feedback predictor, and an action (output). Each element can become the target of social influence, and different types of social influence can be classified and explained by how these elements are targeted. For instance, attempting to persuade another person with strong arguments targets the goal state of the affected individual, while obedience targets the selection mechanism, and violence targets the action. In summary, this article aims to categorize, order, and explain phenomena in social influence research using a feedback loop framework focusing on the influenced individual.

NMDA- and 6-OHDA-induced Lesions in the Nucleus Accumbens Differently Affect Maternal and Infanticidal Behavior in Pup-naïve Female and Male Mice

Virgin and pups-naïve female and male adult mice display two opposite responses when they are exposed to pups for the first time. While females generally take care of the pups, males attack them. Since the nucleus accumbens (NA), and its dopaminergic modulation, is critical in integrating information and processing reward and aversion, we investigated if NMDA- and 6-OHDA-induced lesions, damaging mostly NA output and dopaminergic inputs respectively, affected female maternal behavior (MB) or male infanticidal behavior (IB) in mice. Our results revealed minor or no effects of both smaller and larger NMDA-induced lesions in MB and IB. On the other hand, while 6-OHDA-induced lesions in females reduced the incidence of full MB (12.5% 6-OHDA vs. 85.7% SHAM) increasing the latency to retrieve the pups, those lesions did not affect IB in males. There were no differences in locomotor and exploratory activity between the lesioned- and SHAM- females. Despite those lesions did not induce any major effect on IB, NMDA-lesioned males spent less time in the central area of an open field, while dopaminergic-lesioned males showed reduced number of rearing and peripheral crosses. The current study shows that an intact NA is not necessary for the expression of MB and IB. However, dopaminergic inputs to NA play different role in MB and IB. While damaging dopaminergic terminals into the NA did not affect IB, it clearly delayed the more flexible and rewarding expression of parental behavior.

Role of the basal ganglia in innate and learned behavioural sequences

Integrating individual actions into coherent, organised behavioural units, a process called chunking, is a fundamental, evolutionarily conserved process that renders actions automatic. In vertebrates, evidence points to the basal ganglia - a complex network believed to be involved in action selection - as a key component of action sequence encoding, although the underlying mechanisms are only just beginning to be understood. Central pattern generators control many innate automatic behavioural sequences that form some of the most basic behaviours in an animal's repertoire, and in vertebrates, brainstem and spinal pattern generators are under the control of higher order structures such as the basal ganglia. Evidence suggests that the basal ganglia play a crucial role in the concatenation of simpler behaviours into more complex chunks, in the context of innate behavioural sequences such as chain grooming in rats, as well as sequences in which innate capabilities and learning interact such as birdsong, and sequences that are learned from scratch, such as lever press sequences in operant behaviour. It has been proposed that the role of the striatum, the largest input structure of the basal ganglia, might lie in selecting and allowing the relevant central pattern generators to gain access to the motor system in the correct order, while inhibiting other behaviours. As behaviours become more complex and flexible, the pattern generators seem to become more dependent on descending signals. Indeed, during learning, the striatum itself may adopt the functional characteristics of a higher order pattern generator, facilitated at the microcircuit level by striatal neuropeptides.

Two Distinct Neuronal Populations in the Rat Parafascicular Nucleus Oppositely Encode the Engagement in Stimulus-driven Reward-seeking

BACKGROUND

The thalamus is a phylogenetically well-preserved structure. Known to densely contact cortical regions, its role in the transmission of sensory information to the striatal complex has been widely reconsidered in recent years.

METHODS

The parafascicular nucleus of the thalamus (Pf) has been implicated in the orientation of attention toward salient sensory stimuli. In a stimulus-driven reward-seeking task, we sought to characterize the electrophysiological activity of Pf neurons in rats.

RESULTS

We observed a predominance of excitatory over inhibitory responses for all events in the task. Neurons responded more strongly to the stimulus compared to lever-pressing and reward collecting, confirming the strong involvement of the Pf in sensory information processing. The use of long sessions allowed us to compare neuronal responses to stimuli between trials when animals were engaged in action and those when they were not. We distinguished two populations of neurons with opposite responses: MOTIV+ neurons responded more intensely to stimuli followed by a behavioral response than those that were not. Conversely, MOTIV- neurons responded more strongly when the animal did not respond to the stimulus. In addition, the latency of excitation of MOTIV- neurons was shorter than that of MOTIV+ neurons.

CONCLUSION

Through this encoding, the Pf could perform an early selection of environmental stimuli transmitted to the striatum according to motivational level.

Superior colliculus bidirectionally modulates choice activity in frontal cortex

Action selection occurs through competition between potential choice options. Neural correlates of choice competition are observed across frontal cortex and downstream superior colliculus (SC) during decision-making, yet how these regions interact to mediate choice competition remains unresolved. Here we report that SC can bidirectionally modulate choice competition and drive choice activity in frontal cortex. In the mouse, topographically matched regions of frontal cortex and SC formed a descending motor pathway for directional licking and a re-entrant loop via the thalamus. During decision-making, distinct neuronal populations in both frontal cortex and SC encoded opposing lick directions and exhibited competitive interactions. SC GABAergic neurons encoded ipsilateral choice and locally inhibited glutamatergic neurons that encoded contralateral choice. Activating or suppressing these cell types could bidirectionally drive choice activity in frontal cortex. These results thus identify SC as a major locus to modulate choice competition within the broader action selection network.

How trial-to-trial learning shapes mappings in the mental lexicon: Modelling lexical decision with linear discriminative learning

Trial-to-trial effects have been found in a number of studies, indicating that processing a stimulus influences responses in subsequent trials. A special case are priming effects which have been modelled successfully with error-driven learning (Marsolek, 2008), implying that participants are continuously learning during experiments. This study investigates whether trial-to-trial learning can be detected in an unprimed lexical decision experiment. We used the Discriminative Lexicon Model (DLM; Baayen et al., 2019), a model of the mental lexicon with meaning representations from distributional semantics, which models error-driven incremental learning with the Widrow-Hoff rule. We used data from the British Lexicon Project (BLP; Keuleers et al., 2012) and simulated the lexical decision experiment with the DLM on a trial-by-trial basis for each subject individually. Then, reaction times were predicted with Generalized Additive Models (GAMs), using measures derived from the DLM simulations as predictors. We extracted measures from two simulations per subject (one with learning updates between trials and one without), and used them as input to two GAMs. Learning-based models showed better model fit than the non-learning ones for the majority of subjects. Our measures also provide insights into lexical processing and individual differences. This demonstrates the potential of the DLM to model behavioural data and leads to the conclusion that trial-to-trial learning can indeed be detected in unprimed lexical decision. Our results support the possibility that our lexical knowledge is subject to continuous changes.

Graph analysis of cortical reorganization after virtual reality-based rehabilitation following stroke: a pilot randomized study

Introduction

Stroke is the leading cause of functional disability worldwide. With the increase of the global population, motor rehabilitation of stroke survivors is of ever-increasing importance. In the last decade, virtual reality (VR) technologies for rehabilitation have been extensively studied, to be used instead of or together with conventional treatments such as physiotherapy or occupational therapy. The aim of this work was to evaluate the GestureCollection VR-based rehabilitation tool in terms of the brain changes and clinical outcomes of the patients.

Methods

Two groups of chronic patients underwent a rehabilitation treatment with (experimental) or without (control) complementation with GestureCollection. Functional magnetic resonance imaging exams and clinical assessments were performed before and after the treatment. A functional connectivity graph-based analysis was used to assess differences between the connections and in the network parameters strength and clustering coefficient.

Results

Patients in both groups showed improvement in clinical scales, but there were more increases in functional connectivity in the experimental group than in the control group.

Discussion

The experimental group presented changes in the connections between the frontoparietal and the somatomotor networks, associative cerebellum and basal ganglia, which are regions associated with reward-based motor learning. On the other hand, the control group also had results in the somatomotor network, in its ipsilateral connections with the thalamus and with the motor cerebellum, which are regions more related to a purely mechanical activity. Thus, the use of the GestureCollection system was successfully shown to promote neuroplasticity in several motor-related areas.

A common modular design of nervous systems originating in soft-bodied invertebrates

Nervous systems of vertebrates and invertebrates show a common modular theme in the flow of information for cost-benefit decisions. Sensory inputs are incentivized by integrating stimulus qualities with motivation and memory to affect appetitive state, a system of homeostatic drives, and labelled for directionality. Appetitive state determines action responses from a repertory of possibles and transmits the decision to a premotor system that frames the selected action in motor arousal and appropriate postural and locomotion commands. These commands are then sent to the primary motor pattern generators controlling the motorneurons, with feedback at each stage. In the vertebrates, these stages are mediated by forebrain pallial derivatives for incentive and directionality (olfactory bulb, cerebral cortex, pallial amygdala, etc.) interacting with hypothalamus (homeostasis, motivation, and reward) for action selection in the forebrain basal ganglia, the mid/hindbrain reticular formation as a premotor translator for posture, locomotion, and arousal state, and the spinal cord and cranial nuclei as primary motor pattern generators. Gastropods, like the predatory sea slug Pleurobranchaea californica, show a similar organization but with differences that suggest how complex brains evolved from an ancestral soft-bodied bilaterian along with segmentation, jointed skeletons, and complex exteroceptors. Their premotor feeding network combines functions of hypothalamus and basal ganglia for homeostasis, motivation, presumed reward, and action selection for stimulus approach or avoidance. In Pleurobranchaea, the premotor analogy to the vertebrate reticular formation is the bilateral "A-cluster" of cerebral ganglion neurons that controls posture, locomotion, and serotonergic motor arousal. The A-cluster transmits motor commands to the pedal ganglia analogs of the spinal cord, for primary patterned motor output. Apparent pallial precursors are not immediately evident in Pleurobranchaea's central nervous system, but a notable candidate is a subepithelial nerve net in the peripheral head region that integrates chemotactile stimuli for incentive and directionality. Evolutionary centralization of its computational functions may have led to the olfaction-derived pallial forebrain in the ancestor's vertebrate descendants and their analogs in arthropods and annelids.

Deep brain stimulation alleviates tics in Tourette syndrome via striatal dopamine transmission

Tourette syndrome is a childhood-onset neuropsychiatric disorder characterized by intrusive motor and vocal tics that can lead to self-injury and deleterious mental health complications. While dysfunction in striatal dopamine neurotransmission has been proposed to underlie tic behaviour, evidence is scarce and inconclusive. Deep brain stimulation (DBS) of the thalamic centromedian parafascicular complex (CMPf), an approved surgical interventive treatment for medical refractory Tourette syndrome, may reduce tics by affecting striatal dopamine release. Here, we use electrophysiology, electrochemistry, optogenetics, pharmacological treatments and behavioural measurements to mechanistically examine how thalamic DBS modulates synaptic and tonic dopamine activity in the dorsomedial striatum. Previous studies demonstrated focal disruption of GABAergic transmission in the dorsolateral striatum of rats led to repetitive motor tics recapitulating the major symptom of Tourette syndrome. We employed this model under light anaesthesia and found CMPf DBS evoked synaptic dopamine release and elevated tonic dopamine levels via striatal cholinergic interneurons while concomitantly reducing motor tic behaviour. The improvement in tic behaviour was found to be mediated by D2 receptor activation as blocking this receptor prevented the therapeutic response. Our results demonstrate that release of striatal dopamine mediates the therapeutic effects of CMPf DBS and points to striatal dopamine dysfunction as a driver for motor tics in the pathoneurophysiology of Tourette syndrome.

Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection

The basal ganglia are known to be essential for action selection. However, the functional role of basal ganglia direct and indirect pathways in action selection remains unresolved. Here, by employing cell-type-specific neuronal recording and manipulation in mice trained in a choice task, we demonstrate that multiple dynamic interactions from the direct and indirect pathways control the action selection. While the direct pathway regulates the behavioral choice in a linear manner, the indirect pathway exerts a nonlinear inverted-U-shaped control over action selection, depending on the inputs and the network state. We propose a new center (direct)-surround (indirect)-context (indirect) 'Triple-control' functional model of basal ganglia, which can replicate the physiological and behavioral experimental observations that cannot be simply explained by either the traditional 'Go/No-go' or more recent 'Co-activation' model. These findings have important implications on understanding the basal ganglia circuitry and action selection in health and disease.

Contributions of the Basal Ganglia to Visual Perceptual Decisions

The basal ganglia (BG) make up a prominent nexus between visual and motor-related brain regions. In contrast to the BG's well-established roles in movement control and value-based decision making, their contributions to the transformation of visual input into an action remain unclear, especially in the context of perceptual decisions based on uncertain visual evidence. This article reviews recent progress in our understanding of the BG's contributions to the formation, evaluation, and adjustment of such decisions. From theoretical and experimental perspectives, the review focuses on four key stations in the BG network, namely, the striatum, pallidum, subthalamic nucleus, and midbrain dopamine neurons, which can have different roles and together support the decision process.

Motor cortex is required for flexible but not automatic motor sequences

How motor cortex contributes to motor sequence execution is much debated, with studies supporting disparate views. Here we probe the degree to which motor cortex's engagement depends on task demands, specifically whether its role differs for highly practiced, or 'automatic', sequences versus flexible sequences informed by external events. To test this, we trained rats to generate three-element motor sequences either by overtraining them on a single sequence or by having them follow instructive visual cues. Lesioning motor cortex revealed that it is necessary for flexible cue-driven motor sequences but dispensable for single automatic behaviors trained in isolation. However, when an automatic motor sequence was practiced alongside the flexible task, it became motor cortex-dependent, suggesting that subcortical consolidation of an automatic motor sequence is delayed or prevented when the same sequence is produced also in a flexible context. A simple neural network model recapitulated these results and explained the underlying circuit mechanisms. Our results critically delineate the role of motor cortex in motor sequence execution, describing the condition under which it is engaged and the functions it fulfills, thus reconciling seemingly conflicting views about motor cortex's role in motor sequence generation.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Language disorders in patients with striatal lesions: Deciphering the role of the striatum in language performance

The classical neural model of language refers to a cortical network involving frontal, parietal and temporal regions. However, patients with subcortical lesions of the striatum have language difficulties. We investigated whether the striatum is directly involved in language or whether its role in decision-making has an indirect effect on language performance, by testing carriers of Huntington's disease (HD) mutations and controls. HD is a genetic neurodegenerative disease primarily affecting the striatum and causing language disorders. We asked carriers of the HD mutation in the premanifest (before clinical diagnosis) and early disease stages, and controls to perform two discrimination tasks, one involving linguistic and the other non-linguistic stimuli. We used the hierarchical drift diffusion model (HDDM) to analyze the participants' responses and to assess the decision and non-decision parameters separately. We hypothesized that any language deficits related to decision-making impairments would be reflected in the decision parameters of linguistic and non-linguistic tasks. We also assessed the relative contributions of both HDDM decision and non-decision parameters to the participants' behavioral data (response time and discriminability). Finally, we investigated whether the decision and non-decision parameters of the HDDM were correlated with brain atrophy. The HDDM analysis showed that patients with early HD have impaired decision parameters relative to controls, regardless of the task. In both tasks, decision parameters better explained the variance of response time and discriminability performance than non-decision parameters. In the linguistic task, decision parameters were positively correlated with gray matter volume in the ventral striatum and putamen, whereas non-decision parameters were not. Language impairment in patients with striatal atrophy is better explained by a deficit of decision-making than by a deficit of core linguistic processing. These results suggest that the striatum is involved in language through the modulation of decision-making, presumably by regulating the process of choice between linguistic alternatives.

Separation of Channels Subserving Approach and Avoidance/Escape at the Level of the Basal Ganglia and Related Brainstem Structures

The basal ganglia have the key function of directing our behavior in the context of events from our environment and/or our internal state. This function relies on afferents targeting the main input structures of the basal ganglia, entering bids for action selection at the level of the striatum or sig- nals for behavioral interruption at the level of the subthalamic nucleus, with behavioral reselection facilitated by dopamine signaling. Numerous experiments have studied action selection in relation to inputs from the cerebral cortex. However, less is known about the anatomical and functional link between the basal ganglia and the brainstem. In this review, we describe how brainstem structures also project to the main input structures of the basal ganglia, namely the striatum, the subthalamic nucleus and midbrain dopaminergic neurons, in the context of approach and avoidance (including escape from threat), two fundamental, mutually exclusive behavioral choices in an animal's repertoire in which the brainstem is strongly involved. We focus on three particularly well-described loci involved in approach and avoidance, namely the superior colliculus, the parabrachial nucleus and the periaqueductal grey nucleus. We consider what is known about how these structures are related to the basal ganglia, focusing on their projections toward the striatum, dopaminergic neurons and subthalamic nucleus, and explore the functional consequences of those interactions.

Gamma-Aminobutyric Acid and Glutamate Concentrations in the Striatum and Anterior Cingulate Cortex Not Found to Be Associated with Cognitive Flexibility

Behavioral flexibility and goal-directed behavior heavily depend on fronto-striatal networks. Within these circuits, gamma-aminobutyric acid (GABA) and glutamate play an important role in (motor) response inhibition, but it has remained largely unclear whether they are also relevant for cognitive inhibition. We hence investigated the functional role of these transmitters for cognitive inhibition during cognitive flexibility. Healthy young adults performed two paradigms assessing different aspects of cognitive flexibility. Magnetic resonance spectroscopy (MRS) was used to quantify GABA+ and total glutamate/glutamine (Glx) levels in the striatum and anterior cingulate cortex (ACC) referenced to N-acetylaspartate (NAA). We observed typical task switching and backward inhibition effects, but striatal and ACC concentrations of GABA+/NAA and Glx/NAA were not associated with cognitive flexibility in a functionally relevant manner. The assumption of null effects was underpinned by Bayesian testing. These findings suggest that behavioral and cognitive inhibition are functionally distinct faculties, that depend on (at least partly) different brain structures and neurotransmitter systems. While previous studies consistently demonstrated that motor response inhibition is modulated by ACC and striatal GABA levels, our results suggest that the functionally distinct cognitive inhibition required for successful switching is not, or at least to a much lesser degree, modulated by these factors.

Sensory Reinforced Corticostriatal Plasticity

BACKGROUND

Regional changes in corticostriatal transmission induced by phasic dopaminergic signals are an essential feature of the neural network responsible for instrumental reinforcement during discovery of an action. However, the timing of signals that are thought to contribute to the induction of corticostriatal plasticity is difficult to reconcile within the framework of behavioural reinforcement learning, because the reinforcer is normally delayed relative to the selection and execution of causally-related actions.

OBJECTIVE

While recent studies have started to address the relevance of delayed reinforcement signals and their impact on corticostriatal processing, our objective was to establish a model in which a sensory reinforcer triggers appropriately delayed reinforcement signals relayed to the striatum via intact neuronal pathways and to investigate the effects on corticostriatal plasticity.

METHODS

We measured corticostriatal plasticity with electrophysiological recordings using a light flash as a natural sensory reinforcer, and pharmacological manipulations were applied in an in vivo anesthetized rat model preparation.

RESULTS

We demonstrate that the spiking of striatal neurons evoked by single-pulse stimulation of the motor cortex can be potentiated by a natural sensory reinforcer, operating through intact afferent pathways, with signal timing approximating that required for behavioural reinforcement. The pharmacological blockade of dopamine receptors attenuated the observed potentiation of corticostriatal neurotransmission.

CONCLUSION

This novel in vivo model of corticostriatal plasticity offers a behaviourally relevant framework to address the physiological, anatomical, cellular, and molecular bases of instrumental reinforcement learning.

Multiple dynamic interactions from basal ganglia direct and indirect pathways mediate action selection

The basal ganglia are known to be essential for action selection. However, the functional role of basal ganglia direct and indirect pathways in action selection remains unresolved. Here by employing cell-type-specific neuronal recording and manipulation in mice trained in a choice task, we demonstrate that multiple dynamic interactions from the direct and indirect pathways control the action selection. While the direct pathway regulates the behavioral choice in a linear manner, the indirect pathway exerts a nonlinear inverted-U-shaped control over action selection, depending on the inputs and the network state. We propose a new center (direct) - surround (indirect) - context (indirect) "Triple-control" functional model of basal ganglia, which can replicate the physiological and behavioral experimental observations that cannot be simply explained by either the traditional "Go/No-go" or more recent "Co-activation" model. These findings have important implications on understanding the basal ganglia circuitry and action selection in health and disease.

Understanding brain functional architecture through robotics

Robotics is increasingly seen as a useful test bed for computational models of the brain functional architecture underlying animal behavior. We provide an overview of past and current work, focusing on probabilistic and dynamical models, including approaches premised on the free energy principle, situating this endeavor in relation to evidence that the brain constitutes a layered control system. We argue that future neurorobotic models should integrate multiple neurobiological constraints and be hybrid in nature.

Altered parabrachial nucleus nociceptive processing may underlie central pain in Parkinson's disease

The presence of central neuropathic pain in Parkinson's disease suggests that the brain circuits that allow us to process pain could be dysfunctional in the disorder. However, there is to date no clear pathophysiological mechanism to explain these symptoms. In this work, we present evidence that the dysfunction of the subthalamic nucleus and/or substantia nigra pars reticulata may impact nociceptive processing in the parabrachial nucleus (PBN), a low level primary nociceptive structure in the brainstem, and induce a cellular and molecular neuro-adaptation in this structure. In rat models of Parkinson's disease with a partial dopaminergic lesion in the substantia nigra compacta, we found that the substantia nigra reticulata showed enhanced nociceptive responses. Such responses were less impacted in the subthalamic nucleus. A total dopaminergic lesion produced an increase in the nociceptive responses as well as an increase of the firing rate in both structures. In the PBN, inhibited nociceptive responses and increased expression of GABA_A receptors were found following a total dopaminergic lesion. However, neuro-adaptations at the level of dendritic spine density and post-synaptic density were found in both dopaminergic lesion groups. These results suggest that the molecular changes within the PBN following a larger dopaminergic lesion, such as increased GABA_A expression, is a key mechanism to produce nociceptive processing impairment, whilst other changes may protect function after smaller dopaminergic lesions. We also propose that these neuro-adaptations follow increased inhibitory tone from the substantia nigra pars reticulata and may represent the mechanism generating central neuropathic pain in Parkinson's disease.

Computational models of behavioral addictions: State of the art and future directions

Non-pharmacological behavioral addictions, such as pathological gambling, videogaming, social networking, or internet use, are becoming major public health concerns. It is not yet clear how behavioral addictions could share many major neurobiological and behavioral characteristics with substance use disorders, despite the absence of direct pharmacological influences. A deeper understanding of the neurocognitive mechanisms of addictive behavior is needed, and computational modeling could be one promising approach to explain intricately entwined cognitive and neural dynamics. This review describes computational models of addiction based on reinforcement learning algorithms, Bayesian inference, and biophysical neural simulations. We discuss whether computational frameworks originally conceived to explain maladaptive behavior in substance use disorders can be effectively extended to non-substance-related behavioral addictions. Moreover, we introduce recent studies on behavioral addictions that exemplify the possibility of such extension and propose future directions.

Superior colliculus cell types bidirectionally modulate choice activity in frontal cortex

Action selection occurs through competition between potential choice options. Neural correlates of choice competition are observed across frontal cortex and downstream superior colliculus (SC) during decision-making, yet how these regions interact to mediate choice competition remains unresolved. Here we report that cell types within SC can bidirectionally modulate choice competition and drive choice activity in frontal cortex. In the mouse, topographically matched regions of frontal cortex and SC formed a descending motor pathway for directional licking and a re-entrant loop via the thalamus. During decision-making, distinct neuronal populations in both frontal cortex and SC encoded opposing lick directions and exhibited push-pull dynamics. SC GABAergic neurons encoded ipsilateral choice and glutamatergic neurons encoded contralateral choice, and activating or suppressing these cell types could bidirectionally drive push-pull choice activity in frontal cortex. These results thus identify SC as a major locus to modulate choice competition within the broader action selection network.

Dorsolateral Striatum is a Bottleneck for Responding to Task-Relevant Stimuli in a Learned Whisker Detection Task in Mice

A learned sensory-motor behavior engages multiple brain regions, including the neocortex and the basal ganglia. How a target stimulus is detected by these regions and converted to a motor response remains poorly understood. Here, we performed electrophysiological recordings and pharmacological inactivations of whisker motor cortex and dorsolateral striatum to determine the representations within, and functions of, each region during performance in a selective whisker detection task in male and female mice. From the recording experiments, we observed robust, lateralized sensory responses in both structures. We also observed bilateral choice probability and preresponse activity in both structures, with these features emerging earlier in whisker motor cortex than dorsolateral striatum. These findings establish both whisker motor cortex and dorsolateral striatum as potential contributors to the sensory-to-motor (sensorimotor) transformation. We performed pharmacological inactivation studies to determine the necessity of these brain regions for this task. We found that suppressing the dorsolateral striatum severely disrupts responding to task-relevant stimuli, without disrupting the ability to respond, whereas suppressing whisker motor cortex resulted in more subtle changes in sensory detection and response criterion. Together these data support the dorsolateral striatum as an essential node in the sensorimotor transformation of this whisker detection task.SIGNIFICANCE STATEMENT Selecting an item in a grocery store, hailing a cab - these daily practices require us to transform sensory stimuli into motor responses. Many decades of previous research have studied goal-directed sensory-to-motor transformations within various brain structures, including the neocortex and the basal ganglia. Yet, our understanding of how these regions coordinate to perform sensory-to-motor transformations is limited because these brain structures are often studied by different researchers and through different behavioral tasks. Here, we record and perturb specific regions of the neocortex and the basal ganglia and compare their contributions during performance of a goal-directed somatosensory detection task. We find notable differences in the activities and functions of these regions, which suggests specific contributions to the sensory-to-motor transformation process.

Basal ganglia network dynamics and function: Role of direct, indirect and hyper-direct pathways in action selection

Basal ganglia (BG) are a widely recognized neural basis for action selection, but its decision-making mechanism is still a difficult problem for researchers. Therefore, we constructed a spiking neural network inspired by the BG anatomical data. Simulation experiments were based on the principle of dis-inhibition and our functional hypothesis within the BG: the direct pathway, the indirect pathway, and the hyper-direct pathway of the BG jointly implement the initiation execution and termination of motor programs. Firstly, we studied the dynamic process of action selection with the network, which contained intra-group competition and inter-group competition. Secondly, we focused on the effects of the stimulus intensity and the proportion of excitation and inhibition on the GPi/SNr. The results suggested that inhibition and excitation shape action selection. They also explained why the firing rate of GPi/SNr did not continue to increase in the action-selection experiment. Finally, we discussed the experimental results with the functional hypothesis. Uniquely, this paper summarized the decision-making neural mechanism of action selection based on the direct pathway, the indirect pathway, and the hyper-direct pathway within BG.

Experiential values are underweighted in decisions involving symbolic options

Standard models of decision-making assume each option is associated with subjective value, regardless of whether this value is inferred from experience (experiential) or explicitly instructed probabilistic outcomes (symbolic). In this study, we present results that challenge the assumption of unified representation of experiential and symbolic value. Across nine experiments, we presented participants with hybrid decisions between experiential and symbolic options. Participants' choices exhibited a pattern consistent with a systematic neglect of the experiential values. This normatively irrational decision strategy held after accounting for alternative explanations, and persisted even when it bore an economic cost. Overall, our results demonstrate that experiential and symbolic values are not symmetrically considered in hybrid decisions, suggesting they recruit different representational systems that may be assigned different priority levels in the decision process. These findings challenge the dominant models commonly used in value-based decision-making research.

The Subthalamic Nucleus: A Hub for Sensory Control via Short Three- Lateral Loop Connections with the Brainstem?

The subthalamic nucleus (STN) is classically subdivided into sensori-motor, associative and limbic regions, which is consistent with the involvement of this structure in not only motor control, but also in cognitive and emotional tasks. However, the function of the sensory inputs to the STN's sensori-motor territory is comparatively less well explored, although sensory responses have been reported in this structure. There is still a paucity of information regarding the characteristics of that subdivision and its potential functional role in basal ganglia processing and more widely in associated networks. In this perspective paper, we summarize the type of sensory stimuli that have been reported to activate the STN, and describe the complex sensory properties of the STN and its anatomical link to a sensory network involving the brainstem, characterized in our recent work. Analyzing the sensory input to the STN led us to suggest the existence of previously unreported threelateral subcortical loops between the basal ganglia and the brainstem which do not involve the cortex. Anatomically, these loops closely link the STN, the substantia nigra pars reticulata and various structures from the brainstem such as the superior colliculus and the parabrachial nucleus. We also discuss the potential role of the STN in the control of sensory activity in the brainstem and its possible contribution to favoring sensory habituation or sensitization over brainstem structures to optimize the best selection of action at a given time.

Cortical-subcortical interactions in goal-directed behavior

Flexibly selecting appropriate actions in response to complex, ever-changing environments requires both cortical and subcortical regions, which are typically described as participating in a strict hierarchy. In this traditional view, highly specialized subcortical circuits allow for efficient responses to salient stimuli, at the cost of adaptability and context specificity, which are attributed to the neocortex. Their interactions are often described as the cortex providing top-down command signals for subcortical structures to implement; however, as available technologies develop, studies increasingly demonstrate that behavior is represented by brainwide activity and that even subcortical structures contain early signals of choice, suggesting that behavioral functions emerge as a result of different regions interacting as truly collaborative networks. In this review, we discuss the field's evolving understanding of how cortical and subcortical regions in placental mammals interact cooperatively, not only via top-down cortical-subcortical inputs but through bottom-up interactions, especially via the thalamus. We describe our current understanding of the circuitry of both the cortex and two exemplar subcortical structures, the superior colliculus and striatum, to identify which information is prioritized by which regions. We then describe the functional circuits these regions form with one another, and the thalamus, to create parallel loops and complex networks for brainwide information flow. Finally, we challenge the classic view that functional modules are contained within specific brain regions; instead, we propose that certain regions prioritize specific types of information over others, but the subnetworks they form, defined by their anatomical connections and functional dynamics, are the basis of true specialization.

Integrated neural dynamics of sensorimotor decisions and actions

Recent theoretical models suggest that deciding about actions and executing them are not implemented by completely distinct neural mechanisms but are instead two modes of an integrated dynamical system. Here, we investigate this proposal by examining how neural activity unfolds during a dynamic decision-making task within the high-dimensional space defined by the activity of cells in monkey dorsal premotor (PMd), primary motor (M1), and dorsolateral prefrontal cortex (dlPFC) as well as the external and internal segments of the globus pallidus (GPe, GPi). Dimensionality reduction shows that the four strongest components of neural activity are functionally interpretable, reflecting a state transition between deliberation and commitment, the transformation of sensory evidence into a choice, and the baseline and slope of the rising urgency to decide. Analysis of the contribution of each population to these components shows meaningful differences between regions but no distinct clusters within each region, consistent with an integrated dynamical system. During deliberation, cortical activity unfolds on a two-dimensional "decision manifold" defined by sensory evidence and urgency and falls off this manifold at the moment of commitment into a choice-dependent trajectory leading to movement initiation. The structure of the manifold varies between regions: In PMd, it is curved; in M1, it is nearly perfectly flat; and in dlPFC, it is almost entirely confined to the sensory evidence dimension. In contrast, pallidal activity during deliberation is primarily defined by urgency. We suggest that these findings reveal the distinct functional contributions of different brain regions to an integrated dynamical system governing action selection and execution.

Cerebellar Contributions to the Basal Ganglia Influence Motor Coordination, Reward Processing, and Movement Vigor

Both the cerebellum and the basal ganglia are known for their roles in motor control and motivated behavior. These two systems have been classically considered as independent structures that coordinate their contributions to behavior via separate cortico-thalamic loops. However, recent evidence demonstrates the presence of a rich set of direct connections between these two regions. Although there is strong evidence for connections in both directions, for brevity we limit our discussion to the better-characterized connections from the cerebellum to the basal ganglia. We review two sets of such connections: disynaptic projections through the thalamus and direct monosynaptic projections to the midbrain dopaminergic nuclei, the VTA and the SNc. In each case, we review the evidence for these pathways from anatomic tracing and physiological recordings, and discuss their potential functional roles. We present evidence that the disynaptic pathway through the thalamus is involved in motor coordination, and that its dysfunction contributes to motor deficits, such as dystonia. We then discuss how cerebellar projections to the VTA and SNc influence dopamine release in the respective targets of these nuclei: the NAc and the dorsal striatum. We argue that the cerebellar projections to the VTA may play a role in reward-based learning and therefore contribute to addictive behavior, whereas the projection to the SNc may contribute to movement vigor. Finally, we speculate how these projections may explain many of the observations that indicate a role for the cerebellum in mental disorders, such as schizophrenia.

Black-box and surrogate optimization for tuning spiking neural models of striatum plasticity

The basal ganglia (BG) is a brain structure that has long been proposed to play an essential role in action selection, and theoretical models of spiking neurons have tried to explain how the BG solves this problem. A recently proposed functional and biologically inspired network model of the striatum (an important nucleus of the BG) is based on spike-timing-dependent eligibility (STDE) and captured important experimental features of this nucleus. The model can recognize complex input patterns and consistently choose rewarded actions to respond to such sensory inputs. However, model tuning is challenging due to two main reasons. The first is the expert knowledge required, resulting in tedious and potentially biased trial-and-error procedures. The second is the computational cost of assessing model configurations (approximately 1.78 h per evaluation). This study addresses the model tuning problem through numerical optimization. Considering the cost of assessing solutions, the selected methods stand out due to their low requirements for solution evaluations and compatibility with high-performance computing. They are the SurrogateOpt solver of Matlab and the RBFOpt library, both based on radial basis function approximations, and DIRECT-GL, an enhanced version of the widespread black-box optimizer DIRECT. Besides, a parallel random search serves as a baseline reference of the outcome of opting for sophisticated methods. SurrogateOpt turns out to be the best option for tuning this kind of model. It outperforms, on average, the quality of the configuration found by an expert and works significantly faster and autonomously. RBFOpt and the random search share the second position, but their average results are below the option found by hand. Finally, DIRECT-GL follows this line becoming the worst-performing method.

Neurophysiological Mechanisms of Implicit and Explicit Memory in the Process of Consciousness

Neurophysiological mechanisms are increasingly understood to constitute the foundations of human conscious experience. These include the capacity for ongoing memory, achieved through a hierarchy of reentrant cross-laminar connections across limbic, heteromodal, unimodal, and primary cortices. The neurophysiological mechanisms of consciousness also include the capacity for volitional direction of attention to the ongoing cognitive process, through a reentrant fronto-thalamo-cortical network regulation of the inhibitory thalamic reticular nucleus. More elusive is the way that discrete objects of subjective experience, such as the color of deep blue or the sound of middle C, could be generated by neural mechanisms. Explaining such ineffable qualities of subjective experience is what Chalmers has called “the hard problem of consciousness,” which has divided modern neuroscientists and philosophers alike. We propose that insight into the appearance of the hard problem can be gained through integrating classical phenomenological studies of experience with recent progress in the differential neurophysiology of consolidating explicit versus implicit memory. Although the achievement of consciousness, once it is reflected upon, becomes explicit, the underlying process of generating consciousness, through neurophysiological mechanisms, is largely implicit. Studying the neurophysiological mechanisms of adaptive implicit memory, including brain stem, limbic, and thalamic regulation of neocortical representations, may lead to a more extended phenomenological understanding of both the neurophysiological process and the subjective experience of consciousness.

NEW & NOTEWORTHY The process of consciousness, generating the qualia that may appear to be irreducible qualities of experience, can be understood to arise from neurophysiological mechanisms of memory. Implicit memory, organized by the lemnothalamic brain stem projections and dorsal limbic consolidation in REM sleep, supports the unconscious field and the quasi-conscious fringe of current awareness. Explicit memory, organized by the collothalamic midbrain projections and ventral limbic consolidation of NREM sleep, supports the focal objects of consciousness.

A computational model of inner speech supporting flexible goal-directed behaviour in Autism

Experimental and computational studies propose that inner speech boosts categorisation skills and executive functions, making human behaviour more focused and flexible. In addition, many clinical studies highlight a relationship between poor inner-speech and an executive impairment in autism spectrum condition (ASC), but contrasting findings are reported. Here we directly investigate the latter issue through a previously implemented and validated computational model of the Wisconsin Cards Sorting Tests. In particular, the model was applied to explore potential individual differences in cognitive flexibility and inner speech contribution in autistic and neurotypical participants. Our model predicts that the use of inner-speech could increase along the life-span of neurotypical participants but would be reduced in autistic ones. Although we found more attentional failures (i.e., wrong behavioural rule switches) in autistic children/teenagers and more perseverative behaviours in autistic young/older adults, only autistic children and older adults exhibited a lower performance (i.e., fewer consecutive correct rule switches) than matched control groups. Overall, our results corroborate the idea that the reduced use of inner speech could represent a disadvantage for autistic children and autistic older adults. Moreover, the results suggest that cognitive-behavioural therapies should focus on developing inner speech skills in autistic children as this could provide cognitive support throughout their whole life span.

An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits

The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

Reward prediction errors, not sensory prediction errors, play a major role in model selection in human reinforcement learning

Model-based reinforcement learning enables an agent to learn in variable environments and tasks by optimizing its actions based on the predicted states and outcomes. This mechanism has also been considered in the brain. However, exactly how the brain selects an appropriate model for confronting environments has remained unclear. Here, we investigated the model selection algorithm in the human brain during a reinforcement learning task. One primary theory of model selection in the brain is based on sensory prediction errors. Here, we compared this theory with an alternative possibility of internal model selection with reward prediction errors. To compare these two theories, we devised a switching experiment from a first-order Markov decision process to a second-order Markov decision process that provides either reward- or sensory prediction error regarding environmental change. We tested two representative computational models driven by different prediction errors. One is the sensory prediction-error-driven Bayesian algorithm, which has been discussed as a representative internal model selection algorithm in the animal reinforcement learning task. The other is the reward-prediction-error-driven policy gradient algorithm. We compared the simulation results of these two computational models with human reinforcement learning behaviors. The model fitting result supports that the policy gradient algorithm is preferable to the Bayesian algorithm. This suggests that the human brain employs the reward prediction error to select an appropriate internal model in the reinforcement learning task.

Action suppression reveals opponent parallel control via striatal circuits

The direct and indirect pathways of the basal ganglia are classically thought to promote and suppress action, respectively¹. However, the observed co-activation of striatal direct and indirect medium spiny neurons² (dMSNs and iMSNs, respectively) has challenged this view. Here we study these circuits in mice performing an interval categorization task that requires a series of self-initiated and cued actions and, critically, a sustained period of dynamic action suppression. Although movement produced the co-activation of iMSNs and dMSNs in the sensorimotor, dorsolateral striatum (DLS), fibre photometry and photo-identified electrophysiological recordings revealed signatures of functional opponency between the two pathways during action suppression. Notably, optogenetic inhibition showed that DLS circuits were largely engaged to suppress-and not promote-action. Specifically, iMSNs on a given hemisphere were dynamically engaged to suppress tempting contralateral action. To understand how such regionally specific circuit function arose, we constructed a computational reinforcement learning model that reproduced key features of behaviour, neural activity and optogenetic inhibition. The model predicted that parallel striatal circuits outside the DLS learned the action-promoting functions, generating the temptation to act. Consistent with this, optogenetic inhibition experiments revealed that dMSNs in the associative, dorsomedial striatum, in contrast to those in the DLS, promote contralateral actions. These data highlight how opponent interactions between multiple circuit- and region-specific basal ganglia processes can lead to behavioural control, and establish a critical role for the sensorimotor indirect pathway in the proactive suppression of tempting actions.

Differential Dopamine Receptor-Dependent Sensitivity Improves the Switch Between Hard and Soft Selection in a Model of the Basal Ganglia

The problem of selecting one action from a set of different possible actions, simply referred to as the problem of action selection, is a ubiquitous challenge in the animal world. For vertebrates, the basal ganglia (BG) are widely thought to implement the core computation to solve this problem, as its anatomy and physiology are well suited to this end. However, the BG still display physiological features whose role in achieving efficient action selection remains unclear. In particular, it is known that the two types of dopaminergic receptors (D1 and D2) present in the BG give rise to mechanistically different responses. The overall effect will be a difference in sensitivity to dopamine, which may have ramifications for action selection. However, which receptor type leads to a stronger response is unclear due to the complexity of the intracellular mechanisms involved. In this study, we use an existing, high-level computational model of the BG, which assumes that dopamine contributes to action selection by enabling a switch between different selection regimes, to predict which of D1 or D2 has the greater sensitivity. Thus, we ask, Assuming dopamine enables a switch between action selection regimes in the BG, what functional sensitivity values would result in improved action selection computation? To do this, we quantitatively assessed the model's capacity to perform action selection as we parametrically manipulated the sensitivity weights of D1 and D2. We show that differential (rather than equal) D1 and D2 sensitivity to dopaminergic input improves the switch between selection regimes during the action selection computation in our model. Specifically, greater D2 sensitivity compared to D1 led to these improvements.

An insula hierarchical network architecture for active interoceptive inference

In the brain, the insular cortex receives a vast amount of interoceptive information, ascending through deep brain structures, from multiple visceral organs. The unique hierarchical and modular architecture of the insula suggests specialization for processing interoceptive afferents. Yet, the biological significance of the insula's neuroanatomical architecture, in relation to deep brain structures, remains obscure. In this opinion piece, we propose the Insula Hierarchical Modular Adaptive Interoception Control (IMAC) model to suggest that insula modules (granular, dysgranular and agranular), forming parallel networks with the prefrontal cortex and striatum, are specialized to form higher order interoceptive representations. These interoceptive representations are recruited in a context-dependent manner to support habitual, model-based and exploratory control of visceral organs and physiological processes. We discuss how insula interoceptive representations may give rise to conscious feelings that best explain lower order deep brain interoceptive representations, and how the insula may serve to defend the body and mind against pathological depression.

Neural Control of Action Selection Among Innate Behaviors

Nervous systems must not only generate specific adaptive behaviors, such as reproduction, aggression, feeding, and sleep, but also select a single behavior for execution at any given time, depending on both internal states and external environmental conditions. Despite their tremendous biological importance, the neural mechanisms of action selection remain poorly understood. In the past decade, studies in the model animal Drosophila melanogaster have demonstrated valuable neural mechanisms underlying action selection of innate behaviors. In this review, we summarize circuit mechanisms with a particular focus on a small number of sexually dimorphic neurons in controlling action selection among sex, fight, feeding, and sleep behaviors in both sexes of flies. We also discuss potentially conserved circuit configurations and neuromodulation of action selection in both the fly and mouse models, aiming to provide insights into action selection and the sexually dimorphic prioritization of innate behaviors.

Uncertainty-guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.

Dopamine Modulation of Drosophila Ellipsoid Body Neurons, a Nod to the Mammalian Basal Ganglia

The central complex (CX) is a neural structure located on the midline of the insect brain that has been widely studied in the last few years. Its role in navigation and goal-oriented behaviors resembles those played by the basal ganglia in mammals. However, the neural mechanisms and the neurotransmitters involved in these processes remain unclear. Here, we exploited an in vivo bioluminescence Ca²⁺ imaging technique to record the activity in targeted neurons of the ellipsoid body (EB). We used different drugs to evoke excitatory Ca²⁺-responses, depending on the putative neurotransmitter released by their presynaptic inputs, while concomitant dopamine administration was employed to modulate those excitations. By using a genetic approach to knockdown the dopamine 1-like receptors, we showed that different dopamine modulatory effects are likely due to specific receptors expressed by the targeted population of neurons. Altogether, these results provide new data concerning how dopamine modulates and shapes the response of the ellipsoid body neurons. Moreover, they provide important insights regarding the similitude with mammals as far as the role played by dopamine in increasing and stabilizing the response of goal-related information.

Thalamic bursts modulate cortical synchrony locally to switch between states of global functional connectivity in a cognitive task

Performing a cognitive task requires going through a sequence of functionally diverse stages. Although it is typically assumed that these stages are characterized by distinct states of cortical synchrony that are triggered by sub-cortical events, little reported evidence supports this hypothesis. To test this hypothesis, we first identified cognitive stages in single-trial MEG data of an associative recognition task, showing with a novel method that each stage begins with local modulations of synchrony followed by a state of directed functional connectivity. Second, we developed the first whole-brain model that can simulate cortical synchrony throughout a task. The model suggests that the observed synchrony is caused by thalamocortical bursts at the onset of each stage, targeted at cortical synapses and interacting with the structural anatomical connectivity. These findings confirm that cognitive stages are defined by distinct states of cortical synchrony and explains the network-level mechanisms necessary for reaching stage-dependent synchrony states.

A novel machine-learning method was applied to unveil the dynamics of local and cortex-wide neural coordination underlying the fundamental cognitive processes involved in a memory task. To explain how neural activity–and ultimately behavior–was coordinated throughout the task, we developed a whole-brain model that incorporates cognitive mechanisms, anatomy, and neural biophysics. Similar models are regularly used with resting state data, but simulating a cognitive task remained elusive. By using hidden semi-Markov models to divide the task into stages with separate connectivity patterns, we were able to generalize the whole brain model from resting state to cognitive task data. The model showed that sub-cortical pulses at the onset of cognitive processes–as hypothesized by cognitive and neurophysiological theories–were sufficient to switch between the states of neural coordination observed. These findings have implications for understanding goal-directed cognitive processing and the mechanisms needed to reach states of neural coordination.