Learning robust cortico-cortical associations with the basal ganglia: An integrative review

Emotional Theory of Rationality

In recent decades, the existence of a close relationship between emotional phenomena and rational processes has certainly been established, yet there is still no unified definition or effective model to describe them. To advance our understanding of the mechanisms governing the behavior of living beings, we must integrate multiple theories, experiments, and models from both fields. In this article we propose a new theoretical framework that allows integrating and understanding the emotion-cognition duality, from a functional point of view. Based on evolutionary principles, our reasoning adds to the definition and understanding of emotion, justifying its origin, explaining its mission and dynamics, and linking it to higher cognitive processes, mainly with attention, cognition, decision-making, and consciousness. According to our theory, emotions are the mechanism for brain function optimization, aside from the contingency and stimuli prioritization system. As a result of this approach, we have developed a dynamic systems-level model capable of providing plausible explanations for certain psychological and behavioral phenomena and establishing a new framework for the scientific definition of some fundamental psychological terms.

Habits without values

Habits form a crucial component of behavior. In recent years, key computational models have conceptualized habits as arising from model-free reinforcement learning mechanisms, which typically select between available actions based on the future value expected to result from each. Traditionally, however, habits have been understood as behaviors that can be triggered directly by a stimulus, without requiring the animal to evaluate expected outcomes. Here, we develop a computational model instantiating this traditional view, in which habits develop through the direct strengthening of recently taken actions rather than through the encoding of outcomes. We demonstrate that this model accounts for key behavioral manifestations of habits, including insensitivity to outcome devaluation and contingency degradation, as well as the effects of reinforcement schedule on the rate of habit formation. The model also explains the prevalent observation of perseveration in repeated-choice tasks as an additional behavioral manifestation of the habit system. We suggest that mapping habitual behaviors onto value-free mechanisms provides a parsimonious account of existing behavioral and neural data. This mapping may provide a new foundation for building robust and comprehensive models of the interaction of habits with other, more goal-directed types of behaviors and help to better guide research into the neural mechanisms underlying control of instrumental behavior more generally. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

A Computational Model of Dual Competition between the Basal Ganglia and the Cortex

We propose a model that includes interactions between the cortex, the basal ganglia (BG), and the thalamus based on a dual competition. We hypothesize that the striatum, the subthalamic nucleus (STN), the internal globus pallidus (GPi), the thalamus, and the cortex are involved in closed feedback loops through the hyperdirect and direct pathways. These loops support a competition process that results in the ability of BG to make a cognitive decision followed by a motor one. Considering lateral cortical interactions, another competition takes place inside the cortex allowing the latter to make a cognitive and a motor decision. We show how this dual competition endows the model with two regimes. One is driven by reinforcement learning and the other by Hebbian learning. The final decision is made according to a combination of these two mechanisms with a gradual transfer from the former to the latter. We confirmed these theoretical results on primates (Macaca mulatta) using a novel paradigm predicted by the model.

Abnormal asymmetries in subcortical brain volume in early adolescents with subclinical psychotic experiences

Subcortical structures may have an important role in the pathophysiology of psychosis. Our recent mega-analysis of structural magnetic resonance imaging (MRI) data has reported subcortical volumetric and lateralization alterations in chronic schizophrenia, including leftward asymmetric increases in pallidal volume. The question remains, however, whether these characteristics may represent vulnerability to the development of psychosis or whether they are epiphenomena caused by exposure to medication or illness chronicity. Subclinical psychotic experiences (SPEs) occur in some adolescents in the general population and increase the odds of developing psychosis in young adulthood. Investigations into the association between SPEs and MRI-measured volumes of subcortical structures in the general adolescent population would clarify the issue. Here, we collected structural MRI data in a subsample (10.5-13.3 years old) of a large-scale population-based cohort and explored subcortical volume and lateralization alterations related to SPEs (N = 203). Adolescents with SPEs demonstrated significant volumetric increases in the left hippocampus, right caudate, and right lateral ventricle, as well as a marginally significant increase in the left pallidum. Furthermore, adolescents with SPEs showed significantly more leftward laterality of pallidal volume than individuals without SPEs, which replicates our mega-analysis findings in chronic schizophrenia. We suggest that leftward asymmetries in pallidal volume already present in early adolescence may underlie the premorbid predisposition for developing psychosis in later life.

On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach

In addition to the prefrontal cortex (PFC), the basal ganglia (BG) have been increasingly often reported to play a fundamental role in category learning, but the circuit mechanisms mediating their interaction remain to be explored. We developed a novel neurocomputational model of category learning that particularly addresses the BG-PFC interplay. We propose that the BG bias PFC activity by removing the inhibition of cortico-thalamo-cortical loop and thereby provide a teaching signal to guide the acquisition of category representations in the corticocortical associations to the PFC. Our model replicates key behavioral and physiological data of macaque monkey learning a prototype distortion task from Antzoulatos and Miller (2011) Our simulations allowed us to gain a deeper insight into the observed drop of category selectivity in striatal neurons seen in the experimental data and in the model. The simulation results and a new analysis of the experimental data based on the model's predictions show that the drop in category selectivity of the striatum emerges as the variability of responses in the striatum rises when confronting the BG with an increasingly larger number of stimuli to be classified. The neurocomputational model therefore provides new testable insights of systems-level brain circuits involved in category learning that may also be generalized to better understand other cortico-BG-cortical loops.SIGNIFICANCE STATEMENT Inspired by the idea that basal ganglia (BG) teach the prefrontal cortex (PFC) to acquire category representations, we developed a novel neurocomputational model and tested it on a task that was recently applied in monkey experiments. As an advantage over previous models of category learning, our model allows to compare simulation data with single-cell recordings in PFC and BG. We not only derived model predictions, but already verified a prediction to explain the observed drop in striatal category selectivity. When testing our model with a simple, real-world face categorization task, we observed that the fast striatal learning with a performance of 85% correct responses can teach the slower PFC learning to push the model performance up to almost 100%.

A natural history of skills

The dorsal pallium (a.k.a. cortex in mammals) makes a loop circuit with the basal ganglia and the thalamus known to control and adapt behavior but the who's who of the functional roles of these structures is still debated. Influenced by the Triune brain theory that was proposed in the early sixties, many current theories propose a hierarchical organization on the top of which stands the cortex to which the subcortical structures are subordinated. In particular, habits formation has been proposed to reflect a switch from conscious on-line control of behavior by the cortex, to a fully automated subcortical control. In this review, we propose to revalue the function of the network in light of the current experimental evidence concerning the anatomy and physiology of the basal ganglia-cortical circuits in vertebrates. We briefly review the current theories and show that they could be encompassed in a broader framework of skill learning and performance. Then, after reminding the state of the art concerning the anatomical architecture of the network and the underlying dynamic processes, we summarize the evolution of the anatomical and physiological substrate of skill learning and performance among vertebrates. We then review experimental evidence supporting for the hypothesis that the development of automatized skills relies on the BG teaching cortical circuits and is actually a late feature linked with the development of a specialized cortex or pallium that evolved in parallel in different taxa. We finally propose a minimal computational framework where this hypothesis can be explicitly implemented and tested.

Neural circuits activated by error amplification and haptic guidance training techniques during performance of a timing-based motor task by healthy individuals

To promote motor learning, robotic devices have been used to improve subjects' performance by guiding desired movements (haptic guidance-HG) or by artificially increasing movement errors to foster a more rapid learning (error amplification-EA). To better understand the neurophysiological basis of motor learning, a few studies have evaluated brain regions activated during EA/HG, but none has compared both approaches. The goal of this study was to investigate using fMRI which brain networks were activated during a single training session of HG/EA in healthy adults learning to play a computerized pinball-like timing task. Subjects had to trigger a robotic device by flexing their wrist at the correct timing to activate a virtual flipper and hit a falling ball towards randomly positioned targets. During training with HG/EA, subjects' timing errors were decreased/increased, respectively, by the robotic device to delay or accelerate their wrist movement. The results showed that at the beginning of the training period with HG/EA, an error-detection network, including cerebellum and angular gyrus, was activated, consistent with subjects recognizing discrepancies between their intended actions and the actual movement timing. At the end of the training period, an error-detection network was still present for EA, while a memory consolidation/automatization network (caudate head and parahippocampal gyrus) was activated for HG. The results indicate that training movement with various kinds of robotic input relies on different brain networks. Better understanding the neurophysiological underpinnings of brain processes during HG/EA could prove useful for optimizing rehabilitative movement training for people with different patterns of brain damage.

Get a grip: individual variations in grip strength are a marker of brain health

Demonstrations that grip strength has predictive power in relation to a range of health conditions-even when these are assessed decades later-has motivated claims that hand-grip dynamometry has the potential to serve as a "vital sign" for middle-aged and older adults. Central to this belief has been the assumption that grip strength is a simple measure of physical performance that provides a marker of muscle status in general, and sarcopenia in particular. It is now evident that while differences in grip strength between individuals are influenced by musculoskeletal factors, "lifespan" changes in grip strength within individuals are exquisitely sensitive to integrity of neural systems that mediate the control of coordinated movement. The close and pervasive relationships between age-related declines in maximum grip strength and expressions of cognitive dysfunction can therefore be understood in terms of the convergent functional and structural mediation of cognitive and motor processes by the human brain. In the context of aging, maximum grip strength is a discriminating measure of neurological function and brain health.

Cortical Power-Density Changes of Different Frequency Bands in Visually Guided Associative Learning: A Human EEG-Study

The computer-based Rutgers Acquired Equivalence test (RAET) is a widely used paradigm to test the function of subcortical structures in visual associative learning. The test consists of an acquisition (pair learning) and a test (rule transfer) phase, associated with the function of the basal ganglia and the hippocampi, respectively. Obviously, such a complex task also requires cortical involvement. To investigate the activity of different cortical areas during this test, 64-channel EEG recordings were recorded in 24 healthy volunteers. Fast-Fourier and Morlet wavelet convolution analyses were performed on the recordings. The most robust power changes were observed in the theta (4–7 Hz) and gamma (>30 Hz) frequency bands, in which significant power elevation was observed in the vast majority of the subjects, over the parieto-occipital and temporo-parietal areas during the acquisition phase. The involvement of the frontal areas in the acquisition phase was remarkably weaker. No remarkable cortical power elevations were found in the test phase. In fact, the power of the alpha and beta bands was significantly decreased over the parietooccipital areas. We conclude that the initial acquisition of the image pairs requires strong cortical involvement, but once the pairs have been learned, neither retrieval nor generalization requires strong cortical contribution.

How neuroscience can inform the study of individual differences in cognitive abilities

Theories of human mental abilities should be consistent with what is known in neuroscience. Currently, tests of human mental abilities are modeled by cognitive constructs such as attention, working memory, and speed of information processing. These constructs are in turn related to a single general ability. However, brains are very complex systems and whether most of the variability between the operations of different brains can be ascribed to a single factor is questionable. Research in neuroscience suggests that psychological processes such as perception, attention, decision, and executive control are emergent properties of interacting distributed networks. The modules that make up these networks use similar computational processes that involve multiple forms of neural plasticity, each having different time constants. Accordingly, these networks might best be characterized in terms of the information they process rather than in terms of abstract psychological processes such as working memory and executive control.

Temporal-prefrontal cortical network for discrimination of valuable objects in long-term memory

Remembering and discriminating objects based on their previously learned values are essential for goal-directed behaviors. While the cerebral cortex is known to contribute to object recognition, surprisingly little is known about its role in retaining long-term object-value associations. To address this question, we trained macaques to arbitrarily associate small or large rewards with many random fractal objects (>100) and then used fMRI to study the long-term retention of value-based response selectivity across the brain. We found a pronounced long-term value memory in core subregions of temporal and prefrontal cortex where, several months after training, fractals previously associated with high reward ("good" stimuli) elicited elevated fMRI responses compared with those associated with low reward ("bad" stimuli). Similar long-term value-based modulation was also observed in subregions of the striatum, amygdala, and claustrum, but not in the hippocampus. The value-modulated temporal-prefrontal subregions showed strong resting-state functional connectivity to each other. Moreover, for areas outside this core, the magnitude of long-term value responses was predicted by the strength of resting-state functional connectivity to the core subregions. In separate testing, free-viewing gaze behavior indicated that the monkeys retained stable long-term memory of object value. These results suggest an implicit and high-capacity memory mechanism in the temporal-prefrontal circuitry and its associated subcortical regions for long-term retention of object-value memories that can guide value-oriented behavior.

Circuit Mechanisms of Sensorimotor Learning

The relationship between the brain and the environment is flexible, forming the foundation for our ability to learn. Here we review the current state of our understanding of the modifications in the sensorimotor pathway related to sensorimotor learning. We divide the process into three hierarchical levels with distinct goals: (1) sensory perceptual learning, (2) sensorimotor associative learning, and (3) motor skill learning. Perceptual learning optimizes the representations of important sensory stimuli. Associative learning and the initial phase of motor skill learning are ensured by feedback-based mechanisms that permit trial-and-error learning. The later phase of motor skill learning may primarily involve feedback-independent mechanisms operating under the classic Hebbian rule. With these changes under distinct constraints and mechanisms, sensorimotor learning establishes dedicated circuitry for the reproduction of stereotyped neural activity patterns and behavior.

White matter developmental trajectories associated with persistence and recovery of childhood stuttering

Stuttering affects the fundamental human ability of fluent speech production, and can have a significant negative impact on an individual's psychosocial development. While the disorder affects about 5% of all preschool children, approximately 80% of them recover naturally within a few years of stuttering onset. The pathophysiology and neuroanatomical development trajectories associated with persistence and recovery of stuttering are still largely unknown. Here, the first mixed longitudinal diffusion tensor imaging (DTI) study of childhood stuttering has been reported. A total of 195 high quality DTI scans from 35 children who stutter (CWS) and 43 controls between 3 and 12 years of age were acquired, with an average of three scans per child, each collected approximately a year apart. Fractional anisotropy (FA), a measure reflecting white matter structural coherence, was analyzed voxel-wise to examine group and age-related differences using a linear mixed-effects (LME) model. Results showed that CWS exhibited decreased FA relative to controls in the left arcuate fasciculus, underlying the inferior parietal and posterior temporal areas, and the mid body of corpus callosum. Further, white matter developmental trajectories reflecting growth rate of these tract regions differentiated children with persistent stuttering from those who recovered from stuttering. Specifically, a reduction in FA growth rate (i.e., slower FA growth with age) in persistent children relative to fluent controls in the left arcuate fasciculus and corpus callosum was found, which was not evident in recovered children. These findings provide first glimpses into the possible neural mechanisms of onset, persistence, and recovery of childhood stuttering. Hum Brain Mapp 38:3345-3359, 2017. © 2017 Wiley Periodicals, Inc.

The Role of Interneurons in Autism and Tourette Syndrome

The brain includes multiple types of interconnected excitatory and inhibitory neurons that together allow us to move, think, feel, and interact with the environment. Inhibitory interneurons (INs) comprise a small, heterogeneous fraction, but they exert a powerful and tight control over neuronal activity and consequently modulate the magnitude of neuronal output and, ultimately, information processing. IN abnormalities are linked to two pediatric psychiatric disorders with high comorbidity: autism spectrum disorder (ASD) and Tourette syndrome (TS). Studies probing the basis of this link have been contradictory regarding whether the causative mechanism is a reduction in number, dysfunction, or gene aberrant expression (or a combination thereof). Here, we integrate different theories into a more comprehensive view of INs as responsible for the symptomatology observed in these disorders.

Learning multiple variable-speed sequences in striatum via cortical tutoring

Sparse, sequential patterns of neural activity have been observed in numerous brain areas during timekeeping and motor sequence tasks. Inspired by such observations, we construct a model of the striatum, an all-inhibitory circuit where sequential activity patterns are prominent, addressing the following key challenges: (i) obtaining control over temporal rescaling of the sequence speed, with the ability to generalize to new speeds; (ii) facilitating flexible expression of distinct sequences via selective activation, concatenation, and recycling of specific subsequences; and (iii) enabling the biologically plausible learning of sequences, consistent with the decoupling of learning and execution suggested by lesion studies showing that cortical circuits are necessary for learning, but that subcortical circuits are sufficient to drive learned behaviors. The same mechanisms that we describe can also be applied to circuits with both excitatory and inhibitory populations, and hence may underlie general features of sequential neural activity pattern generation in the brain.

DOI:http://dx.doi.org/10.7554/eLife.26084.001

The roles of valuation and reward processing in cognitive function and psychiatric disorders

In neuroeconomics, valuation refers to the process of assigning values to states and actions on the basis of the animal's current representation of the environment, while reward processing corresponds to processing the feedback received from the environment to update the values of states and actions. In this article, we review the brain circuits associated with valuation and reward processing and argue that these are fundamental processes critical to many cognitive functions. Specifically, we focus on the role of valuation and reward processing in attention, memory, decision making, and learning. Next, the extant neuroimaging literature on a number of psychiatric disorders is reviewed (i.e., addiction, pathological gambling, schizophrenia, and mood disorders), and an argument is made that associated deficits in cognitive functions can be explained in terms of abnormal valuation and reward processing. The review concludes with the impact of this framework in clinical settings and prescriptions for future research, in particular with regard to the conversions of qualitatively different valuation systems into a system of common currency.

Striatal functional connectivity changes following specific balance training in elderly people: MRI results of a randomized controlled pilot study

BACKGROUND

Practice-induced effects of specific balance training on brain structure and activity in elderly people are largely unknown.

AIM

In the present study, we investigated morphological and functional brain changes following slacking training (balancing over nylon ribbons) in a group of elderly people.

METHODS

Twenty-eight healthy volunteers were recruited and randomly assigned to the intervention (mean age: 62.3±5.4years) or control group (mean age: 61.8±5.3years). The intervention group completed six-weeks of slackline training. Brain morphological changes were investigated using voxel-based morphometry and functional connectivity changes were computed via independent component analysis and seed-based analyses. All analyses were applied to the whole sample and to a subgroup of participants who improved in slackline performance.

RESULTS

The repeated measures analysis of variance showed a significant interaction effect between groups and sessions. Specifically, the Tukey post-hoc analysis revealed a significantly improved slackline standing performance after training for the left leg stance time (pre: 4.5±3.6s vs. 26.0±30.0s, p<0.038) as well as for tandem stance time (pre: 1.4±0.6s vs. post: 4.5±4.0s, p=0.003) in the intervention group. No significant changes in balance performance were observed in the control group. The MRI analysis did not reveal morphological or functional connectivity differences before or after the training between the intervention and control groups (whole sample). However, subsequent analysis in subjects with improved slackline performance showed a decrease of connectivity between the striatum and other brain areas during the training period.

CONCLUSION

These preliminary results suggest that improved balance performance with slackline training goes along with an increased efficiency of the striatal network.

Dissociable changes in functional network topology underlie early category learning and development of automaticity

Recent work has shown that multimodal association areas-including frontal, temporal, and parietal cortex-are focal points of functional network reconfiguration during human learning and performance of cognitive tasks. On the other hand, neurocomputational theories of category learning suggest that the basal ganglia and related subcortical structures are focal points of functional network reconfiguration during early learning of some categorization tasks but become less so with the development of automatic categorization performance. Using a combination of network science and multilevel regression, we explore how changes in the connectivity of small brain regions can predict behavioral changes during training in a visual categorization task. We find that initial category learning, as indexed by changes in accuracy, is predicted by increasingly efficient integrative processing in subcortical areas, with higher functional specialization, more efficient integration across modules, but a lower cost in terms of redundancy of information processing. The development of automaticity, as indexed by changes in the speed of correct responses, was predicted by lower clustering (particularly in subcortical areas), higher strength (highest in cortical areas), and higher betweenness centrality. By combining neurocomputational theories and network scientific methods, these results synthesize the dissociative roles of multimodal association areas and subcortical structures in the development of automaticity during category learning.

What is automatized during perceptual categorization?

An experiment is described that tested whether stimulus-response associations or an abstract rule are automatized during extensive practice at perceptual categorization. Twenty-seven participants each completed 12,300 trials of perceptual categorization, either on rule-based (RB) categories that could be learned explicitly or information-integration (II) categories that required procedural learning. Each participant practiced predominantly on a primary category structure, but every third session they switched to a secondary structure that used the same stimuli and responses. Half the stimuli retained their same response on the primary and secondary categories (the congruent stimuli) and half switched responses (the incongruent stimuli). Several results stood out. First, performance on the primary categories met the standard criteria of automaticity by the end of training. Second, for the primary categories in the RB condition, accuracy and response time (RT) were identical on congruent and incongruent stimuli. In contrast, for the primary II categories, accuracy was higher and RT was lower for congruent than for incongruent stimuli. These results are consistent with the hypothesis that rules are automatized in RB tasks, whereas stimulus-response associations are automatized in II tasks. A cognitive neuroscience theory is proposed that accounts for these results.

Simulating the Effect of Reinforcement Learning on Neuronal Synchrony and Periodicity in the Striatum

The study of rhythms and oscillations in the brain is gaining attention. While it is unclear exactly what the role of oscillation, synchrony, and rhythm is, it appears increasingly likely that synchrony is related to normal and abnormal brain states and possibly cognition. In this article, we explore the relationship between basal ganglia (BG) synchrony and reinforcement learning. We simulate a biologically-realistic model of the striatum initially proposed by Ponzi and Wickens (2010) and enhance the model by adding plastic cortico-BG synapses that can be modified using reinforcement learning. The effect of reinforcement learning on striatal rhythmic activity is then explored, and disrupted using simulated deep brain stimulation (DBS). The stimulator injects current in the brain structure to which it is attached, which affects neuronal synchrony. The results show that training the model without DBS yields a high accuracy in the learning task and reduced the number of active neurons in the striatum, along with an increased firing periodicity and a decreased firing synchrony between neurons in the same assembly. In addition, a spectral decomposition shows a stronger signal for correct trials than incorrect trials in high frequency bands. If the DBS is ON during the training phase, but not the test phase, the amount of learning in the model is reduced, along with firing periodicity. Similar to when the DBS is OFF, spectral decomposition shows a stronger signal for correct trials than for incorrect trials in high frequency domains, but this phenoemenon happens in higher frequency bands than when the DBS is OFF. Synchrony between the neurons is not affected. Finally, the results show that turning the DBS ON at test increases both firing periodicity and striatal synchrony, and spectral decomposition of the signal show that neural activity synchronizes with the DBS fundamental frequency (and its harmonics). Turning the DBS ON during the test phase results in chance performance regardless of whether the DBS was ON or OFF during training. We conclude that reinforcement learning is related to firing periodicity, and a stronger signal for correct trials when compared to incorrect trials in high frequency bands.

The temporal dynamics of reversal learning: P3 amplitude predicts valence-specific behavioral adjustment

Adapting behavior to dynamic stimulus-reward contingences is a core feature of reversal learning and a capacity thought to be critical to socio-emotional behavior. Impairment in reversal learning has been linked to multiple psychiatric outcomes, including depression, Parkinson's disorder, and substance abuse. A recent influential study introduced an innovative laboratory reversal-learning paradigm capable of disentangling the roles of feedback valence and expectancy. Here, we sought to use this paradigm in order to examine the time-course of reward and punishment learning using event-related potentials among a large, representative sample (N=101). Three distinct phases of processing were examined: initial feedback evaluation (reward positivity, or RewP), allocation of attention (P3), and sustained processing (late positive potential, or LPP). Results indicate a differential pattern of valence and expectancy across these processing stages: the RewP was uniquely related to valence (i.e., positive vs. negative feedback), the P3 was uniquely associated with expectancy (i.e., unexpected vs. expected feedback), and the LPP was sensitive to both valence and expectancy (i.e., main effects of each, but no interaction). The link between ERP amplitudes and behavioral performance was strongest for the P3, and this association was valence-specific. Overall, these findings highlight the potential utility of the P3 as a neural marker for feedback processing in reversal-based learning and establish a foundation for future research in clinical populations.

Mesolimbic confidence signals guide perceptual learning in the absence of external feedback

It is well established that learning can occur without external feedback, yet normative reinforcement learning theories have difficulties explaining such instances of learning. Here, we propose that human observers are capable of generating their own feedback signals by monitoring internal decision variables. We investigated this hypothesis in a visual perceptual learning task using fMRI and confidence reports as a measure for this monitoring process. Employing a novel computational model in which learning is guided by confidence-based reinforcement signals, we found that mesolimbic brain areas encoded both anticipation and prediction error of confidence—in remarkable similarity to previous findings for external reward-based feedback. We demonstrate that the model accounts for choice and confidence reports and show that the mesolimbic confidence prediction error modulation derived through the model predicts individual learning success. These results provide a mechanistic neurobiological explanation for learning without external feedback by augmenting reinforcement models with confidence-based feedback.

DOI:http://dx.doi.org/10.7554/eLife.13388.001

Much of our behavior is shaped by feedback from the environment. We repeat behaviors that previously led to rewards and avoid those with negative outcomes. At the same time, we can learn in many situations without such feedback. Our ability to perceive sensory stimuli, for example, improves with training even in the absence of external feedback.

Guggenmos et al. hypothesized that this form of perceptual learning may be guided by self-generated feedback that is based on the confidence in our performance. The general idea is that the brain reinforces behaviors associated with states of high confidence, and weakens behaviors that lead to low confidence.

To test this idea, Guggenmos et al. used a technique called functional magnetic resonance imaging to record the brain activity of healthy volunteers as they performed a visual learning task. In this task, the participants had to judge the orientation of barely visible line gratings and then state how confident they were in their decisions.

Feedback signals derived from the participants’ confidence reports activated the same brain areas typically engaged for external feedback or reward. Moreover, just as these regions were previously found to signal the difference between actual and expected rewards, so did they signal the difference between actual confidence levels and those expected on the basis of previous confidence levels. This parallel suggests that confidence may take over the role of external feedback in cases where no such feedback is available. Finally, the extent to which an individual exhibited these signals predicted overall learning success.

Future studies could investigate whether these confidence signals are automatically generated, or whether they only emerge when participants are required to report their confidence levels. Another open question is whether such self-generated feedback applies in non-perceptual forms of learning, where learning without feedback has likewise been a long-standing puzzle.

DOI:http://dx.doi.org/10.7554/eLife.13388.002

Gray and White Matter Contributions to Cognitive Frontostriatal Deficits in Non-Demented Parkinson's Disease

Objective

This prospective investigation examined: 1) processing speed and working memory relative to other cognitive domains in non-demented medically managed idiopathic Parkinson’s disease, and 2) the predictive role of cortical/subcortical gray thickness/volume and white matter fractional anisotropy on processing speed and working memory.

Methods

Participants completed a neuropsychological protocol, Unified Parkinson’s Disease Rating Scale, brain MRI, and fasting blood draw to rule out vascular contributors. Within group a priori anatomical contributors included bilateral frontal thickness, caudate nuclei volume, and prefrontal white matter fractional anisotropy.

Results

Idiopathic Parkinson’s disease (n = 40; Hoehn & Yahr stages 1–3) and non-Parkinson’s disease ‘control’ peers (n = 40) matched on demographics, general cognition, comorbidity, and imaging/blood vascular metrics. Cognitively, individuals with Parkinson’s disease were significantly more impaired than controls on tests of processing speed, secondary deficits on working memory, with subtle impairments in memory, abstract reasoning, and visuoperceptual/spatial abilities. Anatomically, Parkinson’s disease individuals were not statistically different in cortical gray thickness or subcortical gray volumes with the exception of the putamen. Tract Based Spatial Statistics showed reduced prefrontal fractional anisotropy for Parkinson’s disease relative to controls. Within Parkinson’s disease, prefrontal fractional anisotropy and caudate nucleus volume partially explained processing speed. For controls, only prefrontal white matter was a significant contributor to processing speed. There were no significant anatomical predictors of working memory for either group.

Conclusions

Caudate nuclei volume and prefrontal fractional anisotropy, not frontal gray matter thickness, showed unique and combined significance for processing speed in Parkinson’s disease. Findings underscore the relevance for examining gray-white matter interactions and also highlight clinical processing speed metrics as potential indicators of early cognitive impairment in PD.

Voxel seed coherent source analysis on transient global amnesia patients

Transient global amnesia (TGA) is a rare neurological disorder with a sudden, temporary episode of memory loss which usually occurs in old age. The episodic loss of memory becomes normal after a stipulated time of approximately 24 hours. The precise pathology is not yet completely understood. Moreover, there is no proper neuroimaging method to assess this condition. In this study, the EEG was measured at two time points one with the occurrence of the episode (acute) and the second time point after the patient returns to the normal memory condition (follow-up). The aim of the study was to look at the pathological network involved during the acute phase and the follow up phase in these patients for the five frequency bands, namely, delta, theta, alpha, beta, and gamma. The method used for the source analyses was a beamforming approach called dynamic imaging of coherent sources in the frequency domain. The seed voxel was the lesion area taken from the anatomical MRI of each patient. The cortical and subcortical network comprised of the caudate and cerebellum in case of the delta band frequency. Two temporal sources in case of the theta band. Temporal, medial frontal, parietal, putamen, and thalamus sources were found in case of the alpha band. Prefrontal, parietal, and thalamus sources were found in case of the beta band. Temporal and thalamus in case of the gamma band frequency. All these sources were involved in the acute phase. Moreover, in the follow-up phase the motor area, in all frequency bands except gamma band, was additionally active followed by parietal and occipital regions in alpha and gamma frequencies. The differences involved in the network of sources between the two phases gives us better understanding of this neurological disorder.

Real-Time Strategy Video Game Experience and Visual Perceptual Learning

Visual perceptual learning (VPL) is defined as long-term improvement in performance on a visual-perception task after visual experiences or training. Early studies have found that VPL is highly specific for the trained feature and location, suggesting that VPL is associated with changes in the early visual cortex. However, the generality of visual skills enhancement attributable to action video-game experience suggests that VPL can result from improvement in higher cognitive skills. If so, experience in real-time strategy (RTS) video-game play, which may heavily involve cognitive skills, may also facilitate VPL. To test this hypothesis, we compared VPL between RTS video-game players (VGPs) and non-VGPs (NVGPs) and elucidated underlying structural and functional neural mechanisms. Healthy young human subjects underwent six training sessions on a texture discrimination task. Diffusion-tensor and functional magnetic resonance imaging were performed before and after training. VGPs performed better than NVGPs in the early phase of training. White-matter connectivity between the right external capsule and visual cortex and neuronal activity in the right inferior frontal gyrus (IFG) and anterior cingulate cortex (ACC) were greater in VGPs than NVGPs and were significantly correlated with RTS video-game experience. In both VGPs and NVGPs, there was task-related neuronal activity in the right IFG, ACC, and striatum, which was strengthened after training. These results indicate that RTS video-game experience, associated with changes in higher-order cognitive functions and connectivity between visual and cognitive areas, facilitates VPL in early phases of training. The results support the hypothesis that VPL can occur without involvement of only visual areas. Significance statement: Although early studies found that visual perceptual learning (VPL) is associated with involvement of the visual cortex, generality of visual skills enhancement by action video-game experience suggests that higher-order cognition may be involved in VPL. If so, real-time strategy (RTS) video-game experience may facilitate VPL as a result of heavy involvement of cognitive skills. Here, we compared VPL between RTS video-game players (VGPs) and non-VGPs (NVGPs) and investigated the underlying neural mechanisms. VGPs showed better performance in the early phase of training on the texture discrimination task and greater level of neuronal activity in cognitive areas and structural connectivity between visual and cognitive areas than NVGPs. These results support the hypothesis that VPL can occur beyond the visual cortex.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

Flexible Coding of Task Rules in Frontoparietal Cortex: An Adaptive System for Flexible Cognitive Control

How do our brains achieve the cognitive control that is required for flexible behavior? Several models of cognitive control propose a role for frontoparietal cortex in the structure and representation of task sets or rules. For behavior to be flexible, however, the system must also rapidly reorganize as mental focus changes. Here we used multivoxel pattern analysis of fMRI data to demonstrate adaptive reorganization of frontoparietal activity patterns following a change in the complexity of the task rules. When task rules were relatively simple, frontoparietal cortex did not hold detectable information about these rules. In contrast, when the rules were more complex, frontoparietal cortex showed clear and decodable rule discrimination. Our data demonstrate that frontoparietal activity adjusts to task complexity, with better discrimination of rules that are behaviorally more confusable. The change in coding was specific to the rule element of the task and was not mirrored in more specialized cortex (early visual cortex) where coding was independent of difficulty. In line with an adaptive view of frontoparietal function, the data suggest a system that rapidly reconfigures in accordance with the difficulty of a behavioral task. This system may provide a neural basis for the flexible control of human behavior.

Brain Events Underlying Episodic Memory Changes in Aging: A Longitudinal Investigation of Structural and Functional Connectivity

Episodic memories are established and maintained by close interplay between hippocampus and other cortical regions, but degradation of a fronto-striatal network has been suggested to be a driving force of memory decline in aging. We wanted to directly address how changes in hippocampal-cortical versus striatal-cortical networks over time impact episodic memory with age. We followed 119 healthy participants (20-83 years) for 3.5 years with repeated tests of episodic verbal memory and magnetic resonance imaging for quantification of functional and structural connectivity and regional brain atrophy. While hippocampal-cortical functional connectivity predicted memory change in young, changes in cortico-striatal functional connectivity were related to change in recall in older adults. Within each age group, effects of functional and structural connectivity were anatomically closely aligned. Interestingly, the relationship between functional connectivity and memory was strongest in the age ranges where the rate of reduction of the relevant brain structure was lowest, implying selective impacts of the different brain events on memory. Together, these findings suggest a partly sequential and partly simultaneous model of brain events underlying cognitive changes in aging, where different functional and structural events are more or less important in various time windows, dismissing a simple uni-factorial view on neurocognitive aging.

Criterion learning in rule-based categorization: simulation of neural mechanism and new data

In perceptual categorization, rule selection consists of selecting one or several stimulus-dimensions to be used to categorize the stimuli (e.g., categorize lines according to their length). Once a rule has been selected, criterion learning consists of defining how stimuli will be grouped using the selected dimension(s) (e.g., if the selected rule is line length, define 'long' and 'short'). Very little is known about the neuroscience of criterion learning, and most existing computational models do not provide a biological mechanism for this process. In this article, we introduce a new model of rule learning called Heterosynaptic Inhibitory Criterion Learning (HICL). HICL includes a biologically-based explanation of criterion learning, and we use new category-learning data to test key aspects of the model. In HICL, rule selective cells in prefrontal cortex modulate stimulus-response associations using pre-synaptic inhibition. Criterion learning is implemented by a new type of heterosynaptic error-driven Hebbian learning at inhibitory synapses that uses feedback to drive cell activation above/below thresholds representing ionic gating mechanisms. The model is used to account for new human categorization data from two experiments showing that: (1) changing rule criterion on a given dimension is easier if irrelevant dimensions are also changing (Experiment 1), and (2) showing that changing the relevant rule dimension and learning a new criterion is more difficult, but also facilitated by a change in the irrelevant dimension (Experiment 2). We conclude with a discussion of some of HICL's implications for future research on rule learning.

A neurocomputational model of automatic sequence production

Most behaviors unfold in time and include a sequence of submovements or cognitive activities. In addition, most behaviors are automatic and repeated daily throughout life. Yet, relatively little is known about the neurobiology of automatic sequence production. Past research suggests a gradual transfer from the associative striatum to the sensorimotor striatum, but a number of more recent studies challenge this role of the BG in automatic sequence production. In this article, we propose a new neurocomputational model of automatic sequence production in which the main role of the BG is to train cortical-cortical connections within the premotor areas that are responsible for automatic sequence production. The new model is used to simulate four different data sets from human and nonhuman animals, including (1) behavioral data (e.g., RTs), (2) electrophysiology data (e.g., single-neuron recordings), (3) macrostructure data (e.g., TMS), and (4) neurological circuit data (e.g., inactivation studies). We conclude with a comparison of the new model with existing models of automatic sequence production and discuss a possible new role for the BG in automaticity and its implication for Parkinson's disease.