Comparison of population activity in the dorsal premotor cortex and putamen during the learning of arbitrary visuomotor mappings

Static and dynamic coding in distinct cell types during associative learning in the prefrontal cortex

The prefrontal cortex maintains information in memory through static or dynamic population codes depending on task demands, but whether the population coding schemes used are learning-dependent and differ between cell types is currently unknown. We investigate the population coding properties and temporal stability of neurons recorded from male macaques in two mapping tasks during and after stimulus-response associative learning, and then we use a Strategy task with the same stimuli and responses as control. We identify a heterogeneous population coding for stimuli, responses, and novel associations: static for putative pyramidal cells and dynamic for putative interneurons that show the strongest selectivity for all the variables. The population coding of learned associations shows overall the highest stability driven by cell types, with interneurons changing from dynamic to static coding after successful learning. The results support that prefrontal microcircuitry expresses mixed population coding governed by cell types and changes its stability during associative learning.

Instruction-based learning: A review

Humans are able to learn to implement novel rules from instructions rapidly, which is termed "instruction-based learning" (IBL). This remarkable ability is very important in our daily life in both learning individually or working as a team, and almost every psychology experiment starts with instructing participants. Many recent progresses have been made in IBL research both psychologically and neuroscientifically. In this review, we discuss the role of language in IBL, the importance of the first trial performance in IBL, why IBL should be considered as a goal-directed behavior, intelligence and IBL, cognitive flexibility and IBL, how behaviorally relevant information is processed in the lateral prefrontal cortex (LPFC), how the lateral frontal cortex (LFC) networks work as a functional hierarchy during IBL, and the cortical and subcortical contributions to IBL. Finally, we develop a neural working model for IBL and provide some sensible directions for future research.

Striatal activity topographically reflects cortical activity

The cortex projects to the dorsal striatum topographically^1,2 to regulate behaviour^3-5, but spiking activity in the two structures has previously been reported to have markedly different relations to sensorimotor events^6-9. Here we show that the relationship between activity in the cortex and striatum is spatiotemporally precise, topographic, causal and invariant to behaviour. We simultaneously recorded activity across large regions of the cortex and across the width of the dorsal striatum in mice that performed a visually guided task. Striatal activity followed a mediolateral gradient in which behavioural correlates progressed from visual cue to response movement to reward licking. The summed activity in each part of the striatum closely and specifically mirrored activity in topographically associated cortical regions, regardless of task engagement. This relationship held for medium spiny neurons and fast-spiking interneurons, whereas the activity of tonically active neurons differed from cortical activity with stereotypical responses to sensory or reward events. Inactivation of the visual cortex abolished striatal responses to visual stimuli, supporting a causal role of cortical inputs in driving the striatum. Striatal visual responses were larger in trained mice than untrained mice, with no corresponding change in overall activity in the visual cortex. Striatal activity therefore reflects a consistent, causal and scalable topographical mapping of cortical activity.

Stereotyped, automatized and habitual behaviours: are they similar constructs under the control of the same cerebral areas?

Comprehensive knowledge about higher executive functions of motor control has been covered in the last decades. Critical goals have been targeted through many different technological approaches. An abundant flow of new results greatly progressed our ability to respond at better-posited answers to look more than ever at the challenging neural system functioning. Behaviour is the observable result of the invisible, as complex cerebral functioning. Many pathological states are approached after symptomatology categorisation of behavioural impairments is achieved. Motor, non-motor and psychiatric signs are greatly shared by many neurological/psychiatric disorders. Together with the cerebral cortex, the basal ganglia contribute to the expression of behaviour promoting the correct action schemas and the selection of appropriate sub-goals based on the evaluation of action outcomes. The present review focus on the basic classification of higher motor control functioning, taking into account the recent advances in basal ganglia structural knowledge and the computational model of basal ganglia functioning. We discuss about the basal ganglia capability in executing ordered motor patterns in which any single movement is linked to each other into an action, and many actions are ordered into each other, giving them a syntactic value to the final behaviour. The stereotypic, automatized and habitual behaviour's constructs and controls are the expression of successive stages of rule internalization and categorisation aimed in producing the perfect spatial-temporal control of motor command.

Neural Interactions Underlying Visuomotor Associations in the Human Brain

Rapid and flexible learning during behavioral choices is critical to our daily endeavors and constitutes a hallmark of dynamic reasoning. An important paradigm to examine flexible behavior involves learning new arbitrary associations mapping visual inputs to motor outputs. We conjectured that visuomotor rules are instantiated by translating visual signals into actions through dynamic interactions between visual, frontal and motor cortex. We evaluated the neural representation of such visuomotor rules by performing intracranial field potential recordings in epilepsy subjects during a rule-learning delayed match-to-behavior task. Learning new visuomotor mappings led to the emergence of specific responses associating visual signals with motor outputs in 3 anatomical clusters in frontal, anteroventral temporal and posterior parietal cortex. After learning, mapping selective signals during the delay period showed interactions with visual and motor signals. These observations provide initial steps towards elucidating the dynamic circuits underlying flexible behavior and how communication between subregions of frontal, temporal, and parietal cortex leads to rapid learning of task-relevant choices.

Common and Distinct Functional Brain Networks for Intuitive and Deliberate Decision Making

Reinforcement learning studies in rodents and primates demonstrate that goal-directed and habitual choice behaviors are mediated through different fronto-striatal systems, but the evidence is less clear in humans. In this study, functional magnetic resonance imaging (fMRI) data were collected whilst participants (n = 20) performed a conditional associative learning task in which blocks of novel conditional stimuli (CS) required a deliberate choice, and blocks of familiar CS required an intuitive choice. Using standard subtraction analysis for fMRI event-related designs, activation shifted from the dorso-fronto-parietal network, which involves dorsolateral prefrontal cortex (DLPFC) for deliberate choice of novel CS, to ventro-medial frontal (VMPFC) and anterior cingulate cortex for intuitive choice of familiar CS. Supporting this finding, psycho-physiological interaction (PPI) analysis, using the peak active areas within the PFC for novel and familiar CS as seed regions, showed functional coupling between caudate and DLPFC when processing novel CS and VMPFC when processing familiar CS. These findings demonstrate separable systems for deliberate and intuitive processing, which is in keeping with rodent and primate reinforcement learning studies, although in humans they operate in a dynamic, possibly synergistic, manner particularly at the level of the striatum.

Modulation of associative learning in the hippocampal-striatal circuit based on item-set similarity

Mounting evidence suggests that the medial temporal lobe (MTL) and striatal learning systems support different forms of learning, which can be competitive or cooperative depending on task demands. We have previously shown how activity in these regions can be modulated in a conditional visuomotor associative learning task based on the consistency of response mappings or reward feedback (Mattfeld & Stark, 2015). Here, we examined the shift in learning towards the MTL and away from the striatum by placing strong demands on pattern separation, a process of orthogonalizing similar inputs into distinct representations. Mnemonically, pattern separation processes have been shown to rely heavily on processing in the hippocampus. Therefore, we predicted modulation of hippocampal activity by pattern separation demands, but no such modulation of striatal activity. Using a variant of the conditional visuomotor associative learning task that we have used previously, we presented participants with two blocked conditions: items with high and low perceptual overlap during functional magnetic resonance imaging (fMRI). As predicted, we observed learning-related activity in the hippocampus, which was greater in the high than the low overlap condition, particularly in the dentate gyrus. In contrast, the associative striatum also showed learning related activity, but it was not modulated by overlap condition. Using functional connectivity analyses, we showed that the correlation between the hippocampus and dentate gyrus with the associative striatum was differentially modulated by high vs. low overlap, suggesting that the coordination between these regions was affected when pattern separation demands were high. These findings contribute to a growing literature that suggests that the hippocampus and striatal network both contribute to the learning of arbitrary associations that are computationally distinct and can be altered by task demands.

Dissociating Two Stages of Preparation in the Stop Signal Task Using fMRI

Often we must balance being prepared to act quickly with being prepared to suddenly stop. The stop signal task (SST) is widely used to study inhibitory control, and provides a measure of the speed of the stop process that is robust to changes in subjects' response strategy. Previous studies have shown that preparation affects inhibition. We used fMRI to separate activity that occurs after a brief (500 ms) warning stimulus (warning-phase) from activity that occurs during responses that follow (response-phase). Both of these phases could contribute to the preparedness to stop because they both precede stop signals. Warning stimuli activated posterior networks that signal the need for top-down control, whereas response phases engaged prefrontal and subcortical networks that implement top-down control. Regression analyses revealed that both of these phases affect inhibitory control in different ways. Warning-phase activity in the cerebellum and posterior cingulate predicted stop latency and accuracy, respectively. By contrast, response-phase activity in fronto-temporal areas and left striatum predicted go speed and stop accuracy, in pre-supplementary motor area affected stop accuracy, and in right striatum predicted stop latency and accuracy. The ability to separate hidden contributions to inhibitory control during warning-phases from those during response-phases can aid in the study of models of preparation and inhibitory control, and of disorders marked by poor top-down control.

Dorsal premotor cortex: neural correlates of reach target decisions based on a color-location matching rule and conflicting sensory evidence

We recorded single-neuron activity in dorsal premotor (PMd) and primary motor cortex (M1) of two monkeys in a reach-target selection task. The monkeys chose between two color-coded potential targets by determining which target's color matched the predominant color of a multicolored checkerboard-like Decision Cue (DC). Different DCs contained differing numbers of colored squares matching each target. The DCs provided evidence about the correct target ranging from unambiguous (one color only) to very ambiguous and conflicting (nearly equal number of squares of each color). Differences in choice behavior (reach response times and success rates as a function of DC ambiguity) of the monkeys suggested that each applied a different strategy for using the target-choice evidence in the DCs. Nevertheless, the appearance of the DCs evoked a transient coactivation of PMd neurons preferring both potential targets in both monkeys. Reach response time depended both on how long it took activity to increase in neurons that preferred the chosen target and on how long it took to suppress the activity of neurons that preferred the rejected target, in both correct-choice and error-choice trials. These results indicate that PMd neurons in this task are not activated exclusively by a signal proportional to the net color bias of the DCs. They are instead initially modulated by the conflicting evidence supporting both response choices; final target selection may result from a competition between representations of the alternative choices. The results also indicate a temporal overlap between action selection and action initiation processes in PMd and M1.

Functional contributions and interactions between the human hippocampus and subregions of the striatum during arbitrary associative learning and memory

The hippocampus and striatum are thought to have different functional roles in learning and memory. It is unknown under what experimental conditions their contributions are dissimilar or converge, and the extent to which they interact over the course of learning. In order to evaluate both the functional contributions of as well as the interactions between the human hippocampus and striatum, the present study used high-resolution functional magnetic resonance imaging (fMRI) and variations of a conditional visuomotor associative learning task that either taxed arbitrary associative learning (Experiment 1) or stimulus-response learning (Experiment 2). In the first experiment, we observed changes in activity in the hippocampus and anterior caudate that reflect differences between the two regions consistent with distinct computational principles. In the second experiment, we observed activity in the putamen that reflected content specific representations during the learning of arbitrary conditional visuomotor associations. In both experiments, the hippocampus and ventral striatum demonstrated dynamic functional coupling during the learning of new arbitrary associations, but not during retrieval of well-learned arbitrary associations using control variants of the tasks that did not preferentially tax one system versus the other. These findings suggest that both the hippocampus and subregions of the dorsal striatum contribute uniquely to the learning of arbitrary associations while the hippocampus and ventral striatum interact over the course of learning.

Reach target selection in humans using ambiguous decision cues containing variable amounts of conflicting sensory evidence supporting each target choice

Human subjects chose between two color-coded reach targets using multicolored checkerboard-like decision cues (DCs) that presented variable amounts of conflicting sensory evidence supporting both target choices. Different DCs contained different numbers of small squares of the two target colors. The most ambiguous DCs contained nearly equal numbers of squares of both target colors. The subjects reached as soon as they selected a target after the appearance of the DC (“choose-and-go” task). The choice behavior of the subjects showed many similarities to prior studies using other stimulus properties (e.g., visual motion coherence, brightness), including progressively longer response times and higher target-choice error rates for more ambiguous DCs. However, certain trends in their choice behavior could not be fully captured by simple drift-diffusion models. Allowing the subjects to view the DCs for a period of time before presenting the targets (“match-to-sample” task) resulted in much shorter response times overall, but also revealed a reluctance of subjects to commit to a decision about the predominant color of the more ambiguous DCs during the initial extended observation period. Model processing and simulation analyses suggest that the subjects might adjust the dynamics of their decision-making process on a trial-to-trial basis in response to the variable level of ambiguous and conflicting evidence in different DCs between trials.

Comparing neural substrates of emotional vs. non-emotional conflict modulation by global control context

The efficiency with which the brain resolves conflict in information processing is determined by contextual factors that modulate internal control states, such as the recent (local) and longer-term (global) occurrence of conflict. Local “control context” effects can be observed in trial-by-trial adjustments to conflict (congruency sequence effects: less interference following incongruent trials), whereas global control context effects are reflected in adjustments to the frequency of conflict encountered over longer sequences of trials (“proportion congruent effects”: less interference when incongruent trials are frequent). Previous neuroimaging and lesion studies suggest that the modulation of conflict-control processes by local control context relies on partly dissociable neural circuits for cognitive (non-emotional) vs. emotional conflicts. By contrast, emotional and non-emotional conflict-control processes have not been contrasted with respect to their modulation by global control context. We addressed this aim in a functional magnetic resonance imaging (fMRI) study that varied the proportion of congruent trials in emotional vs. non-emotional conflict tasks across blocks. We observed domain-general conflict-related signals in the dorsal anterior cingulate cortex (dACC) and pre-supplementary motor area and, more importantly, task-domain also interacted with global control context effects: specifically, the dorsal striatum and anterior insula tracked control-modulated conflict effects exclusively in the emotional domain. These results suggest that, similar to the neural mechanisms of local control context effects, there are both overlapping as well as distinct neural substrates involved in the modulation of emotional and non-emotional conflict-control by global control context.

Many hats: intratrial and reward level-dependent BOLD activity in the striatum and premotor cortex

Human functional magnetic resonance imaging (fMRI) studies, as well as lesion, drug, and single-cell recording studies in animals, suggest that the striatum plays a key role in associating sensory events with rewarding actions, both by facilitating reward processing and prediction (i.e., reinforcement learning) and by biasing and later updating action selection. Previous human neuroimaging research has failed to dissociate striatal activity associated with reward, stimulus, and response processing, and previous electrophysiological research in nonhuman animals has typically only examined single striatal subregions. Overcoming both these limitations, we isolated blood oxygen level-dependent (BOLD) signal associated with four intratrial processes (stimulus, preparation of response, response, and feedback) in a visuomotor learning task and examined activity associated with each within four striatal subregions (ventral striatum, putamen, head of the caudate nucleus, and body of the caudate) and the lateral premotor cortex. Overall, the striatum and lateral premotor cortex were recruited during all trial components, confirming their importance in all aspects of visuomotor learning. However, the caudate was most active at stimulus and feedback, whereas the putamen peaked in activity at response. Activation in the lateral premotor cortex was, surprisingly, strongest during stimulus and following response as feedback approached. Activity was additionally examined at three reward magnitudes. Reward magnitude affected neural activity only during stimulus in the caudate, putamen, and premotor cortex, whereas the ventral striatum showed reward sensitivity during both stimulus and feedback. Collectively, these results indicate that each striatal region makes a unique contribution to visuomotor learning through functions performed at different points within single trials.

Advanced Parkinson's disease effect on goal-directed and habitual processes involved in visuomotor associative learning

The present behavioral study re-addresses the question of habit learning in Parkinson's disease (PD). Patients were early onset, non-demented, dopa-responsive, candidates for surgical treatment, similar to those we found earlier as suffering greater dopamine depletion in the putamen than in the caudate nucleus. The task was the same conditional associative learning task as that used previously in monkeys and healthy humans to unveil the striatum involvement in habit learning. Sixteen patients and 20 age- and education-matched healthy control subjects learned sets of 3 visuo-motor associations between complex patterns and joystick displacements during two testing sessions separated by a few hours. We distinguished errors preceding vs. following the first correct response to compare patients' performance during the earliest phase of learning dominated by goal-directed actions with that observed later on, when responses start to become habitual. The disease significantly retarded both learning phases, especially in patients under 60 years of age. However, only the late phase deficit was disease severity-dependent and persisted on the second testing session. These findings provide the first corroboration in Parkinson patients of two ideas well-established in the animal literature. The first is the idea that associating visual stimuli to motor acts is a form of habit learning that engages the striatum. It is confirmed here by the global impairment in visuo-motor learning induced by PD. The second idea is that goal-directed behaviors are predominantly caudate-dependent whereas habitual responses are primarily putamen-dependent. At the advanced PD stages tested here, dopamine depletion is greater in the putamen than in the caudate nucleus. Accordingly, the late phase of learning corresponding to the emergence of habitual responses was more vulnerable to the disease than the early phase dominated by goal-directed actions.

An investigation of implicit memory through left temporal lobectomy for epilepsy

Temporal lobe epilepsy patients have demonstrated a relative preservation in the integrity of implicit memory procedures. We examined performance in a verbal implicit and explicit memory task in left anterior temporal lobectomy patients (LATL) and healthy normal controls (NCs) while undergoing fMRI. We hypothesized that despite the relative integrity of implicit memory in both the LATL patients and normal controls, the two groups would show distinct functional neuroanatomic profiles during implicit memory. LATLs and NCs performed Jacoby's Process Dissociation Process (PDP) procedure during fMRI, requiring completion of word stems based on the previously studied words or new/unseen words. Measures of automaticity and recollection provided uncontaminated indices of implicit and explicit memory, respectively. The behavioral data showed that in the face of temporal lobe pathology implicit memory can be carried out, suggesting implicit verbal memory retrieval is non-mesial temporal in nature. Compared to NCs, the LATL patients showed reliable activation, not deactivation, during implicit (automatic) responding. The regions mediating this response were cortical (left medial frontal and precuneus) and striatal. The active regions in LATL patients have the capacity to implement associative, conditioned responses that might otherwise be carried out by a healthy temporal lobe, suggesting this represented a compensatory activity. Because the precuneus has also been implicated in explicit memory, the data suggests this structure may have a highly flexible functionality, capable of supporting implementation of either explicit memory, or automatic processes such as implicit memory retrieval. Our data suggest that a healthy mesial/anterior temporal lobe may be needed for generating the posterior deactivation perceptual priming response seen in normals.

Multisynaptic projections from the ventrolateral prefrontal cortex to the dorsal premotor cortex in macaques - anatomical substrate for conditional visuomotor behavior

Lines of evidence indicate that both the ventrolateral prefrontal cortex (vlPFC) (areas 45/12) and dorsal premotor cortex (PMd) (rostral F2 in area 6) are crucially involved in conditional visuomotor behavior, in which it is required to determine an action based on an associated visual object. However, virtually no direct projections appear to exist between the vlPFC and PMd. In the present study, to elucidate possible multisynaptic networks linking the vlPFC to the PMd, we performed a series of neuroanatomical tract-tracing experiments in macaque monkeys. First, we identified cortical areas that send projection fibers directly to the PMd by injecting Fast Blue into the PMd. Considerable retrograde labeling occurred in the dorsal prefrontal cortex (dPFC) (areas 46d/9/8B/8Ad), dorsomedial motor cortex (dmMC) (F7 and presupplementary motor area), rostral cingulate motor area, and ventral premotor cortex (F5 and area 44), whereas the vlPFC was virtually devoid of neuronal labeling. Second, we injected the rabies virus, a retrograde transneuronal tracer, into the PMd. At 3 days after the rabies injections, second-order neurons were labeled in the vlPFC (mainly area 45), indicating that the vlPFC disynaptically projects to the PMd. Finally, to determine areas that connect the vlPFC to the PMd indirectly, we carried out an anterograde/retrograde dual-labeling experiment in single monkeys. By examining the distribution of axon terminals labeled from the vlPFC and cell bodies labeled from the PMd, we found overlapping labels in the dPFC and dmMC. These results indicate that the vlPFC outflow is directed toward the PMd in a multisynaptic fashion through the dPFC and/or dmMC.

Frontostriatal mechanisms in instruction-based learning as a hallmark of flexible goal-directed behavior

The present review intends to provide a neuroscientific perspective on the flexible (here: almost instantaneous) adoption of novel goal-directed behaviors. The overarching goal is to sketch the emerging framework for examining instruction-based learning and how this can be related to more established research approaches to instrumental learning and goal-directed action. We particularly focus on the contribution of frontal and striatal brain regions drawing on studies in both, animals and humans, but with an emphasize put on human neuroimaging studies. In section one, we review and integrate a selection of previous studies that are suited to generally delineate the neural underpinnings of goal-directed action as opposed to more stimulus-based (i.e., habitual) action. Building on that the second section focuses more directly on the flexibility to rapidly implement novel behavioral rules as a hallmark of goal-directed action with a special emphasis on instructed rules. In essence, the current neuroscientific evidence suggests that the prefrontal cortex and associative striatum are able to selectively and transiently code the currently relevant relationship between stimuli, actions, and the effects of these actions in both, instruction-based learning as well as in trial-and-error learning. The premotor cortex in turn seems to form more durable associations between stimuli and actions or stimuli, actions and effects (but not incentive values) thus representing the available action possibilities. Together, the central message of the present review is that instruction-based learning should be understood as a prime example of goal-directed action, necessitating a closer interlacing with basic mechanisms of goal-directed action on a more general level.

Neurons controlling voluntary vocalization in the macaque ventral premotor cortex

The voluntary control of phonation is a crucial achievement in the evolution of speech. In humans, ventral premotor cortex (PMv) and Broca's area are known to be involved in voluntary phonation. In contrast, no neurophysiological data are available about the role of the oro-facial sector of nonhuman primates PMv in this function. In order to address this issue, we recorded PMv neurons from two monkeys trained to emit coo-calls. Results showed that a population of motor neurons specifically fire during vocalization. About two thirds of them discharged before sound onset, while the remaining were time-locked with it. The response of vocalization-selective neurons was present only during conditioned (voluntary) but not spontaneous (emotional) sound emission. These data suggest that the control of vocal production exerted by PMv neurons constitutes a newly emerging property in the monkey lineage, shedding light on the evolution of phonation-based communication from a nonhuman primate species.

FMRI supports the sensorimotor theory of motor resonance

The neural mechanisms mediating the activation of the motor system during action observation, also known as motor resonance, are of major interest to the field of motor control. It has been proposed that motor resonance develops in infants through Hebbian plasticity of pathways connecting sensory and motor regions that fire simultaneously during imitation or self movement observation. A fundamental problem when testing this theory in adults is that most experimental paradigms involve actions that have been overpracticed throughout life. Here, we directly tested the sensorimotor theory of motor resonance by creating new visuomotor representations using abstract stimuli (motor symbols) and identifying the neural networks recruited through fMRI. We predicted that the network recruited during action observation and execution would overlap with that recruited during observation of new motor symbols. Our results indicate that a network consisting of premotor and posterior parietal cortex, the supplementary motor area, the inferior frontal gyrus and cerebellum was activated both by new motor symbols and by direct observation of the corresponding action. This tight spatial overlap underscores the importance of sensorimotor learning for motor resonance and further indicates that the physical characteristics of the perceived stimulus are irrelevant to the evoked response in the observer.

Rewiring the brain: potential role of the premotor cortex in motor control, learning, and recovery of function following brain injury

The brain is a plastic organ with a capability to reorganize in response to behavior and/or injury. Following injury to the motor cortex or emergent corticospinal pathways, recovery of function depends on the capacity of surviving anatomical resources to recover and repair in response to task-specific training. One such area implicated in poststroke reorganization to promote recovery of upper extremity recovery is the premotor cortex (PMC). This study reviews the role of distinct subdivisions of PMC: dorsal (PMd) and ventral (PMv) premotor cortices as critical anatomical and physiological nodes within the neural networks for the control and learning of goal-oriented reach and grasp actions in healthy individuals and individuals with stroke. Based on evidence emerging from studies of intrinsic and extrinsic connectivity, transcranial magnetic stimulation, functional neuroimaging, and experimental studies in animals and humans, the authors propose 2 distinct patterns of reorganization that differentially engage ipsilesional and contralesional PMC. Research directions that may offer further insights into the role of PMC in motor control, learning, and poststroke recovery are also proposed. This research may facilitate neuroplasticity for maximal recovery of function following brain injury.

Differential roles of caudate nucleus and putamen during instrumental learning

The dorsal striatum is crucial for the acquisition and consolidation of instrumental behaviour, but the underlying computations and internal dynamics remain elusive. To address this issue, we combined a model of key computations supporting decision-making during instrumental learning with human behavioural and functional magnetic resonance imaging (fMRI) data. The results showed that the associative and sensorimotor dorsal striatum host complementary computations that, we suggest, may differentially support goal-directed and habitual processes. The anterior caudate nucleus integrates information about performance and cognitive control demands, whereas the putamen tracks how likely the conditioning stimuli lead to correct response. Contrary to current models, the putamen is recruited during initial acquisition. As the exploratory phase proceeds, the relative contribution of the caudate nucleus becomes dominant over the putamen. During early consolidation, caudate nucleus and putamen settle to asymptotic values and share control. We then investigated how dorsal striatal computations may affect decision-making. We found that portion of reaction times' variance parallels the combined cost associated with the dorsal striatal computations. Overall, our findings provide a deeper insight into the functional heterogeneity within the dorsal striatum and suggest that the dynamic interplay between caudate nucleus and putamen, rather than their serial recruitment, underlies the acquisition and early consolidation of instrumental behaviours.

Striatal and medial temporal lobe functional interactions during visuomotor associative learning

A network of regions including the medial temporal lobe (MTL) and the striatum are integral to visuomotor associative learning. Here, we evaluated the contributions of the structures of the striatum and the MTL, as well as their interactions during an arbitrary associative learning task. We hypothesized that activity in the striatum would correlate with the rate of learning, while activity in the MTL would track how well associations were learned. Further, we expected functional correlations to show both facilitative as well as competitive relationships depending on the regions involved. Results showed that activity throughout the striatum was modulated by the rate of learning, while the sensorimotor and ventral striatum were also modulated by probability correct. Across the MTL, activity correlated with the probability of being correct, while the perirhinal cortex and right parahippocampal cortex were modulated by the rate of learning. The activity in the ventral striatum robustly coupled with activity in the MTL during learning, while interactions between the associative striatum and the MTL showed the opposite pattern. These findings suggest dissociable computational roles for different subregions of the striatum and MTL. These subregions interact in distinct ways, perhaps forming functionally integrated networks during the learning of arbitrary associations.

Neural mechanisms for interacting with a world full of action choices

The neural bases of behavior are often discussed in terms of perceptual, cognitive, and motor stages, defined within an information processing framework that was originally inspired by models of human abstract problem solving. Here, we review a growing body of neurophysiological data that is difficult to reconcile with this influential theoretical perspective. As an alternative foundation for interpreting neural data, we consider frameworks borrowed from ethology, which emphasize the kinds of real-time interactive behaviors that animals have engaged in for millions of years. In particular, we discuss an ethologically-inspired view of interactive behavior as simultaneous processes that specify potential motor actions and select between them. We review how recent neurophysiological data from diverse cortical and subcortical regions appear more compatible with this parallel view than with the classical view of serial information processing stages.

Dissociating the contributions of independent corticostriatal systems to visual categorization learning through the use of reinforcement learning modeling and Granger causality modeling

We dissociated the contributions to learning of four corticostriatal "loops" (interacting striatal and cortical regions): motor (putamen and motor cortex), visual (posterior caudate and visual cortex), executive (anterior caudate and prefrontal cortex), and motivational (ventral striatum and ventromedial frontal cortex). Subjects learned to categorize individual repeated images into one of two arbitrary categories via trial and error. We identified (1) regions sensitive to correct categorization, categorization learning, and feedback valence; (2) regions sensitive to prediction error (violation of feedback expectancy) and reward prediction (expected feedback associated with category response) using reinforcement learning modeling; and (3) directed influences between regions using Granger causality modeling. Each loop showed a unique pattern of sensitivity to each of these factors. Both the motor and visual loops were involved in acquisition of categorization ability: activity during correct categorization increased across learning and was sensitive to reward prediction. However, the posterior caudate received directed influence from visual cortex, whereas the putamen exerted directed influence on motor cortex. The motivational and executive loops were involved in feedback processing: both regions were sensitive to feedback valence, which interacted with learning across scans. However, the motivational loop activity reflected prediction error, whereas executive loop activity reflected reward prediction, consistent with the executive loop role in integrating reward and action. Granger causality modeling found directed influences between striatal and cortical regions within each loop. Across loops, the motor loop exerted directed influence on the executive loop which is consistent with the role of the executive loop in integrating feedback with stimulus-response history.

Processing of visual signals for direct specification of motor targets and for conceptual representation of action targets in the dorsal and ventral premotor cortex

Previous reports have indicated that the premotor cortex (PM) uses visual information for either direct guidance of limb movements or indirect specification of action targets at a conceptual level. We explored how visual inputs signaling these two different categories of information are processed by PM neurons. Monkeys performed a delayed reaching task after receiving two different sets of visual instructions, one directly specifying the spatial location of a motor target (a direct spatial-target cue) and the other providing abstract information about the spatial location of a motor target by indicating whether to select the right or left target at a conceptual level (a symbolic action-selection cue). By comparing visual responses of PM neurons to the two sets of visual cues, we found that the conceptual action plan indicated by the symbolic action-selection cue was represented predominantly in dorsal PM (PMd) neurons with a longer latency (150 ms), whereas both PMd and ventral PM (PMv) neurons responded with a shorter latency (90 ms) when the motor target was directly specified with the direct spatial-target cue. We also found that excited, but not inhibited, responses of PM neurons to the direct spatial-target cue were biased toward contralateral preference. In contrast, responses to the symbolic action-selection cue were either excited or inhibited without laterality preference. Taken together, these results suggest that the PM constitutes a pair of distinct circuits for visually guided motor act; one circuit, linked more strongly with PMd, carries information for retrieving action instruction associated with a symbolic cue, and the other circuit, linked with PMd and PMv, carries information for directly specifying a visuospatial position of a reach target.

On a basal ganglia role in learning and rehearsing visual-motor associations

Fronto-striatal circuitry interacts with the midbrain dopaminergic system to mediate the learning of stimulus-response associations, and these associations often guide everyday actions, but the precise role of these circuits in forming and consolidating rules remains uncertain. A means to examine basal ganglia circuit contributions to associative motor learning is to examine these process in a lesion model system, such as Parkinson's disease (PD), a basal ganglia disorder characterized by the loss of dopamine neurons. We used functional magnetic resonance imaging (MRI) to compare brain activation of PD patients with a group of healthy aged-match participants during a visual-motor associative learning task that entailed discovering and learning arbitrary associations between a set of six visual stimuli and corresponding spatial locations by moving a joystick-controlled cursor. We tested the hypothesis that PD would recruit more areas than age-matched controls during learning and also show increased activation in commonly activated regions, probably in the parietal and premotor cortices, and the cerebellum, perhaps as compensatory mechanisms for their disrupted fronto-striatal networks. PD had no effect in acquiring the associative relationships and learning-related activation in several key frontal cortical and subcortical structures. However, we found that PD modified activation in other areas, including those in the cerebellum and frontal, and parietal cortex, particularly during initial learning. These results may suggest that the basal ganglia circuits become active more so during the initial formation of rule-based behavior.

Neural basis of sensorimotor learning: modifying internal models

The neural basis of the internal models used in sensorimotor transformations is beginning to be uncovered. Sensorimotor learning involves the modification of such models. Different stages of sensory-motor processing have been explored with a continuum of experimental tasks, from learning arbitrary associations of sensory cues to movements, to adapting to altered kinematic and dynamic environments. Several groups have been studying changes in neuronal activity in cortical and subcortical areas that may be related to the acquisition and consolidation processes. We discuss the progress and challenges in understanding how these learning-related neural changes are involved in the modification of internal models, and offer future directions.

Neurodynamics of the prefrontal cortex during conditional visuomotor associations

The prefrontal cortex is believed to be important for cognitive control, working memory, and learning. It is known to play an important role in the learning and execution of conditional visuomotor associations, a cognitive task in which stimuli have to be associated with actions by trial-and-error learning. In our modeling study, we sought to integrate several hypotheses on the function of the prefrontal cortex using a computational model, and compare the results to experimental data. We constructed a module of prefrontal cortex neurons exposed to two different inputs, which we envision to originate from the inferotemporal cortex and the basal ganglia. We found that working memory properties do not describe the dominant dynamics in the prefrontal cortex, but the activation seems to be transient, probably progressing along a pathway from sensory to motor areas. During the presentation of the cue, the dynamics of the prefrontal cortex is bistable, yielding a distinct activation for correct and error trails. We find that a linear change in network parameters relates to the changes in neural activity in consecutive correct trials during learning, which is important evidence for the underlying learning mechanisms.

Delay-related cerebral activity and motor preparation

Flexible goal-oriented behavior requires the ability to carry information across temporal delays. This ability is associated with sustained neural firing. In cognitive terms, this ability has often been associated with the maintenance of sensory material online, as during short-term memory tasks, or with the retention of a motor code, as during movement preparation tasks. The general issue addressed in this paper is whether short-term storage of sensory information and preparation of motor responses rely on different anatomical substrates. We used functional magnetic resonance imaging (fMRI) to measure sustained and time-varying delay-related cerebral activity evoked during performance of a delay non-match to sample (DNMS) task, where task contingencies rather than explicit instructions ensured that either sensory or motor representations were used to cross the delay period on each trial. This approach allowed us to distinguish sensory from motor characteristics of delay-related activity evoked by task contingencies, rather than differences in the control of short-term storage driven by verbal instructions. Holding sensory material online evoked both sustained and time-varying delay-related activity in prefrontal regions, whereas movement preparation evoked delay-related responses in precentral areas. Intraparietal cortex was sensitive to the presence of memoranda, but indifferent to the type of information that was retained in memory. Our findings indicate that short-term storage of sensory information and preparation of motor responses rely on partially segregated cerebral circuits. In the frontal lobe, these circuits are organized along a rostro-caudal dimension, corresponding to the sensory or motor nature of the stored material.

Sensorimotor adaptation in Parkinson's disease: evidence for a dopamine dependent remapping disturbance

Sensorimotor adaptation is thought to involve a remapping of the kinematic and kinetic parameters associated with movements performed within a changing environment. Patients with Parkinson's disease (PD) are known to be affected on this type of learning process, although the specific role of dopamine depletion in these deficits has not yet been elucidated. The present study was an attempt to clarify whether dopamine depletion in PD may directly affect the capacity to internally reorganize the visuomotor remapping of a distorted environment. Fourteen PD patients were tested twice, while they were treated and while they were withdrawn from their regular levodopa treatment. Fourteen control subjects were also enrolled and tested twice. Two parallel forms of the Computed Mirror Pointing Task (CMPT), requiring making a reaching movement in a visually transformed environment (mirror inversion), were administered to each participant. Each of them had to perform 40 trials at each of the 2 testing sessions. At each trial, sensorimotor adaptation was evaluated by the initial direction angle (IDA), which reflects the direction of movement before any visually guided readjustment. Results revealed no IDA difference at baseline, between control subject and PD patients, whether they were treated or not. In all group, IDA values at that time were large, reflecting a tendency to make movements according to the real life visuomotor mapping (based on the natural direct vision). However, striking differences appeared during sensorimotor learning, in that IDA reduction along trials was poorer in patient not treated with levodopa than both control subjects and the same PD patient treated with levodopa. No difference was observed between the treated PD patients and control subjects. Given that IDA is thought to reflect the internal representation of the visuomotor mapping, it is concluded that dopamine depletion in PD would affects sensorimotor adaptation, in that it facilitates old and poorly adapted movements (real life mapping), instead of new and more adapted ones (mirror transformed mapping).

Dopamine replacement therapy does not restore the ability of Parkinsonian patients to make rapid adjustments in motor strategies according to changing sensorimotor contexts

The ability of dopamine replacement to restore rapid motor adjustments in Parkinson's disease (PD) was investigated. Medicated and non-medicated patients performed finger-to-nose movements while simultaneously bending the trunk forward, without vision. Trunk motion was blocked unexpectedly, necessitating rapid adjustments in arm trajectories. Patients exhibited irregular hand paths, plateaus in hand velocity, and prolonged movement times, which were significantly greater in perturbed trials. Medication improved kinematics but perturbation-induced disturbances persisted and did not approximate the levels of non-perturbed trials nor those of controls. Dopaminergic replenishment in PD may therefore have limited restorative benefits for rapid context-specific motor control.

Neural mechanisms for response selection: comparing selection of responses and items from working memory

Recent functional imaging studies of working memory (WM) have suggested a relationship between the requirement for response selection and activity in dorsolateral prefrontal (DLPFC) and parietal regions. Although a number of WM operations are likely to occur during response selection, the current study was particularly interested in the contribution of this neural network to WM-based response selection when compared to the selection of an item from a list being maintained in memory, during a verbal learning task. The design manipulated stimulus-response mappings so that selecting an item from memory was not always accompanied with selecting a motor response. Functional activation during selection supported previous findings of fronto-parietal involvement, although in contrast to previous findings left, rather than right, DLPFC activity was significantly more active for selecting a memory-guided motor response, when compared to selecting an item currently maintained in memory or executing a memory-guided response. Our results contribute to the debate over the role of fronto-parietal activity during WM tasks, suggesting that this activity appears particularly related to response selection, potentially supporting the hypothesized role of prefrontal activity in biasing attention toward task-relevant material in more posterior regions.