Towards an understanding of the neural dynamics of intentional learning: Considering the timescale

The connected learning brain

This paper extends a recent study on the neural mechanisms underlying initial learning through instruction, trial-and-error, and observation of stimulus-response associations. Adopting a network perspective, we examine the functional connectivity patterns during the early stages of learning, demonstrating that the brain undergoes extensive network reorganization, regardless of the acquisition method. Our findings reveal a general segregation of task-positive networks from the default mode network, which is paralleled by and may facilitate the integration within and between task-positive networks. This segregation-integration pattern likely reflects a balance between internal and external task-related processes, modulated by learning progression and task difficulty across different acquisition modes. Differences between learning conditions, as well as brain connectivity-behavior associations between rule learning and rule implementation, point to varying cognitive demands: more efficient learning in instruction-based learning, inhibitory processes in observation-based learning, and the integration of reward, valence, and somatomotor processes in trial-and-error learning. We conclude that while extensive neural reorganization occurs during the initial learning trials, irrespective of response implementation or acquisition mode, this reorganization also exhibits distinct features that support the unique demands of each learning method.

The dynamics of functional brain network segregation in feedback-driven learning

Prior evidence suggests that increasingly efficient task performance in human learning is associated with large scale brain network dynamics. However, the specific nature of this general relationship has remained unclear. Here, we characterize performance improvement during feedback-driven stimulus-response (S-R) learning by learning rate as well as S-R habit strength and test whether and how these two behavioral measures are associated with a functional brain state transition from a more integrated to a more segregated brain state across learning. Capitalizing on two separate fMRI studies using similar but not identical experimental designs, we demonstrate for both studies that a higher learning rate is associated with a more rapid brain network segregation. By contrast, S-R habit strength is not reliably related to changes in brain network segregation. Overall, our current study results highlight the utility of dynamic functional brain state analysis. From a broader perspective taking into account previous study results, our findings align with a framework that conceptualizes brain network segregation as a general feature of processing efficiency not only in feedback-driven learning as in the present study but also in other types of learning and in other task domains.

Stimulating parietal regions of the multiple-demand cortex impairs novel vocabulary learning

Neuroimaging research demonstrated that the early stages of learning engage domain-general networks, non-specialist brain regions that process a wide variety of cognitive tasks. Those networks gradually disengage as learning progresses and learned information becomes processed in brain networks specialised for the specific function (e.g., language). In the current study, we used repetitive transcranial magnetic stimulation (rTMS) in the form of continuous theta burst stimulation (cTBS) to test whether stimulation of the bilateral parietal region of the domain-general network impairs learning new vocabulary, indicating its causal engagement in this process. Twenty participants, with no prior knowledge of Polish, learned Polish words for well-known objects across three training stages. The first training stage started with cTBS applied to either the experimental domain-general bilateral parietal site or the control bilateral precentral site. Immediately after cTBS, the vocabulary training commenced. A different set of words was learned for each site. Immediately after the training stage, participants performed a novel vocabulary test, designed to measure their knowledge of the new words and the effect of stimulation on learning. To measure stimulation effect when the words were more established in the mental lexicon, participants received additional training on the same words but without cTBS (second training stage) and then the full procedures from the first training stage were repeated (third training stage). Results demonstrated that stimulation impaired novel word learning when applied to the bilateral parietal site at the first stage of learning only. This effect was not present when newly learned words were used more proficiently in the third training stage, or at any learning stage during control site stimulation. Our results show that the bilateral parietal region of the domain-general network causally contributes to the successful learning of novel words.

Disrupted Salience and Cingulo-Opercular Network Connectivity During Impaired Rapid Instructed Task Learning in Schizophrenia

Rapid instructed task learning (RITL) is the uniquely human ability to transform task information into goal-directed behavior without relying on trial-and-error learning. RITL is a core cognitive process supported by functional brain networks. In patients with schizophrenia, RITL ability is impaired, but the role of functional network connectivity in these RITL deficits is unknown. We investigated task-based connectivity of eight a priori network pairs in participants with schizophrenia (n = 29) and control participants (n = 31) during the performance of an RITL task. Multivariate pattern analysis was used to determine which network connectivity patterns predicted diagnostic group. Of all network pairs, only the connectivity between the cingulo-opercular network (CON) and salience network (SAN) during learning classified patients and control participants with significant accuracy (80%). CON-SAN connectivity during learning was significantly associated with task performance in participants with schizophrenia. These findings suggest that impaired interactions between identification of salient stimuli and maintenance of task goals contributes to RITL deficits in participants with schizophrenia.

Neural representation of newly instructed rule identities during early implementation trials

By following explicit instructions, humans instantaneously get the hang of tasks they have never performed before. We used a specially calibrated multivariate analysis technique to uncover the elusive representational states during the first few implementations of arbitrary rules such as ‘for coffee, press red button’ following their first-time instruction. Distributed activity patterns within the ventrolateral prefrontal cortex (VLPFC) indicated the presence of neural representations specific of individual stimulus-response (S-R) rule identities, preferentially for conditions requiring the memorization of instructed S-R rules for correct performance. Identity-specific representations were detectable starting from the first implementation trial and continued to be present across early implementation trials. The increasingly fluent application of novel rule representations was channelled through increasing cooperation between VLPFC and anterior striatum. These findings inform representational theories on how the prefrontal cortex supports behavioral flexibility specifically by enabling the ad-hoc coding of newly instructed individual rule identities during their first-time implementation.

Probing cortical and sub-cortical contributions to instruction-based learning: Regional specialisation and global network dynamics

Diverse cortical networks and striatal brain regions are implicated in instruction-based learning (IBL); however, their distinct contributions remain unclear. We use a modified fMRI paradigm to test two hypotheses regarding the brain mechanisms that underlie IBL. One hypothesis proposes that anterior caudate and frontoparietal regions transiently co-activate when new rules are being bound in working memory. The other proposes that they mediate the application of the rules at different stages of the consolidation process. In accordance with the former hypothesis, we report strong activation peaks within and increased connectivity between anterior caudate and frontoparietal regions when rule-instruction slides are presented. However, similar effects occur throughout a broader set of cortical and sub-cortical regions, indicating a metabolically costly reconfiguration of the global brain state. The distinct functional roles of cingulo-opercular, frontoparietal and default-mode networks are apparent from their activation throughout, early and late in the practice phase respectively. Furthermore, there is tentative evidence of a peak in anterior caudate activity mid-way through the practice stage. These results demonstrate how performance of the same simple task involves a steadily shifting balance of brain systems as learning progresses. They also highlight the importance of distinguishing between regional specialisation and global dynamics when studying the network mechanisms that underlie cognition and learning.

Deterministic response strategies in a trial-and-error learning task

Trial-and-error learning is a universal strategy for establishing which actions are beneficial or harmful in new environments. However, learning stimulus-response associations solely via trial-and-error is often suboptimal, as in many settings dependencies among stimuli and responses can be exploited to increase learning efficiency. Previous studies have shown that in settings featuring such dependencies, humans typically engage high-level cognitive processes and employ advanced learning strategies to improve their learning efficiency. Here we analyze in detail the initial learning phase of a sample of human subjects (N = 85) performing a trial-and-error learning task with deterministic feedback and hidden stimulus-response dependencies. Using computational modeling, we find that the standard Q-learning model cannot sufficiently explain human learning strategies in this setting. Instead, newly introduced deterministic response models, which are theoretically optimal and transform stimulus sequences unambiguously into response sequences, provide the best explanation for 50.6% of the subjects. Most of the remaining subjects either show a tendency towards generic optimal learning (21.2%) or at least partially exploit stimulus-response dependencies (22.3%), while a few subjects (5.9%) show no clear preference for any of the employed models. After the initial learning phase, asymptotic learning performance during the subsequent practice phase is best explained by the standard Q-learning model. Our results show that human learning strategies in the presented trial-and-error learning task go beyond merely associating stimuli and responses via incremental reinforcement. Specifically during initial learning, high-level cognitive processes support sophisticated learning strategies that increase learning efficiency while keeping memory demands and computational efforts bounded. The good asymptotic fit of the Q-learning model indicates that these cognitive processes are successively replaced by the formation of stimulus-response associations over the course of learning.

On the efficiency of instruction-based rule encoding

Instructions have long been considered a highly efficient route to knowledge acquisition especially compared to trial-and-error learning. We aimed at substantiating this claim by identifying boundary conditions for such an efficiency gain, including the influence of active learning intention, repeated instructions, and working memory load and span. Our experimental design allowed us to not only assess how well the instructed stimulus-response (S-R) rules were implemented later on, but also to directly measure prior instruction encoding processes. This revealed that instruction encoding was boosted by an active learning intention which in turn entailed better subsequent rule implementation. As should be expected, instruction-based learning took fewer trials than trial-and-error learning to reach a similar performance level. But more importantly, even when performance was measured relative to the identical number of preceding correct implementation trials, this efficiency gain persisted both in accuracy and in speed. This suggests that the naturally greater number of failed attempts in the initial phase of trial-and-error learning also negatively impacted learning in subsequent trials due to the persistence of erroneous memory traces established beforehand. A single instruction trial was sufficient to establish the advantage over trial-and-error learning but repeated instructions were better. Strategic factors and inter-individual differences in WM span - the latter exclusively affecting trial-and-error learning presumably due to the considerably more demanding working memory operations - could reduce or even abolish this advantage, but only in error rates. The same was not true for response time gains suggesting generally more efficient task automatization in instruction-based learning.

When global rule reversal meets local task switching: The neural mechanisms of coordinated behavioral adaptation to instructed multi-level demand changes

Cognitive flexibility is essential to cope with changing task demands and often it is necessary to adapt to combined changes in a coordinated manner. The present fMRI study examined how the brain implements such multi-level adaptation processes. Specifically, on a "local," hierarchically lower level, switching between two tasks was required across trials while the rules of each task remained unchanged for blocks of trials. On a "global" level regarding blocks of twelve trials, the task rules could reverse or remain the same. The current task was cued at the start of each trial while the current task rules were instructed before the start of a new block. We found that partly overlapping and partly segregated neural networks play different roles when coping with the combination of global rule reversal and local task switching. The fronto-parietal control network (FPN) supported the encoding of reversed rules at the time of explicit rule instruction. The same regions subsequently supported local task switching processes during actual implementation trials, irrespective of rule reversal condition. By contrast, a cortico-striatal network (CSN) including supplementary motor area and putamen was increasingly engaged across implementation trials and more so for rule reversal than for nonreversal blocks, irrespective of task switching condition. Together, these findings suggest that the brain accomplishes the coordinated adaptation to multi-level demand changes by distributing processing resources either across time (FPN for reversed rule encoding and later for task switching) or across regions (CSN for reversed rule implementation and FPN for concurrent task switching).

Stimulating Multiple-Demand Cortex Enhances Vocabulary Learning

It is well established that networks within multiple-demand cortex (MDC) become active when diverse skills and behaviors are being learnt. However, their causal role in learning remains to be established. In the present study, we first performed functional magnetic resonance imaging on healthy female and male human participants to confirm that MDC was most active in the initial stages of learning a novel vocabulary, consisting of pronounceable nonwords (pseudowords), each associated with a picture of a real object. We then examined, in healthy female and male human participants, whether repetitive transcranial magnetic stimulation of a frontal midline node of the cingulo-opercular MDC affected learning rates specifically during the initial stages of learning. We report that stimulation of this node, but not a control brain region, substantially improved both accuracy and response times during the earliest stage of learning pseudoword–object associations. This stimulation had no effect on the processing of established vocabulary, tested by the accuracy and response times when participants decided whether a real word was accurately paired with a picture of an object. These results provide evidence that noninvasive stimulation to MDC nodes can enhance learning rates, thereby demonstrating their causal role in the learning process. We propose that this causal role makes MDC candidate target for experimental therapeutics; for example, in stroke patients with aphasia attempting to reacquire a vocabulary.

SIGNIFICANCE STATEMENT Learning a task involves the brain system within which that specific task becomes established. Therefore, successfully learning a new vocabulary establishes the novel words in the language system. However, there is evidence that in the early stages of learning, networks within multiple-demand cortex (MDC), which control higher cognitive functions, such as working memory, attention, and monitoring of performance, become active. This activity declines once the task is learnt. The present study demonstrated that a node within MDC, located in midline frontal cortex, becomes active during the early stage of learning a novel vocabulary. Importantly, noninvasive brain stimulation of this node improved performance during this stage of learning. This observation demonstrated that MDC activity is important for learning.

Integration and segregation of large-scale brain networks during short-term task automatization

The human brain is organized into large-scale functional networks that can flexibly reconfigure their connectivity patterns, supporting both rapid adaptive control and long-term learning processes. However, it has remained unclear how short-term network dynamics support the rapid transformation of instructions into fluent behaviour. Comparing fMRI data of a learning sample (N=70) with a control sample (N=67), we find that increasingly efficient task processing during short-term practice is associated with a reorganization of large-scale network interactions. Practice-related efficiency gains are facilitated by enhanced coupling between the cingulo-opercular network and the dorsal attention network. Simultaneously, short-term task automatization is accompanied by decreasing activation of the fronto-parietal network, indicating a release of high-level cognitive control, and a segregation of the default mode network from task-related networks. These findings suggest that short-term task automatization is enabled by the brain's ability to rapidly reconfigure its large-scale network organization involving complementary integration and segregation processes.

Humans can quickly learn to efficiently execute tasks yet how the brain activity is dynamically reconfigured during this process remains unknown. Here the authors demonstrate that large-scale functional brain networks are reorganized flexibly to support rapid task automation.