Encoding, preparation and implementation of novel complex verbal instructions

Novel Verbal Instructions Recruit Abstract Neural Patterns of Time-Variable Information Dimensionality

Human performance is endowed by neural representations of information that is relevant for behavior, some of which are also activated in a preparatory fashion to optimize later execution. Most studies to date have focused on highly practiced actions, leaving largely unaddressed the novel reconfiguration of information to generate unique whole task sets. Using electroencephalography, this study investigated the dynamics of the content and geometry reflected on the neural patterns of control representations during reconfiguration of information. We designed a verbal instruction paradigm where each trial involved novel combinations of multicomponent task information. By manipulating three task-relevant factors in a sample of 40 participants (26 females, 14 males), we observed complex coding schemes throughout the trial, during both preparation and implementation stages. The temporal profiles were consistent with a hierarchical structure: whereas task information was active in a sustained manner, the coding of more concrete stimulus features was more transient. Data showed both high dimensionality and abstraction, particularly during instruction encoding and target processing. Our results suggest that whenever task content could be recovered from neural patterns of activity, there was evidence of abstract coding, with an underlying geometry that favored generalization. During target processing, where potential interference across stimulus and response factors increased, orthogonal configurations also appeared. Overall, our findings uncover the dynamic manner with which control representations operate during novel recombination unique scenarios, with changes in dimensionality and abstraction adjusting along processing stages.

Concurrent response and action effect representations across the somatomotor cortices during novel task preparation

Instructions allow us to fulfill novel and complex tasks on the first try. This skill has been linked to preparatory brain signals that encode upcoming demands in advance, facilitating novel performance. To deepen insight into these processes, we explored whether instructions pre-activated task-relevant motoric and perceptual neural states. Critically, we addressed whether these representations anticipated activity patterns guiding overt sensorimotor processing, which could reflect that internally simulating novel tasks facilitates the preparation. To do so, we collected functional magnetic resonance imaging data while female and male participants encoded and implemented novel stimulus-response associations. Participants also completed localizer tasks designed to isolate the neural representations of the mappings-relevant motor responses, perceptual consequences, and stimulus categories. Using canonical template tracking, we identified whether and where these sensorimotor representations were pre-activated. We found that response-related templates were encoded in advance in regions linked with action control, entailing not only the instructed responses but also their somatosensory consequences. This result was particularly robust in primary motor and somatosensory cortices. While, following our predictions, we found a systematic decrease in the irrelevant stimulus templates' representational strength compared to the relevant ones, this difference was due to below-zero estimates linked to the irrelevant category activity patterns. Overall, our findings reflect that instruction processing relies on the sensorimotor cortices to anticipate motoric and kinesthetic representations of prospective action plans, suggesting the engagement of motor imagery during novel task preparation. More generally, they stress that the somatomotor system could participate with higher-level frontoparietal regions during anticipatory task control.

Proactive selective attention across competition contexts

Selective attention is a cognitive function that helps filter out unwanted information. Theories such as the biased competition model (Desimone & Duncan, 1995) explain how attentional templates bias processing towards targets in contexts where multiple stimuli compete for resources. However, it is unclear how the anticipation of different levels of competition influences the nature of attentional templates, in a proactive fashion. In this study, we used electroencephalography (EEG) to investigate how the anticipated demands of attentional selection (either high or low stimuli competition contexts) modulate target-specific preparatory brain activity and its relationship with task performance. To do so, participants performed a sex/gender judgment task in a cue-target paradigm where, depending on the block, target and distractor stimuli appeared simultaneously (high competition) or sequentially (low competition). Multivariate Pattern Analysis (MVPA) showed that, in both competition contexts, there was a preactivation of the target category to select, with a ramping-up profile at the end of the preparatory interval. However, cross-classification showed no generalization across competition conditions, suggesting different preparatory formats. Notably, time-frequency analyses showed differences between anticipated competition demands, with higher theta band power for high than low competition, which mediated the impact of subsequent stimuli competition on behavioral performance. Overall, our results show that, whereas preactivation of the internal templates associated with the category to select are engaged in advance in high and low competition contexts, their underlying neural patterns differ. In addition, these codes could not be associated with theta power, suggesting that they reflect different preparatory processes. The implications of these findings are crucial to increase our understanding of the nature of top-down processes across different contexts.

Theta-phase connectivity between medial prefrontal and posterior areas underlies novel instructions implementation

Implementing novel instructions is a complex and uniquely human cognitive ability, that requires the rapid and flexible conversion of symbolic content into a format that enables the execution of the instructed behavior. Preparing to implement novel instructions, as opposed to their mere maintenance, involves the activation of the instructed motor plans, and the binding of the action information to the specific context in which this should be executed. Recent evidence and prominent computational models suggest that this efficient configuration of the system might involve a central role of frontal theta oscillations in establishing top-down long-range synchronization between distant and task-relevant brain areas. In the present EEG study (human subjects, 30 females, 4 males), we demonstrate that proactively preparing for the implementation of novels instructions, as opposed to their maintenance, involves a strengthened degree of connectivity in the theta frequency range between medial prefrontal and motor/visual areas. Moreover, we replicated previous results showing oscillatory features associated specifically with implementation demands, and extended on them demonstrating the role of theta oscillations in mediating the effect of task demands on behavioral performance. Taken together, these findings support our hypothesis that the modulation of connectivity patterns between frontal and task-relevant posterior brain areas is a core factor in the emergence of a behavior-guiding format from novel instructions.Significance statementEveryday life requires the use and manipulation of currently available information to guide behavior and reach specific goals. In the present study we investigate how the same instructed content elicits different neural activity depending on the task being performed. Crucially, connectivity between medial prefrontal cortex and posterior brain areas is strengthened when novel instructions have to be implemented, rather than simply maintained. This finding suggests that theta oscillations play a role in setting up a dynamic and flexible network of task-relevant regions optimized for the execution of the instructed behavior.

Exploring the Link between Novel Task Proceduralization and Motor Simulation

Our ability to generate efficient behavior from novel instructions is critical for our adaptation to changing environments. Despite the absence of previous experience, novel instructed content is quickly encoded into an action-based or procedural format, facilitating automatic task processing. In the current work, we investigated the link between proceduralization and motor simulation, specifically, whether the covert activation of the task-relevant responses is used during the assembly of action-based instructions representations. Across three online experiments, we used a concurrent finger-tapping task to block motor simulation during the encoding of novel stimulus-response (S-R) associations. The overlap between the mappings and the motor task at the response level was manipulated. We predicted a greater impairment at mapping implementation in the overlapping condition, where the mappings' relevant response representations were already loaded by the motor demands, and thus, could not be used in the upcoming task simulation. This hypothesis was robustly supported by the three datasets. Nonetheless, the overlapping effect was not modulated by further manipulations of proceduralization-related variables (preparation demands in Exp.2, mapping novelty in Exp.3). Importantly, a fourth control experiment ruled out that our results were driven by alternative accounts as fatigue or negative priming. Overall, we provided strong evidence towards the involvement of motor simulation during anticipatory task reconfiguration. However, this involvement was rather general, and not restricted to novelty scenarios. Finally, these findings can be also integrated into broader models of anticipatory task control, stressing the role of the motor system during preparation.

Neural oscillations track the maintenance and proceduralization of novel instructions

Humans are capable of flexibly converting symbolic instructions into novel behaviors. Previous evidence and theoretical models suggest that the implementation of a novel instruction requires the reformatting of its declarative content into an action-oriented code optimized for the execution of the instructed behavior. While neuroimaging research focused on identifying the brain areas involved in such a process, the temporal and electrophysiological mechanisms remain poorly understood. These mechanisms, however, can provide information about the specific cognitive processes that characterize the proceduralization of information. In the present study, we recorded EEG activity while we asked participants to either simply maintain declaratively the content of novel S-R mappings or to proactively prepare for their implementation. By means of time-frequency analyses, we isolated the oscillatory features specific to the proceduralization of instructions. Implementation of the instructed mappings elicited stronger theta activity over frontal electrodes and suppression in mu and beta activity over central electrodes. On the contrary, activity in the alpha band, which has been shown to track the attentional deployment to task-relevant items, showed no differences between tasks. Together, these results support the idea that proceduralization of information is characterized by specific component processes such as orchestrating complex task settings and configuring the motor system that are not observed when instructions are held in a declarative format.

Frontoparietal action-oriented codes support novel instruction implementation

A key aspect of human cognitive flexibility concerns the ability to convert complex symbolic instructions into novel behaviors. Previous research proposes that this transformation is supported by two neurocognitive states: an initial declarative maintenance of task knowledge, and an implementation state necessary for optimal task execution. Furthermore, current models predict a crucial role of frontal and parietal brain regions in this process. However, whether declarative and procedural signals independently contribute to implementation remains unknown. We report the results of an fMRI experiment in which participants executed novel instructed stimulus-response associations. We then used a pattern-tracking procedure to quantify the contribution of format-unique signals during instruction implementation. This revealed independent procedural and declarative representations of novel S-Rs in frontoparietal areas, prior to execution. Critically, the degree of procedural activation predicted subsequent behavioral performance. Altogether, our results suggest an important contribution of frontoparietal regions to the neural architecture that regulates cognitive flexibility.

Neural representations of social valence bias economic interpersonal choices

Prior personal information is highly relevant during social interactions. Such knowledge aids in the prediction of others, and it affects choices even when it is unrelated to actual behaviour. In this investigation, we aimed to study the neural representation of positive and negative personal expectations, how these impact subsequent choices, and the effect of mismatches between expectations and encountered behaviour. We employed functional Magnetic Resonance Imaging in combination with a version of the Ultimatum Game (UG) where participants were provided with information about their partners' moral traits previous to receiving their fair or unfair offers. Univariate and multivariate analyses revealed the implication of the supplementary motor area (SMA) and inferior frontal gyrus (IFG) in the representation of expectations about the partners in the game. Further, these regions also represented the valence of these expectations, together with the ventromedial prefrontal cortex (vmPFC). Importantly, the performance of multivariate classifiers in these clusters correlated with a behavioural choice bias to accept more offers following positive descriptions, highlighting the impact of the valence of the expectations on participants' economic decisions. Altogether, our results suggest that expectations based on social information guide future interpersonal decisions and that the neural representation of such expectations in the vmPFC is related to their influence on behaviour.

Investigating the effect of trustworthiness on instruction-based reflexivity

Unlike other species, humans are capable of rapidly learning new behavior from a single instruction. While previous research focused on the cognitive processes underlying the rapid, automatic implementation of instructions, the fundamentally social nature of instruction following has remained largely unexplored. Here, we investigated whether instructor trustworthiness modulates instruction implementation using both explicit and reflexive measures. In a first preregistered study, we validated a new paradigm to manipulate the perceived trustworthiness of two different virtual characters and showed that such a manipulation reliably induced implicit associations between the virtual characters and trustworthiness attributes. Moreover, we show that trustworthy instructors are followed more frequently and faster. In two additional preregistered experiments, we tested if trustworthiness towards the instructor influenced the cognitive processes underlying instruction implementation. While we show that verbally conveyed instructions led to automatic instruction implementation, this effect was not modulated by the trustworthiness of the instructor. Thus, we succeeded to design and validate a novel trustworthiness manipulation (Experiment 1) and to create a social variant of the instruction-based reflexivity paradigm (Experiments 2 and 3). However, this instruction-based reflexivity effect was not modulated by the instructors' trustworthiness.

Neural representation of newly instructed rule identities during early implementation trials

By following explicit instructions, humans instantaneously get the hang of tasks they have never performed before. We used a specially calibrated multivariate analysis technique to uncover the elusive representational states during the first few implementations of arbitrary rules such as ‘for coffee, press red button’ following their first-time instruction. Distributed activity patterns within the ventrolateral prefrontal cortex (VLPFC) indicated the presence of neural representations specific of individual stimulus-response (S-R) rule identities, preferentially for conditions requiring the memorization of instructed S-R rules for correct performance. Identity-specific representations were detectable starting from the first implementation trial and continued to be present across early implementation trials. The increasingly fluent application of novel rule representations was channelled through increasing cooperation between VLPFC and anterior striatum. These findings inform representational theories on how the prefrontal cortex supports behavioral flexibility specifically by enabling the ad-hoc coding of newly instructed individual rule identities during their first-time implementation.

Representational Organization of Novel Task Sets during Proactive Encoding

Recent multivariate analyses of brain data have boosted our understanding of the organizational principles that shape neural coding. However, most of this progress has focused on perceptual visual regions (Connolly et al., 2012), whereas far less is known about the organization of more abstract, action-oriented representations. In this study, we focused on humans' remarkable ability to turn novel instructions into actions. While previous research shows that instruction encoding is tightly linked to proactive activations in frontoparietal brain regions, little is known about the structure that orchestrates such anticipatory representation. We collected fMRI data while participants (both males and females) followed novel complex verbal rules that varied across control-related variables (integrating within/across stimuli dimensions, response complexity, target category) and reward expectations. Using representational similarity analysis (Kriegeskorte et al., 2008), we explored where in the brain these variables explained the organization of novel task encoding, and whether motivation modulated these representational spaces. Instruction representations in the lateral PFC were structured by the three control-related variables, whereas intraparietal sulcus encoded response complexity and the fusiform gyrus and precuneus organized its activity according to the relevant stimulus category. Reward exerted a general effect, increasing the representational similarity among different instructions, which was robustly correlated with behavioral improvements. Overall, our results highlight the flexibility of proactive task encoding, governed by distinct representational organizations in specific brain regions. They also stress the variability of motivation-control interactions, which appear to be highly dependent on task attributes, such as complexity or novelty.SIGNIFICANCE STATEMENT In comparison with other primates, humans display a remarkable success in novel task contexts thanks to our ability to transform instructions into effective actions. This skill is associated with proactive task-set reconfigurations in frontoparietal cortices. It remains yet unknown, however, how the brain encodes in anticipation the flexible, rich repertoire of novel tasks that we can achieve. Here we explored cognitive control and motivation-related variables that might orchestrate the representational space for novel instructions. Our results showed that different dimensions become relevant for task prospective encoding, depending on the brain region, and that the lateral PFC simultaneously organized task representations following different control-related variables. Motivation exerted a general modulation upon this process, diminishing rather than increasing distances among instruction representations.

Neural representation of current and intended task sets during sequential judgements on human faces

Engaging in a demanding activity while holding in mind another task to be performed in the near future requires the maintenance of information about both the currently-active task set and the intended one. However, little is known about how the human brain implements such action plans. While some previous studies have examined the neural representation of current task sets and others have investigated delayed intentions, to date none has examined the representation of current and intended task sets within a single experimental paradigm. In this fMRI study, we examined the neural representation of current and intended task sets, employing sequential classification tasks on human faces. Multivariate decoding analyses showed that current task sets were represented in the orbitofrontal cortex (OFC) and fusiform gyrus (FG), while intended tasks could be decoded from lateral prefrontal cortex (lPFC). Importantly, a ventromedial region in PFC/OFC contained information about both current and delayed tasks, although cross-classification between the two types of information was not possible. These results help delineate the neural representations of current and intended task sets, and highlight the importance of ventromedial PFC/OFC for maintaining task-relevant information regardless of when it is needed.

Atlas-Based Classification Algorithms for Identification of Informative Brain Regions in fMRI Data

Multi-voxel pattern analysis (MVPA) has been successfully applied to neuroimaging data due to its larger sensitivity compared to univariate traditional techniques. Searchlight is the most widely employed approach to assign functional value to different regions of the brain. However, its performance depends on the size of the sphere, which can overestimate the region of activation when a large sphere size is employed. In the current study, we examined the validity of two different alternatives to Searchlight: an atlas-based local averaging method (ABLA, Schrouff et al. Neuroinformatics 16, 117-143, 2013a) and a Multi-Kernel Learning (MKL, Rakotomamonjy et al. Journal of Machine Learning 9, 2491-2521, 2008) approach, in a scenario where the goal is to find the informative brain regions that support certain mental operations. These methods employ weights to measure the informativeness of a brain region and highly reduce the large computational cost that Searchlight entails. We evaluated their performance in two different scenarios where the differential BOLD activation between experimental conditions was large vs. small, and employed nine different atlases to assess the influence of diverse brain parcellations. Results show that both methods were able to localize informative regions when differences between conditions were large, demonstrating a large sensitivity and stability in the identification of regions across atlases. Moreover, the sign of the weights reported by these methods provided the directionality of univariate approaches. However, when differences were small, only ABLA localized informative regions. Thus, our results show that atlas-based methods are useful alternatives to Searchlight, but that the nature of the classification to perform should be taken into account when choosing the specific method to implement.

Transient and Sustained Control Mechanisms Supporting Novel Instructed Behavior

The success of humans in novel environments is partially supported by our ability to implement new task procedures via instructions. This complex skill has been associated with the activity of control-related brain areas. Current models link fronto-parietal and cingulo-opercular networks with transient and sustained modes of cognitive control, based on observations during repetitive task settings or rest. The current study extends this dual model to novel instructed tasks. We employed a mixed design and an instruction-following task to extract phasic and tonic brain signals associated with the encoding and implementation of novel verbal rules. We also performed a representation similarity analysis to capture consistency in task-set encoding within trial epochs. Our findings show that both networks are involved while following novel instructions: transiently, during the implementation of the instruction, and in a sustained fashion, across novel trials blocks. Moreover, the multivariate results showed that task representations in the cingulo-opercular network were more stable than in the fronto-parietal one. Our data extend the dual model of cognitive control to novel demanding situations, highlighting the high flexibility of control-related regions in adopting different temporal profiles.

Influence of activation pattern estimates and statistical significance tests in fMRI decoding analysis

The use of Multi-Voxel Pattern Analysis (MVPA) has increased considerably in recent functional magnetic resonance imaging (fMRI) studies. A crucial step consists in the choice of a method for the estimation of responses. However, a systematic comparison of the different estimation alternatives and their adequacy to predominant experimental design is missing. In the current study we compared three pattern estimation methods: Least-Squares Unitary (LSU), based on run-wise estimation, Least-Squares All (LSA) and Least-Squares Separate (LSS), which rely on trial-wise estimation. We compared the efficiency of these methods in an experiment where sustained activity needed to be isolated from zero-duration events as well as in a block-design approach and in a event-related design. We evaluated the sensitivity of the t-test in comparison with two non-parametric methods based on permutation testing: one proposed in Stelzer et al. (2013), equivalent to performing a permutation in each voxel separately and the Threshold-Free Cluster Enhancement. LSS resulted the most accurate approach to address the large overlap of signal among close events in the event-related designs. We found a larger sensitivity of Stelzer's method in all settings, especially in the event-related designs, where voxels close to surpass the statistical threshold with the other approaches were now marked as informative regions. Our results provide evidence that LSS is the most accurate approach for unmixing events with different duration and large overlap of signal. This is consistent with previous studies showing that LSS handles large collinearity better than other methods. Moreover, Stelzer's potentiates this better estimation with its large sensitivity.