Incidental covariation learning leading to strategy change

Representational spaces in orbitofrontal and ventromedial prefrontal cortex: task states, values, and beyond

The orbitofrontal cortex (OFC) and ventromedial-prefrontal cortex (vmPFC) play a key role in decision-making and encode task states in addition to expected value. We review evidence suggesting a connection between value and state representations and argue that OFC / vmPFC integrate stimulus, context, and outcome information. Comparable encoding principles emerge in late layers of deep reinforcement learning (RL) models, where single nodes exhibit similar forms of mixed-selectivity, which enables flexible readout of relevant variables by downstream neurons. Based on these lines of evidence, we suggest that outcome-maximization leads to complex representational spaces that are insufficiently characterized by linear value signals that have been the focus of most prior research on the topic. Major outstanding questions concern the role of OFC/ vmPFC in learning across tasks, in encoding of task-irrelevant aspects, and the role of hippocampus-PFC interactions.

Abrupt and spontaneous strategy switches emerge in simple regularised neural networks

Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, and that behaviour is marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate "silent knowledge" that is initially suppressed by regularised gating. This suggests that insight-like behaviour can arise from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation. These results have potential implications for more complex systems, such as the brain, and guide the way for future insight research.

Search and insight processes in card sorting games

Insight problems are particularly interesting, because problems which require restructuring allow researchers to investigate the underpinnings of the Aha-experience, creativity and out of the box thinking. There is a need for new insight tasks to probe and extend the limits of existing theories and cognitive frameworks. To shed more light on this fascinating issue, we addressed the question: Is it possible to convey a well-known card sorting game into an insight task? We introduced different conditions and tested them via two online experiments (N = 546). Between the conditions we systematically varied the available perceptual features, and the existence of non-obvious rules. We found that our card sorting game elicited insight experience. In the first experiment, our data revealed that solution strategies and insight experience varied by the availability and saliency of perceptual features. The discovery of a non-obvious rule, which is not hinted at by perceptual features, was most difficult. With our new paradigm, we were able to construe ambiguous problems which allowed participants to find more than one solution strategy. Interestingly, we realized interindividual preferences for different strategies. The same problem drove strategies which either relied on feature integration or on more deliberate strategies. The second experiment varied the degree of independence of a sorting rule from the standard rules which were in accordance with prior knowledge. It was shown that the more independent the hidden rule was, the more difficult the task became. In sum, we demonstrated a new insight task which extended the available task domains and shed light on sequential and multi-step rule learning problems. Finally, we provided a first sketch of a cognitive model that should help to integrate the data within the existing literature on cognitive models and speculated about the generalizability of the interplay of prior knowledge modification and variation for problem solving.

Using position rather than color at the traffic light - Covariation learning-based deviation from instructions in attention deficit/hyperactivity disorder

Based on instructions people can form task representations that shield relevant from seemingly irrelevant information. It has been documented that instructions can tie people to a particular way of performing a task despite that in principle a more efficient way could be learned and used. Since task shielding can lead to persistence of inefficient variants of task performance, it is relevant to test whether individuals with attention deficit/hyperactivity disorder (ADHD) - characterized by less task shielding - are more likely and quicker to escape a suboptimal instructed variant of performing a task. The paradigm used in this online study builds on the observation that in many environments different covarying features could be used to determine the appropriate response. For instance, as they approach a traffic light, drivers and pedestrians monitor the color (instructed stimulus feature) and/or the position of the signal (covarying stimulus feature, more efficient in case of reduced color sight). Similarly, we instructed participants to respond to the color of a stimulus without mentioning that color covaried with the position of the stimulus. In order to assess whether with practice participants would use the non-instructed feature position to an increasing extent, we compared reaction times and error rates for standard trials to trials in which color was either ambiguous or did not match the usual covariation. Results showed that the covariation learning task can be administered online to adult participants with and without ADHD. Performance differences suggested that with practice ADHD participants (n = 43 out of a total N = 245) might increase attention to non-instructed stimulus features. Yet, they used the non-instructed covarying stimulus feature to a similar extent as other participants. Together the results suggest that participants with ADHD do not lag behind in abandoning instructed task processing in favor of a learned alternative strategy.

Differences in the distribution of attention to trained procedure between finders and non-finders of the alternative better procedure

The human ability to flexibly discover alternatives without fixating on a known solution supports a variety of human creative activities. Previous research has shown that people who discover an alternative procedure relax their attentional bias to information regarding the known solutions just prior to the discovery. This study examined whether the difference in the distribution of attention between the finders and non-finders of the alternative procedure is observed from the phase of solving the problem using the trained procedure. We evaluated the characteristics of the finders' distribution of attention in situations where problem solving using a trained procedure was successful. This aspect has been little examined in previous research. Our study obtained empirical evidence for the fact that, compared to non-finders, finders pay more attention to information unrelated to the trained procedure acquired through knowledge and experience, even time when using a trained procedure. We also confirmed that this difference does not exist from the beginning of the task, but emerges during repeated use of familiar procedures. These findings indicate that in order to find an alternative procedure, one should not only divert attention from a familiar procedure just before the discovery but also pay a certain amount of attention to information unrelated to the familiar procedure even when the familiar procedure is functioning well.

Spontaneous discovery of novel task solutions in children

Children often perform worse than adults on tasks that require focused attention. While this is commonly regarded as a sign of incomplete cognitive development, a broader attentional focus could also endow children with the ability to find novel solutions to a given task. To test this idea, we investigated children's ability to discover and use novel aspects of the environment that allowed them to improve their decision-making strategy. Participants were given a simple choice task in which the possibility of strategy improvement was neither mentioned by instructions nor encouraged by explicit error feedback. Among 47 children (8-10 years of age) who were instructed to perform the choice task across two experiments, 27.5% showed a full strategy change. This closely matched the proportion of adults who had the same insight (28.2% of n = 39). The amount of erroneous choices, working memory capacity and inhibitory control, in contrast, indicated substantial disadvantages of children in task execution and cognitive control. A task difficulty manipulation did not affect the results. The stark contrast between age-differences in different aspects of cognitive performance might offer a unique opportunity for educators in fostering learning in children.

Effects of Spatial Speech Presentation on Listener Response Strategy for Talker-Identification

This study investigates effects of spatial auditory cues on human listeners' response strategy for identifying two alternately active talkers (“turn-taking” listening scenario). Previous research has demonstrated subjective benefits of audio spatialization with regard to speech intelligibility and talker-identification effort. So far, the deliberate activation of specific perceptual and cognitive processes by listeners to optimize their task performance remained largely unexamined. Spoken sentences selected as stimuli were either clean or degraded due to background noise or bandpass filtering. Stimuli were presented via three horizontally positioned loudspeakers: In a non-spatial mode, both talkers were presented through a central loudspeaker; in a spatial mode, each talker was presented through the central or a talker-specific lateral loudspeaker. Participants identified talkers via speeded keypresses and afterwards provided subjective ratings (speech quality, speech intelligibility, voice similarity, talker-identification effort). In the spatial mode, presentations at lateral loudspeaker locations entailed quicker behavioral responses, which were significantly slower in comparison to a talker-localization task. Under clean speech, response times globally increased in the spatial vs. non-spatial mode (across all locations); these “response time switch costs,” presumably being caused by repeated switching of spatial auditory attention between different locations, diminished under degraded speech. No significant effects of spatialization on subjective ratings were found. The results suggested that when listeners could utilize task-relevant auditory cues about talker location, they continued to rely on voice recognition instead of localization of talker sound sources as primary response strategy. Besides, the presence of speech degradations may have led to increased cognitive control, which in turn compensated for incurring response time switch costs.

The interplay between unexpected events and behavior in the development of explicit knowledge in implicit sequence learning

Some studies in implicit learning investigate the mechanisms by which implicitly acquired knowledge (e.g., learning a sequence of responses) becomes consciously aware. It has been suggested that unexpected changes in the own behavior can trigger search processes, of which the outcome then becomes aware. A consistent empirical finding is that participants who develop explicit knowledge show a sudden decrease in reaction times, when responding to sequential events. This so called RT-drop might indicate the point of time when explicit knowledge occurs. We investigated whether an RT-drop is a precursor for the development of explicit knowledge or the consequence of explicit knowledge. To answer this question, we manipulated in a serial reaction time task the timing of long and short stimulus-onset asynchronies (SOA). For some participants, the different SOAs were presented in blocks of either long or short SOAs, while for others, the SOAs changed randomly. We expected the participants who were given a blocked presentation to express an RT-drop because of the predictable timing. In contrast, randomly changing SOAs should hamper the expression of an RT-drop. We found that more participants in the blocked-SOA condition than in the random-SOA condition showed an RT-drop. Furthermore, the amount of explicit knowledge did not differ between the two conditions. The findings suggest that the RT-drop does not seem to be a presupposition to develop explicit knowledge. Rather, it seems that the RT-drop indicates a behavioral strategy shift as a consequence of explicit knowledge.

Brain network dynamics during spontaneous strategy shifts and incremental task optimization

With practice, humans improve their performance in a task by either optimizing a known strategy or discovering a novel, potentially more fruitful strategy. We investigated the neural processes underlying these two fundamental abilities by applying fMRI in a task with two possible alternative strategies. For analysis we combined time-resolved network analysis with Coherence Density Peak Clustering (Allegra et al., 2017), univariate GLM, and multivariate pattern classification. Converging evidence showed that the posterior portion of the default network, i.e. the precuneus and the angular gyrus bilaterally, has a central role in the optimization of the current strategy. These regions encoded the relevant spatial information, increased the strength of local connectivity as well as the long-distance connectivity with other relevant regions in the brain (e.g., visual cortex, dorsal attention network). The connectivity increase was proportional to performance optimization. By contrast, the anterior portion of the default network (i.e. medial prefrontal cortex) and the rostral portion of the fronto-parietal network were associated with new strategy discovery: an early increase of local and long-range connectivity centered on these regions was only observed in the subjects who would later shift to a new strategy. Overall, our findings shed light on the dynamic interactions between regions related to attention and with cognitive control, underlying the balance between strategy exploration and exploitation. Results suggest that the default network, far from being "shut-down" during task performance, has a pivotal role in the background exploration and monitoring of potential alternative courses of action.