Electrophysiological Signatures of Hierarchical Learning

Expectation to rewards modulates learning emotional words: Evidence from a hierarchical Bayesian model

In language acquisition, individuals learn the emotional value of words through external feedback. Previous studies have used emotional words as experimental materials to explore the cognitive mechanisms underlying emotional language processing, but have failed to recognize that languages are acquired in changing environments. To this end, this study aims to combine reinforcement learning with emotional word learning, using a probabilistic reversal learning task to explore how individuals acquire the valence of emotional words in a dynamically changing environment. Computational modeling on both behavioral and event-related potential (ERP) data revealed that individuals' expectations to rewards modulated the learning speed and temporal processing of emotional words, demonstrating a clear negative bias. Specifically, as the expected value increased, individuals responded faster and exhibited higher amplitudes for negative emotional words. These findings shed light on the neural mechanisms of emotional word learning in a volatile environment, highlighting the crucial role of expectations in this process and a preference for learning negative information.

Enhanced "learning to learn" through a hierarchical dual-learning system: the case of action video game players

In contrast to conventional cognitive training paradigms, where learning effects are specific to trained parameters, playing action video games has been shown to produce broad enhancements in many cognitive functions. These remarkable generalizations challenge the conventional theory of generalization that learned knowledge can be immediately applied to novel situations (i.e., immediate generalization). Instead, a new "learning to learn" theory has recently been proposed, suggesting that these broad generalizations are attained because action video game players (AVGPs) can quickly acquire the statistical regularities of novel tasks in order to increase the learning rate and ultimately achieve better performance. Although enhanced learning rate has been found for several tasks, it remains unclear whether AVGPs efficiently learn task statistics and use learned task knowledge to guide learning. To address this question, we tested 34 AVGPs and 36 non-video game players (NVGPs) on a cue-response associative learning task. Importantly, unlike conventional cognitive tasks with fixed task statistics, in this task, cue-response associations either remain stable or change rapidly (i.e., are volatile) in different blocks. To complete the task, participants should not only learn the lower-level cue-response associations through explicit feedback but also actively estimate the high-level task statistics (i.e., volatility) to dynamically guide lower-level learning. Such a dual learning system is modelled using a hierarchical Bayesian learning framework, and we found that AVGPs indeed quickly extract the volatility information and use the estimated higher volatility to accelerate learning of the cue-response associations. These results provide strong evidence for the "learning to learn" theory of generalization in AVGPs. Taken together, our work highlights enhanced hierarchical learning of both task statistics and cognitive abilities as a mechanism underlying the broad enhancements associated with action video game play.

The interplay of cognitive control and feature integration: insights from theta oscillatory dynamics during conflict processing

Adaptive behavior is fundamental to cognitive control and executive functioning. This study investigates how cognitive control mechanisms and episodic feature retrieval interact to influence adaptiveness, focusing particularly on theta (4 to 8 Hz) oscillatory dynamics. We conducted two variations of the Simon task, incorporating response-incompatible, response-compatible, and neutral trials. Experiment 1 demonstrated that cognitive adjustments-specifically, cognitive shielding following incompatible trials and cognitive relaxation following compatible ones-are reflected in midfrontal theta power modulations associated with the Simon effect. Experiment 2 showed that reducing feature overlap between trials leads to less pronounced sequential modulations in behavior and midfrontal theta activity, supporting the hypothesis that cognitive control and feature integration share a common neural mechanism. These findings highlight the interaction of cognitive control processes and episodic feature integration in modulating behavior. The results advocate for hybrid models that combine top-down and bottom-up processes as a comprehensive framework to understand cognitive control dynamics and adaptive behavior.

Alpha oscillations encode Bayesian belief updating underlying attentional allocation in dynamic environments

In a dynamic environment, expectations of the future constantly change based on updated evidence and affect the dynamic allocation of attention. To further investigate the neural mechanisms underlying attentional expectancies, we employed a modified Central Cue Posner Paradigm in which the probability of cues being valid (that is, accurately indicated the upcoming target location) was manipulated. Attentional deployment to the cued location (α), which was governed by precision of predictions on previous trials, was estimated using a hierarchical Bayesian model and was included as a regressor in the analyses of electrophysiological (EEG) data. Our results revealed that before the target appeared, alpha oscillations (8∼13 Hz) for high-predictability cues (88 % valid) were significantly predicted by precision-dependent attention (α). This relationship was not observed under low-predictability conditions (69 % and 50 % valid cues). After the target appeared, precision-dependent attention (α) correlated with alpha band oscillations only in the valid cue condition and not in the invalid condition. Further analysis under conditions of significant attentional modulation by precision suggested a separate effect of cue orientation. These results provide new insights on how trial-by-trial Bayesian belief updating relates to alpha band encoding of environmentally-sensitive allocation of visual spatial attention.

Neurocomputational mechanisms underlying fear-biased adaptation learning in changing environments

Humans are able to adapt to the fast-changing world by estimating statistical regularities of the environment. Although fear can profoundly impact adaptive behaviors, the computational and neural mechanisms underlying this phenomenon remain elusive. Here, we conducted a behavioral experiment (n = 21) and a functional magnetic resonance imaging experiment (n = 37) with a novel cue-biased adaptation learning task, during which we simultaneously manipulated emotional valence (fearful/neutral expressions of the cue) and environmental volatility (frequent/infrequent reversals of reward probabilities). Across 2 experiments, computational modeling consistently revealed a higher learning rate for the environment with frequent versus infrequent reversals following neutral cues. In contrast, this flexible adjustment was absent in the environment with fearful cues, suggesting a suppressive role of fear in adaptation to environmental volatility. This suppressive effect was underpinned by activity of the ventral striatum, hippocampus, and dorsal anterior cingulate cortex (dACC) as well as increased functional connectivity between the dACC and temporal-parietal junction (TPJ) for fear with environmental volatility. Dynamic causal modeling identified that the driving effect was located in the TPJ and was associated with dACC activation, suggesting that the suppression of fear on adaptive behaviors occurs at the early stage of bottom-up processing. These findings provide a neuro-computational account of how fear interferes with adaptation to volatility during dynamic environments.

Modulating hierarchical learning by high-definition transcranial alternating current stimulation at theta frequency

Considerable evidence highlights the dorsolateral prefrontal cortex (DLPFC) as a key region for hierarchical (i.e. multilevel) learning. In a previous electroencephalography (EEG) study, we found that the low-level prediction errors were encoded by frontal theta oscillations (4-7 Hz), centered on right DLPFC (rDLPFC). However, the causal relationship between frontal theta oscillations and hierarchical learning remains poorly understood. To investigate this question, in the current study, participants received theta (6 Hz) and sham high-definition transcranial alternating current stimulation (HD-tACS) over the rDLPFC while performing the probabilistic reversal learning task. Behaviorally, theta tACS induced a significant reduction in accuracy for the stable environment, but not for the volatile environment, relative to the sham condition. Computationally, we implemented a combination of a hierarchical Bayesian learning and a decision model. Theta tACS induced a significant increase in low-level (i.e. probability-level) learning rate and uncertainty of low-level estimation relative to sham condition. Instead, the temperature parameter of the decision model, which represents (inverse) decision noise, was not significantly altered due to theta stimulation. These results indicate that theta frequency may modulate the (low-level) learning rate. Furthermore, environmental features (e.g. its stability) may determine whether learning is optimized as a result.