Distinct neural substrates for visual short-term memory of actions

An active inference model of hierarchical action understanding, learning and imitation

We advance a novel active inference model of the cognitive processing that underlies the acquisition of a hierarchical action repertoire and its use for observation, understanding and imitation. We illustrate the model in four simulations of a tennis learner who observes a teacher performing tennis shots, forms hierarchical representations of the observed actions, and imitates them. Our simulations show that the agent's oculomotor activity implements an active information sampling strategy that permits inferring the kinematic aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions: proximal goals and intentions. Finally, the inferred action representations can steer imitative responses, but interfere with the execution of different actions. Our simulations show that hierarchical active inference provides a unified account of action observation, understanding, learning and imitation and helps explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms.

Characteristics of brain activation in high-level football players at different stages of decision-making tasks off the ball: an fMRI study

Objective

This study aimed to examine the neural mechanisms underlying the decision-making process of off-ball movements among high-level football players and ordinary college students, as well as the effect of long-term skill training on these neural mechanisms using functional magnetic resonance imaging (fMRI).

Methods

The study recruited 20 professional college football players as the expert group (EG) and 20 novice football players with no background in sports-related disciplines as the novice group (NG). The participants performed the motor video observation and button-decision-making tasks, and fMRI data were acquired, pre-processed, and analyzed.

Results

During the decision-making process regarding running without the ball, whole-brain fMRI scans were conducted on both the EG and NG. The analysis of these scans revealed noteworthy disparities in brain activity between the two groups. These disparities were observed during tasks involving motor video observation and button-based decision-making. According to the behavioral data, the EG made more correct decisions than the NG (p < 0.05); however, there was no significant difference in their reaction speed (p > 0.05). During video observation, both the EG and NG exhibited simultaneous activation in the frontoparietal cognitive area, primary somatosensory cortex, visual cortex, and insula. However, there were no significant differences between the two groups in terms of activated brain regions [false discovery rate (FDR) corrected to p < 0.05]. Regarding button-press decisions, the areas of the brain that were commonly activated in both the NG and EG were primarily located in the frontoparietal cognitive area, temporal cortex, and cuneus cortex. Notably, the left superior temporal gyrus, left inferior temporal gyrus, and left middle occipital gyrus exhibited greater activation in the NG compared to those in the EG (FDR corrected to p < 0.05).

Conclusion

This study demonstrated that during motor video observation, the EG's sports experience and professional knowledge can help them achieve better visual information processing strategies in specific areas of sports. During button decision-making, the EG was more economical, whereas the NG required more brain function activity to process visual information, confirming the "neural efficiency" hypothesis.

How are patterned movements stored in working memory?

Introduction

In this study, the change detection paradigm was used to study the working memory of patterned movements and the relationship of this type of memory with the visuospatial sketchpad in three experiments.

Methods

Experiment 1 measured participants' working memory capacity for patterned movements and explored the influence of stimulus type with indicators such as response time and accuracy rate. Experiments 2 and 3 explored the relationship between patterned movements and the visual and spatial subsystems, respectively.

Results

The results of Experiment 1 indicated that individuals can store 3-4 patterned movements in working memory; however, a change in stimulus format or an increase in memory load may decrease the speed and efficiency of working memory processing. The results of Experiment 2 showed that working memory and visual working memory are independent when processing patterned movements. The results of Experiment 3 showed that the working memory of patterned movements was affected by spatial working memory.

Discussion

Changes in stimulus type and memory load exerted different effects on the working memory capacity of participants. These results provide behavioral evidence that the storage of patterned movement information is independent of the visual subsystem but requires the spatial subsystem of the visuospatial sketchpad.

Static and temporal dynamic changes of intrinsic brain activity in pediatric and adults OCD

Epidemiological and clinical age differences in obsessive-compulsive disorder (OCD) have been reported in clinical symptoms and morphometry changes; however, age differences in amplitude of low-frequency fluctuation and the relationship between ALFF imaging and clinical symptoms has not been thoroughly studied in OCD. Age may be an important feature associated with distinct subtypes of OCD. To examine the effect of age on OCD, the current study enrolled 92 OCD patients (32 pediatrics and 60 adults) and matched HCs (33 pediatrics and 84 adults), undergoing resting-state functional magnetic resonance imaging. The spontaneous brain activity was measured by static and dynamic amplitude of low-frequency fluctuation (ALFF) followed by two-way ANOVA. In pediatric OCD patients versus adult patients, we observed a significantly higher ALFF in the default mode network (DMN), including posterior cingulate, precuneus and superior frontal gyrus, and extending to cuneus, lingual gyrus. Additionally, the increased ALFF and dynamic ALFF in the precentral gyrus were found in pediatric patients. In OCD patients compared with controls, we found a significantly increased ALFF in hippocampal gyrus, cerebellum network (CN), and the dALFF in middle and inferior occipital gyrus, bilateral paracentral lobule and sensorimotor network. The findings emphasized the different patterns of static and dynamic intrinsic brain activity alterations associated with pediatric and adult OCD patients. These results provide unique insights into constructing evidenced-based distinct OCD subtypes based on brain activity and point the need of specified management for pediatric and adult OCD patients in clinical setting.

Delay activity during visual working memory: A meta-analysis of 30 fMRI experiments

Visual working memory refers to the temporary maintenance and manipulation of task-related visual information. Recent debate on the underlying neural substrates of visual working memory has focused on the delay period of relevant tasks. Persistent neural activity throughout the delay period has been recognized as a correlate of working memory, yet regions demonstrating sustained hemodynamic responses show inconsistency across individual studies. To develop a more precise understanding of delay-period activations during visual working memory, we conducted a coordinate-based meta-analysis on 30 fMRI experiments involving 515 healthy adults with a mean age of 25.65 years. The main analysis revealed a widespread frontoparietal network associated with delay-period activity, as well as activation in the right inferior temporal cortex. These findings were replicated using different meta-analytical algorithms and were shown to be robust against between-study heterogeneity and publication bias. Further meta-analyses on different subgroups of experiments with specific task demands and stimulus types revealed similar delay-period networks, with activations distributed across the frontal and parietal cortices. The roles of prefrontal regions, posterior parietal regions, and inferior temporal areas are reviewed and discussed in the context of content-specific storage. We conclude that cognitive operations that occur during the unfilled delay period in visual working memory tasks can be flexibly expressed across a frontoparietal-temporal network depending on experimental parameters.

In which case is working memory for movements affected by verbal interference? Evidence from the verbal description of movement

Current perspectives on whether verbal interference affects working memory for movements have not yet reached a consensus. This study explored the causes of this controversy to reveal the relation between working memory for movements and the phonological loop. Experiment 1 explored whether the verbal description of movement moderated the effect of verbal interference (articulatory suppression) on working memory for movements. Verbal interference only affected working memory for easy-to-describe movements (lower accuracy). Experiment 2 excluded the interpretation of familiarity to the controversy and the effect of familiarity on the results of Experiment 1. Experiment 3 verified the results of Experiment 1 with another form of verbal interference, i.e., presenting irrelevant words visually. These three experiments suggest that the phonological loop is not recruited for processing working memory for movements in nature, but the two may interact through the verbal description prestored in the long-term memory. Thus, the current study provides a certain level of support for the separable movement-based subsystem hypothesis (Smyth, M. M., Pearson, N. A., & Pendleton, L. R. (1988). Movement and working memory: Patterns and positions in space. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology, 40(3), 497-514. doi:10.1080/02724988843000041).

Watching video of discrete maneuvers yields better action memory and greater activation in the middle temporal gyrus in half-pipe snowboarding athletes

Although motor performance training often involves action observation, it has been controversial whether individual aesthetic sport athletes benefit more from watching videos of discrete maneuvers (DMs) or continuous runs (CRs). In the present study, half-pipe snowboarding athletes completed a visual 2-back task with DM and CR conditions. To explore the neural mechanisms of action memory processing, brain hemodynamic activity during the task was monitored with functional near-infrared spectroscopy (fNIRS). Compared to watching CR videos, watching DM videos tended to yield better action memory performance and greater activation in the middle temporal gyrus to these athletes, suggesting that watching DM videos may have a tendency to improve action memory more effectively. Evidence of two pathways underlying half-pipe snowboarding action processing was obtained. Watching of CR videos and watching of DM videos might be associated with activation of more sensorimotor regions and more semantic regions, respectively, during memory consolidation.

Cross-modal involvement of the primary somatosensory cortex in visual working memory: A repetitive TMS study

Recent literature suggests that the primary somatosensory cortex (S1), once thought to be a low-level area only modality-specific, is also involved in higher-level, cross-modal, cognitive functions. In particular, electrophysiological studies have highlighted that the cross-modal activation of this area may also extend to visual Working Memory (WM), being part of a mnemonic network specific for the temporary storage and manipulation of visual information concerning bodies and body-related actions. However, the causal recruitment of S1 in the WM network remains speculation. In the present study, by taking advantage of repetitive Transcranial Magnetic Stimulation (rTMS), we look for causal evidence that S1 is implicated in the retention of visual stimuli that are salient for this cortical area. To this purpose, in a first experiment, high-frequency (10 Hz) rTMS was delivered over S1 of the right hemisphere, and over two control sites, the right lateral occipital cortex (LOC) and the right dorsolateral prefrontal cortex (dlPFC), during the maintenance phase of a high-load delayed match-to-sample task in which body-related visual stimuli (non-symbolic hand gestures) have to be retained. In a second experiment, the specificity of S1 recruitment was deepened by using a version of the delayed match-to-sample task in which visual stimuli depict geometrical shapes (non-body related stimuli). Results show that rTMS perturbation of S1 activity leads to an enhancement of participants' performance that is selective for body-related visual stimuli; instead, the stimulation of the right LOC and dlPFC does not affect the temporary storage of body-related visual stimuli. These findings suggest that S1 may be recruited in visual WM when information to store (and recall) is salient for this area, corroborating models which suggest the existence of a dedicated mnemonic system for body-related information in which also somatosensory cortices play a key role, likely thanks to their cross-modal (visuo-tactile) properties.

Beyond action observation: Neurobehavioral mechanisms of memory for visually perceived bodies and actions

Examining the processing of others' body-related information in the perceivers' brain (action observation) is a key topic in cognitive neuroscience. However, what happens beyond the perceptual stage, when the body is not within view and it is transformed into an associative form that can be stored, updated, and later recalled, remains poorly understood. Here we examine neurobehavioural evidence on the memory processing of visually perceived bodily stimuli (dynamic actions and images of bodies). The reviewed studies indicate that encoding and maintaining bodily stimuli in memory recruits the sensorimotor system. This process arises when bodily stimuli are either recalled through action recognition or reproduction. Interestingly, the memory capacity for these stimuli is rather limited: only 2 or 3 bodily stimuli can be simultaneously held in memory. Moreover, this process is disrupted by increasing concurrent bodily operations; i.e., moving one's body, seeing or memorising additional bodies. Overall, the evidence suggests that the neural circuitry allowing us to move and feel ourselves supports the encoding, retention, and memory recall of others' visually perceived bodies.

Event-based encoding of biological motion and location in visual working memory

We make use of discrete yet meaningful events to orient ourselves to the dynamic environment. Among these events, biological motion, referring to the movements of animate entities, is one of the most biologically salient. We usually encounter biological motions of multiple human beings taking place simultaneously at distinct locations. How we encode biological motions into visual working memory (VWM) to form a coherent experience of the external world and guide our social behaviour remains unclear. This study for the first time addressed the VWM encoding mechanism of biological motions and their corresponding locations. We tested an event-based encoding hypothesis for biological motion and location: When one element of an event is required to be memorised, the irrelevant element of an event will also be extracted into VWM. We presented participants with three biological motions at different locations and required them to memorise only the biological motions or their locations while ignoring the other dimension. We examined the event-based encoding by probing a distracting effect: If the event-based encoding took place, the change of irrelevant dimension in the probe would lead to a significant distraction and impair the performance of detecting target dimension. We found significant distracting effects, which lasted for 3 s but vanished at 6 s, regardless of the target dimension (biological motions vs. locations, Experiment 1) and the exposure time of memory array (1 s vs. 3 s, Experiment 2). These results together support an event-based encoding mechanism during VWM encoding of biological motions and their corresponding locations.

Agent identity drives adaptive encoding of biological motion into working memory

To engage in normal social interactions, we have to encode human biological motions (BMs, e.g., walking and jumping), which is one of the most salient and biologically significant types of kinetic information encountered in everyday life, into working memory (WM). Critically, each BM in real life is produced by a distinct person, carrying a dynamic motion signature (i.e., identity). Whether agent identity influences the WM processing of BMs remains unknown. Here, we addressed this question by examining whether memorizing BMs with different identities promoted the WM processing of task-irrelevant clothing colors. Two opposing hypotheses were tested: (a) WM only stores the target action (element-based hypothesis) and (b) WM stores both action and irrelevant clothing color (event-based hypothesis), interpreting each BM as an event. We required participants to memorize actions that either performed by one agent or distinct agents, while ignoring clothing colors. Then we examined whether the irrelevant color was also stored in WM by probing a distracting effect: If the color was extracted into WM, the change of irrelevant color in the probe would lead to a significant distracting effect on action performance. We found that WM encoding of BMs was adaptive: Once the memorized actions had different identities, WM adopted an event-based encoding mode regardless of memory load and probe identity (Experiment 1, different-identity group of Experiment 2, and Experiment 3). However, WM used an element-based encoding mode when memorized-actions shared the same identity (same-identity group of Experiment 2) or were inverted (Experiment 4). Overall, these findings imply that agent identity information has a significant effect on the WM processing of BMs.

Relation Between Working Memory Capacity of Biological Movements and Fluid Intelligence

Studies have revealed that there is an independent buffer for holding biological movements (BM) in working memory (WM), and this BM-WM has a unique link to our social ability. However, it remains unknown as to whether the BM-WM also correlates to our cognitive abilities, such as fluid intelligence (Gf). Since BM processing has been considered as a hallmark of social cognition, which distinguishes from canonical cognitive abilities in many ways, it has been hypothesized that only canonical object-WM (e.g., memorizing color patches), but not BM-WM, emerges to have an intimate relation with Gf. We tested this prediction by measuring the relationship between WM capacity of BM and Gf. With two Gf measurements, we consistently found moderate correlations between BM-WM capacity, the score of both Raven's advanced progressive matrix (RAPM), and the Cattell culture fair intelligence test (CCFIT). This result revealed, for the first time, a close relation between WM and Gf with a social stimulus, and challenged the double-dissociation hypothesis for distinct functions of different WM buffers.

Distinct neural substrates for visual short-term memory of actions

Fundamental theories of human cognition have long posited that the short-term maintenance of actions is supported by one of the "core knowledge" systems of human visual cognition, yet its neural substrates are still not well understood. In particular, it is unclear whether the visual short-term memory (VSTM) of actions has distinct neural substrates or, as proposed by the spatio-object architecture of VSTM, shares them with VSTM of objects and spatial locations. In two experiments, we tested these two competing hypotheses by directly contrasting the neural substrates for VSTM of actions with those for objects and locations. Our results showed that the bilateral middle temporal cortex (MT) was specifically involved in VSTM of actions because its activation and its functional connectivity with the frontal-parietal network (FPN) were only modulated by the memory load of actions, but not by that of objects/agents or locations. Moreover, the brain regions involved in the maintenance of spatial location information (i.e., superior parietal lobule, SPL) was also recruited during the maintenance of actions, consistent with the temporal-spatial nature of actions. Meanwhile, the frontoparietal network (FPN) was commonly involved in all types of VSTM and showed flexible functional connectivity with the domain-specific regions, depending on the current working memory tasks. Together, our results provide clear evidence for a distinct neural system for maintaining actions in VSTM, which supports the core knowledge system theory and the domain-specific and domain-general architectures of VSTM.