Interactions between visual and motor areas during the recognition of plausible actions as revealed by magnetoencephalography

Using virtual reality to assess gesture performance deficits in schizophrenia patients

Introduction

Gesture performance deficits are prevalent in schizophrenia patients and are strongly associated with poor social communication skills and community functioning, affecting their overall quality of life. Currently, video-recording technology is widely used in clinical settings to assess gesture production deficits in schizophrenia patients. Nevertheless, the subjective evaluation of video-recordings can encumber task assessment. The present study will aim to use virtual reality to examine its potential use as an alternative tool to objectively measure gesture performance accuracy in schizophrenia patients and healthy controls.

Methods

Gesture performance in the virtual reality setting will be based on the well-established Test of Upper Limb Apraxia. Participants will be immersed in a virtual environment where they will experience themselves being embodied in a collocated virtual body seen from a first-person perspective. Motion trackers will be placed on participants' hands and elbows to track upper body movements in real-time, and to record gesture movement for later analysis. Participants will see a virtual agent sitting across from them, with a virtual table in between. The agent will perform various types of gestures and the participants' task will be to imitate those gestures as accurately as possible. Measurements from the tracking devices will be stored and analyzed to address gesture performance accuracy across groups.

Discussion

This study aims to provide objective measurements of gesture performance accuracy in schizophrenia patients. If successful, the results will provide new knowledge to the gesture literature and offer the potential for novel therapeutic interventions using virtual reality technologies. Such interventions can improve gesturing and thus advance social communication skills in schizophrenia patients.

Human movements do not look the same in a tilted world: Gravitational constraints influence the perception of biological motion

We investigated whether gravitational constraints influence the interaction of visual, proprioceptive and vestibular cues for Biological Motion Perception (BMP). Participants were asked to distinguish between plausible and random point-light movements, while passively placed in either an upright or a tilted body orientation. Manipulating the body orientation with respect to gravity leads to different gravitational signals transmitted by the visual, proprioceptive, and vestibular systems. Participants were overall faster in distinguishing plausible point-light movements than random movements. Critically, response times for biologically plausible point-light movements - but not for random movements - were significantly prolonged in the tilted body orientation. Our results suggest that BMP depends not only on the spatial-temporal cues embedded in point-light movements but also rely on the congruency between current gravitational signals detected by the sensory systems and our previous knowledge of terrestrial gravity. STATEMENT OF RELEVANCE: As humankind is preparing for a new space age, understanding how gravity influences behaviour and cognition has never been more pressing. All living organisms have evolved to survive in a terrestrial gravitational field. Although we cannot consciously feel gravity, it has an impact in our life: it affects how we move and interact with the external environment. The sensory signals from the vestibular system are continuously combined with visual and proprioceptive cues to help us in maintaining a stable representation of the world. Here we placed participants in a tilted body orientation and were able to determine that a conflict between prior gravitational knowledge and what was actively sensed about gravity affected human Biological Movement Perception. Humans suffer changes in perception under non-terrestrial gravity conditions that may potentially compromise performance during space exploration.

Brain Stimulation and Group Therapy to Improve Gesture and Social Skills in Schizophrenia-The Study Protocol of a Randomized, Sham-Controlled, Three-Arm, Double-Blind Trial

An important component of nonverbal communication is gesture performance, which is strongly impaired in 2/3 of patients with schizophrenia. Gesture deficits in schizophrenia are linked to poor social functioning and reduced quality of life. Therefore, interventions that can help alleviate these deficits in schizophrenia are crucial. Here, we describe an ongoing randomized, double-blind 3-arm, sham-controlled trial that combines two interventions to reduce gesture deficits in schizophrenia patients. The combined interventions are continuous theta burst stimulation (cTBS) and social cognitive remediation therapy (SCRT). We will randomize 72 patients with schizophrenia spectrum disorders in three different groups of 24 patients. The first group will receive real cTBS and real SCRT, the second group will receive sham cTBS and real SCRT, and finally the third group will receive sham SCRT. Here, the sham treatments are, as per definition, inactive interventions that mimic as closely as possible the real treatments (similar to placebo). In addition, 24 age- and gender-matched controls with no interventions will be added for comparison. Measures of nonverbal communication, social cognition, and multimodal brain imaging will be applied at baseline and after intervention. The main research aim of this project will be to test whether the combination of cTBS and SCRT improves gesture performance and social functioning in schizophrenia patients more than standalone cTBS, SCRT or sham psychotherapy. We hypothesize that the patient group receiving the combined interventions will be superior in improving gesture performance.

CLINICAL TRIAL REGISTRATION

[www.ClinicalTrials.gov], identifier [NCT04106427].

Enhanced biological motion perception in deaf native signers

We conducted two studies to test how deaf signed language users perceive biological motions. We created 18 Biological Motion point-light displays (PLDs) depicting everyday human actions, and 18 Scrambled control PLDs. First, we conducted an online behavioral rating survey, in which deaf and hearing raters identified the biological motion PLDs and rated how easy it was for them to identify the actions. Then, we conducted an EEG study in which Deaf Signers and Hearing Non-Signers watched both the Biological Motion PLDs and the Scrambled PLDs, and we computed the time-frequency responses within the theta, alpha, and beta EEG rhythms. From the behavioral rating task, we show that the deaf raters reported significantly less effort required for identifying the Biological motion PLDs, across all stimuli. The EEG results showed that the Deaf Signers showed theta, mu, and beta differentiation between Scrambled and Biological PLDs earlier and more consistently than Hearing Non-Signers. We conclude that native ASL users exhibit experience-dependent neuroplasticity in the domain of biological human motion perception.

Cortical beta oscillations reflect the contextual gating of visual action feedback

•

We decouple seen and felt hand postures during action via virtual reality.

•

Vision of the hand is either task-relevant or a distractor.

•

Task-relevance of vision is reflected by in- or decreases of occipital beta power.

•

DCM suggests underlying changes in cortical (visual) excitability.

•

Occipital beta may indicate the contextual gating of visual action feedback.

In sensorimotor integration, the brain needs to decide how its predictions should accommodate novel evidence by ‘gating’ sensory data depending on the current context. Here, we examined the oscillatory correlates of this process by recording magnetoencephalography (MEG) data during a new task requiring action under intersensory conflict. We used virtual reality to decouple visual (virtual) and proprioceptive (real) hand postures during a task in which the phase of grasping movements tracked a target (in either modality). Thus, we rendered visual information either task-relevant or a (to-be-ignored) distractor. Under visuo-proprioceptive incongruence, occipital beta power decreased (relative to congruence) when vision was task-relevant but increased when it had to be ignored. Dynamic causal modeling (DCM) revealed that this interaction was best explained by diametrical, task-dependent changes in visual gain. These results suggest a crucial role for beta oscillations in the contextual gating (i.e., gain or precision control) of visual vs proprioceptive action feedback, depending on current behavioral demands.

Observation of Point-Light-Walker Locomotion Induces Motor Resonance When Explicitly Represented; An EEG Source Analysis Study

Understanding human motion, to infer the goal of others' actions, is thought to involve the observer's motor repertoire. One prominent class of actions, the human locomotion, has been object of several studies, all focused on manipulating the shape of degraded human figures like point-light walker (PLW) stimuli, represented as walking on the spot. Nevertheless, since the main goal of the locomotor function is to displace the whole body from one position to the other, these stimuli might not fully represent a goal-directed action and thus might not be able to induce the same motor resonance mechanism expected when observing a natural locomotion. To explore this hypothesis, we recorded the event-related potentials (ERP) of canonical/scrambled and translating/centered PLWs decoding. We individuated a novel ERP component (N2c) over central electrodes, around 435 ms after stimulus onset, for translating compared to centered PLW, only when the canonical shape was preserved. Consistently with our hypothesis, sources analysis associated this component to the activation of trunk and lower legs primary sensory-motor and supplementary motor areas. These results confirm the role of own motor repertoire in processing human action and suggest that ERP can detect the associated motor resonance only when the human figure is explicitly involved in performing a meaningful action.

Event-related desynchronization of mu and beta oscillations during the processing of novel tool names

According to the embodied cognition framework, the formation of conceptual representations integrates the type of experience during learning. In this electroencephalographic study, we applied a linguistic variant of a training paradigm, in which participants learned to associate novel names to novel tools while either manipulating or visually exploring them. The analysis focused on event-related desynchronization (ERD) of oscillations in the mu and beta frequency range, which reflects activation of sensorimotor brain areas. After three training sessions, processing names of manipulated tools elicited a stronger ERD of the beta (18-25 Hz, 140-260 ms) and the lower mu rhythm (8-10 Hz, 320-440 ms) than processing names of visually explored tools, reflecting a possible reactivation of experiential sensorimotor information. Given the unexpected result that familiarized pseudo-words elicited an ERD comparable to names of manipulated tools, our findings could reflect a suppression of sensorimotor activity during the processing of objects with exclusively visual features.

Beyond the Peak - Tactile Temporal Discrimination Does Not Correlate with Individual Peak Frequencies in Somatosensory Cortex

The human sensory systems constantly receive input from different stimuli. Whether these stimuli are integrated into a coherent percept or segregated and perceived as separate events, is critically determined by the temporal distance of the stimuli. This temporal distance has prompted the concept of temporal integration windows or perceptual cycles. Although this concept has gained considerable support, the neuronal correlates are still discussed. Studies suggested that neuronal oscillations might provide a neuronal basis for such perceptual cycles, i.e., the cycle lengths of alpha oscillations in visual cortex and beta oscillations in somatosensory cortex might determine the length of perceptual cycles. Specifically, recent studies reported that the peak frequency (the frequency with the highest spectral power) of alpha oscillations in visual cortex correlates with subjects' ability to discriminate two visual stimuli. In the present study, we investigated whether peak frequencies in somatosensory cortex might serve as the correlate of perceptual cycles in tactile discrimination. Despite several different approaches, we were unable to find a significant correlation between individual peak frequencies in the alpha- and beta-band and individual discrimination abilities. In addition, analysis of Bayes factor provided evidence that peak frequencies and discrimination thresholds are unrelated. The results suggest that perceptual cycles in the somatosensory domain are not necessarily to be found in the peak frequency, but in other frequencies. We argue that studies based solely on analysis of peak frequencies might thus miss relevant information.

MEG Multivariate Analysis Reveals Early Abstract Action Representations in the Lateral Occipitotemporal Cortex

Understanding other people's actions is a fundamental prerequisite for social interactions. Whether action understanding relies on simulating the actions of others in the observers' motor system or on the access to conceptual knowledge stored in nonmotor areas is strongly debated. It has been argued previously that areas that play a crucial role in action understanding should (1) distinguish between different actions, (2) generalize across the ways in which actions are performed (Dinstein et al., 2008; Oosterhof et al., 2013; Caramazza et al., 2014), and (3) have access to action information around the time of action recognition (Hauk et al., 2008). Whereas previous studies focused on the first two criteria, little is known about the dynamics underlying action understanding. We examined which human brain regions are able to distinguish between pointing and grasping, regardless of reach direction (left or right) and effector (left or right hand), using multivariate pattern analysis of magnetoencephalography data. We show that the lateral occipitotemporal cortex (LOTC) has the earliest access to abstract action representations, which coincides with the time point from which there was enough information to allow discriminating between the two actions. By contrast, precentral regions, though recruited early, have access to such abstract representations substantially later. Our results demonstrate that in contrast to the LOTC, the early recruitment of precentral regions does not contain the detailed information that is required to recognize an action. We discuss previous theoretical claims of motor theories and how they are incompatible with our data.

SIGNIFICANCE STATEMENT

It is debated whether our ability to understand other people's actions relies on the simulation of actions in the observers' motor system, or is based on access to conceptual knowledge stored in nonmotor areas. Here, using magnetoencephalography in combination with machine learning, we examined where in the brain and at which point in time it is possible to distinguish between pointing and grasping actions regardless of the way in which they are performed (effector, reach direction). We show that, in contrast to the predictions of motor theories of action understanding, the lateral occipitotemporal cortex has access to abstract action representations substantially earlier than precentral regions.

Lateralized modulation of beta-band power in sensorimotor areas during action observation

The cortical network for action observation includes areas of the visual cortex and non-visual areas, including areas of the motoric system. Parts of this network are known for their contralateral organization during motion execution, i.e., they predominantly control and respond to movements of the contralateral body side. We were interested whether this lateralized organization was also present during action observation. Human participants viewed point-light displays of human actors, where the actor was facing and moving either to the right or to the left, while participants' neuromagnetic activity was recorded using magnetoencephalography (MEG). We found that right and left facing movements elicited different activity in left and right motoric areas. This lateralization effect was found in two distinct spatio-temporal-spectral clusters: An early lateralization effect in medial sensors at 12-16 Hz and ~276-675 ms after stimulus onset, and a second cluster in more lateral sensors at 22-28 Hz and ~1275-1775 ms. Our results demonstrate that in addition to the known somatotopic organization of parts of the human motoric system, these areas also show a lateralization effect during action observation. Thus, our results indicate that the hemispheric organization of one's own body map known for motion execution extends to the visual observation of others' bodily actions and movements.

Beta oscillations and their functional role in movement perception

Neuronal oscillations refer to periodic changes of neuronal activity. A prominent neuronal oscillation, especially in sensorimotor areas, is the beta-frequency-band (∼ 13–29 Hz). Sensorimotor beta oscillations are predominantly linked to motor-related functions such as preparation and/or execution of movements. In addition, beta oscillations have been suggested to play a role in long-range communication between multiple brain areas. In this review, we assess different studies that show that sensorimotor beta oscillations are additionally involved in the visual perception and imagery of biological movements. We propose that sensorimotor beta oscillations reflect a mechanism of attempted matching to internally stored representations of movements. We additionally, provide evidence that beta oscillations play a role for the integration of visual and sensorimotor areas to a functional network that incorporates perceptual components at specific spatial-temporal profiles and transmits information across different areas.