Body ownership and agency: task-dependent effects of the virtual hand illusion on proprioceptive drift

Differences in the early stages of motor learning between visual-motor illusion and action observation

The visual-motor illusion (VMI) induces a kinesthetic illusion by watching one's physically-moving video while the body is at rest. It remains unclear whether the early stages (immediately to one hour later) of motor learning are promoted by VMI. This study investigated whether VMI changes the early stages of motor learning in healthy individuals. Thirty-six participants were randomly assigned to two groups: the VMI or action observation condition. Each condition was performed with the left hand for 20 min. The VMI condition induced a kinesthetic illusion by watching one's ball-rotation task video. The action observation condition involved watching the same video as the VMI condition but did not induce a kinesthetic illusion. The ball-rotation task and brain activity during the task were measured pre, post1 (immediately), and post2 (after 1 h) in both conditions, and brain activity was measured using functional near-infrared spectroscopy. The rate of the ball-rotation task improved significantly at post1 and post2 in the VMI condition than in the action observation condition. VMI condition lowers left dorsolateral prefrontal cortex and right premotor area activity from post1 to pre compared to the action observation condition. In conclusion, VMI effectively aids early stages of motor learning in healthy individuals.

Narrative simulation of social experiences in naturalistic context - A neurocinematic approach

Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.

Tactile temporal order judgment during rubber hand illusion: Distinct modulation of the point of subjective simultaneity and temporal resolution

During the rubber hand illusion (RHI), individuals feel a fake hand as their own (ownership) and a perceived position of their real hand shifts toward the fake hand (proprioceptive drift; PD), which represents updating of multisensory hand representations. Bimanual tactile temporal order judgment (TOJ) includes processes of localizing tactile stimuli in space, for which multisensory hand representations are essential. According to the common processes, we examined tactile TOJ performance during the RHI and non-RHI. Temporal resolution (TR) as TOJ accuracy worsened during the non-RHI compared to the RHI. Additionally, a significant correlation between TR and PD was observed only in the non-RHI condition. However, the point of subjective simultaneity (PSS), which offers relative weighting of tactile inputs from the right and left hands, was correlated with illusory hand ownership. These results suggest that PSS and TR from tactile TOJ during RHI relate to self-attribution and localization of the hand, respectively.

Bodily ownership of an independent supernumerary limb: an exploratory study

Can our brain perceive a sense of ownership towards an independent supernumerary limb; one that can be moved independently of any other limb and provides its own independent movement feedback? Following the rubber-hand illusion experiment, a plethora of studies have shown that the human representation of “self” is very plastic. But previous studies have almost exclusively investigated ownership towards “substitute” artificial limbs, which are controlled by the movements of a real limb and/or limbs from which non-visual sensory feedback is provided on an existing limb. Here, to investigate whether the human brain can own an independent artificial limb, we first developed a novel independent robotic “sixth finger.” We allowed participants to train using the finger and examined whether it induced changes in the body representation using behavioral as well as cognitive measures. Our results suggest that unlike a substitute artificial limb (like in the rubber hand experiment), it is more difficult for humans to perceive a sense of ownership towards an independent limb. However, ownership does seem possible, as we observed clear tendencies of changes in the body representation that correlated with the cognitive reports of the sense of ownership. Our results provide the first evidence to show that an independent supernumerary limb can be embodied by humans.

The role of motor memory dynamics in structuring bodily self-consciousness

Bodily self-consciousness has been considered a sensorimotor root of self-consciousness. If this is the case, how does sensorimotor memory, which is important for the prediction of sensory consequences of volitional actions, influence awareness of bodily self-consciousness? This question is essential for understanding the effective acquisition and recovery of self-consciousness following its impairment, but it has remained unexamined. Here, we investigated how body ownership and agency recovered following body schema distortion in a virtual reality environment along with two kinds of motor memories: memories that were rapidly updated and memories that were gradually updated. We found that, although agency and body ownership recovered in parallel, the recovery of body ownership was predicted by fast memories and that of agency was predicted by slow memories. Thus, the bodily self was represented in multiple motor memories with different dynamics. This finding demystifies the controversy about the causal relationship between body ownership and agency.

Predictability of Delayed Visual Feedback Under Rubber Hand Illusion Modulates Localization but Not Ownership of the Hand

The rubber hand illusion (RHI) is a perceptual illusion, whereby a fake hand is recognized as one’s own hand when a fake hand and felt real hand are stroked synchronously. RHI strength is mainly assessed using a questionnaire rating and proprioceptive drift (PD). PD is characterized by the proprioceptively sensed location of the participant’s own hand shifting toward the location of the fake hand after RHI. However, the relationship between the two measures of hand ownership and location remains controversial due to mixed findings: some studies report correlations between them, while others show that they are independent. Here, we demonstrated significant PD without RHI using delayed visual feedback. In this RHI study, video images of the fake hand were delivered to the subjects, and four delay intervals of visual feedback (80, 280, 480, and 680ms) were introduced. In four of six conditions, the delay interval was fixed throughout the condition. In the other two conditions, four delays were delivered in a predetermined order (i.e., serial condition; higher predictability) or in a pseudo-random order (i.e., random condition; low predictability). For the four conditions with a fixed delay, the questionnaire ratings and PD declined significantly when the delay interval exceeded circa 300ms. In both the serial and random conditions, no illusory ownership of the fake hand was reported in the questionnaire. In contrast, greater PD was found in the random condition but not in the serial condition. Our findings suggest that hand ownership and localization are caused by distinct multisensory integration processes.

Contribution of interaction force to the sense of hand ownership and the sense of hand agency

When performing willed actions, we have the unified and coherent experience of owning and controlling our body. Body ownership is believed to emerge from the integration of coherent multisensory signals, while agency is believed to emerge from the coherence between predicted and perceived outcomes of actions. As a consequence, body ownership and agency can both be modulated by multisensory conflicts. The contribution of active movement generation to ownership and agency has not been parametrically explored. Here, we investigated the contribution of interaction force between the agent and the environment to the sense of hand ownership (SO) and the sense of hand agency (SA). By combining robotics and virtual reality, we manipulated the sensorimotor and visual information during immersive scenarios to induce and quantify altered states of SO and SA. First, we demonstrated that SO and SA could be successfully manipulated by our experimental paradigms. Second, we showed that interaction force strongly contributes to SA, but to a lesser extent to SO. Finally, we showed that SO and SA interact beyond their common multisensory basis. Our results, based on two independent studies, provide a direct link between sensorimotor interactions and subjective body experience and demonstrate a new dissociation between SO and SA.

Distinct modulation of mu and beta rhythm desynchronization during observation of embodied fake hand rotation

The rubber hand illusion (RHI) is a phenomenon whereby participants recognize a fake hand as their own. Studies have examined the effects of observing fake hand movements after the RHI on brain sensorimotor activity, although results remain controversial. To address these discrepancies, we investigated the effects of observation of fake hand rotation after the RHI on sensorimotor mu (μ: 8-13 Hz) and beta (β: 15-25 Hz) rhythm event-related desynchronization (ERD) using electroencephalography (EEG). Questionnaire results and proprioceptive drift revealed that the RHI occurred in participants when their invisible hand and fake visible hand were stroked synchronously but not during asynchronous stroking. Independent component (IC) clustering from EEG data during movement observation identified three IC clusters, including the right sensorimotor, left sensorimotor, and left occipital cluster. In the right sensorimotor cluster, we observed distinct modulation of μ and β ERD during fake hand rotation. Illusory ownership over the fake hand enhanced μ ERD but inversely attenuated β ERD. Further, the extent of μ ERD correlated with proprioceptive drift, but not with questionnaire ratings, whereas the converse results were obtained for β ERD. No ownership-dependent ERD modulation was detected in the left sensorimotor cluster. Alpha (α: 8-13 Hz) rhythm ERD of the left occipital cluster was smaller in the synchronous condition than in the asynchronous condition, but α ERD was not correlated with questionnaire rating or drift. These findings suggest that observing embodied fake hand rotation induces distinct cortical processing in sensorimotor brain areas.

Did My Hand Move in a Mirror? Body Ownership Induced by the Mirror Hand Illusion

Although the illusion that the mirror image of a hand or limb could be recognized as a part of one’s body behind the mirror, the effect of adding tactile stimulation to this illusion remains unknown. We, therefore, examined how the timing of tactile stimulation affects the induction of body ownership on the mirror image. Twenty-one healthy, right-handed participants (mean age = 23.0 ± 1.0 years, no medical history of neurological and/or psychiatric disorders) were enrolled and a crossover design was adopted in this study. Participants’ right and left hands were placed on the front and back sides of the mirror, respectively, then they were asked to keep looking at their right hand in the mirror. All participants experienced two experiments; one was with tactile stimulation that was synchronized with the movement of a mirror image (synchronous condition), and the other one was with tactile stimulation that was not synchronized (asynchronous condition). The qualitative degree of body ownership for the mirrored hand was evaluated by a questionnaire. Proprioceptive drift (PD), an illusory shift of the felt position of the real hand toward the mirrored hand was used for quantitative evaluation of body ownership and measured at “baseline,” “immediately after stimulation,” “2 min after stimulation,” and “4 min after stimulation.” The results of the questionnaire revealed that some items of body ownership rating were higher in the synchronous condition than in the asynchronous condition (p < 0.05). We found that PD occurred from immediately after to 4 min after stimulation in both conditions (p < 0.01) and there was no difference in the results between the conditions. From the dissociation of these results, we interpreted that body ownership could be elicited by different mechanisms depending on the task demand. Our results may contribute to the understanding of the multisensory integration mechanism of visual and tactile stimulation during mirror illusion induction.

Tactile information counteracts the attenuation of rubber hand illusion attributable to increased visuo-proprioceptive divergence

Sense of body ownership is an immediate and distinct experience of one's body as belonging to oneself. While it is well-recognized that ownership feelings emerge from the integration of visual and somatosensory signals, the principles upon which they are integrated are still intensely debated. Here, we used the rubber hand illusion (RHI) to examine how the interplay of visual, tactile, and proprioceptive signals is governed depending on their spatiotemporal properties. For this purpose, the RHI was elicited in different conditions varying with respect to the extent of visuo-proprioceptive divergence (i.e., the distance between the real and fake hands) and differing in terms of the availability and spatiotemporal complexity of tactile stimulation (none, simple, or complex). We expected that the attenuating effect of distance on illusion strength will be more pronounced in the absence of touch (when proprioception gains relatively higher importance) and absent in the presence of complex tactile signals. Additionally, we hypothesized that participants with greater proprioceptive acuity-assessed using an elbow joint position discrimination task-will be less susceptible to the illusion, but only under the conditions of limited tactile stimulation. In line with our prediction, RHI was attenuated at the farthest distance only when tactile information was absent or simplified, but the attenuation was effectively prevented by the use of complex tactile stimulation-in this case, RHI was comparably vivid at both distances. However, passive proprioceptive acuity was not related to RHI strength in either of the conditions. The results indicate that complex-structured tactile signals can override the influence of proprioceptive signals in body attribution processes. These findings extend our understanding of body ownership by showing that it is primarily determined by informative cues from the most relevant sensory domains, rather than mere accumulation of multisensory evidence.

Effects of Orientation and Appearance of a Synchronously Moving Object on Hand Movements

Various devices have been developed to enable humans to control remote objects using active hand movements. However, it is still unclear how the visual characteristics of a synchronously moving object influences hand movements. This study investigates the effects of visual appearance and orientation of a hand-controlled object on hand movements using a novel visuomotor task. The motion of a visual image on a monitor reflected the participants’ right hand movements in the forwards-backwards direction, but not in the lateral direction (i.e., the lateral position of the image was fixed). Participants performed continuous goal-directed back and forth movements of the visual image for one minute. The image’s appearance (hand and arrow) and orientation (forward (FW), leftward (LW), and rightward (RW)) were manipulated. Unconscious lateral deviations (i.e., drift movements) of the participant’s hand during the task were evaluated. Regardless of appearance, the leftward and rightward image induced leftward and rightward drift movements, compared to the forward image. However, the modulation sizes were similar using arrow images, but not using hand images. Specifically, anatomically plausible hand images elicited greater drift movements than anatomically implausible images. This suggests that both orientation and appearance of a hand-controlled object influences hand movements according to stimulus-response compatibility and body-representation changes.

Virtual training leads to physical, cognitive and neural benefits in healthy adults

Physical activity, such as high-intensity intermittent aerobic exercise (HIE), can improve executive functions. Although performing strength or aerobic training might be problematic or not feasible for someone. An experimental situation where there is no actual movement, but the body shows physiological reactions, is during the illusion through immersive virtual reality (IVR). We aimed to demonstrate whether a virtual HIE-based intervention (vHIE) performed exclusively by the own virtual body has physical, cognitive, and neural benefits on the real body. 45 healthy young adults (cross-over design) experienced HIE training in IVR (i.e., the virtual body performed eight sets of 30 s of running followed by 30 s of slow walking, while the subject is completely still) in two random-ordered conditions (administered in two sessions one week apart): the virtual body is displayed in first-person perspective (1PP) or third-person perspective (3PP). During the vHIE, we recorded the heart rate and subjective questionnaires to confirm the effectiveness of the illusion; before and after vHIE, we measured cortical hemodynamic changes in the participants' left dorsolateral prefrontal cortex (lDLPFC) using the fNIRS device during the Stroop task to test our main hypothesis. Preliminary, we confirmed that the illusion was effective: during the vHIE in 1PP, subjects' heart rate increased coherently with the virtual movements, and they reported subjective feelings of ownership and agency. Primarily, subjects were faster in executing the Stroop task after the vHIE in 1PP; also, the lDLPFC activity increased coherently. Clinically, these results might be exploited to train cognition and body simultaneously. Theoretically, we proved that the sense of body ownership and agency can affect other parameters, even in the absence of actual movements.

Mechanotactile Sensory Feedback Improves Embodiment of a Prosthetic Hand During Active Use

There have been several advancements in the field of myoelectric prostheses to improve dexterity and restore hand grasp patterns for persons with upper limb loss, including robust control strategies, novel sensory feedback, and multifunction prosthetic terminal devices. Although these advancements have shown to improve prosthesis performance, a key element that may further improve acceptance is often overlooked. Embodiment, which encompasses the feeling of owning, controlling and locating the device without the need to constantly look at it, has been shown to be affected by sensory feedback. However, the specific aspects of embodiment that are influenced are not clearly understood, particularly when a prosthesis is actively controlled. In this work, we used a sensorized simulated prosthesis in able-bodied participants to investigate the contribution of sensory feedback, active motor control, and the combination of both to the components of embodiment; using a common methodology in the literature, namely the rubber hand illusion (RHI). Our results indicate that (1) the sensorized simulated prosthesis may be embodied by able-bodied users in a similar fashion as prosthetic devices embodied by persons with upper limb amputation, and (2) mechanotactile sensory feedback might not only be useful for improving certain aspects of embodiment, i.e., ownership and location, but also may have a modulating effect on other aspects, namely sense of agency, when provided asynchronously during active motor control tasks. This work may allow us to further investigate and manipulate factors contributing to the complex phenomenon of embodiment in relation to active motor control of a device, enabling future study of more precise quantitative measures of embodiment that do not rely as much on subjective perception.

Toward Enhanced Teleoperation Through Embodiment

Telerobotics aims to transfer human manipulation skills and dexterity over an arbitrary distance and at an arbitrary scale to a remote workplace. A telerobotic system that is transparent enables a natural and intuitive interaction. We postulate that embodiment (with three sub-components: sense of ownership, agency, and self-location) of the robotic system leads to optimal perceptual transparency and increases task performance. However, this has not yet been investigated directly. We reason along four premises and present findings from the literature that substantiate each of them: (1) the brain can embody non-bodily objects (e.g., robotic hands), (2) embodiment can be elicited with mediated sensorimotor interaction, (3) embodiment is robust against inconsistencies between the robotic system and the operator's body, and (4) embodiment positively correlates to dexterous task performance. We use the predictive encoding theory as a framework to interpret and discuss the results reported in the literature. Numerous previous studies have shown that it is possible to induce embodiment over a wide range of virtual and real extracorporeal objects (including artificial limbs, avatars, and android robots) through mediated sensorimotor interaction. Also, embodiment can occur for non-human morphologies including for elongated arms and a tail. In accordance with the predictive encoding theory, none of the sensory modalities is critical in establishing ownership, and discrepancies in multisensory signals do not necessarily lead to loss of embodiment. However, large discrepancies in terms of multisensory synchrony or visual likeness can prohibit embodiment from occurring. The literature provides less extensive support for the link between embodiment and (dexterous) task performance. However, data gathered with prosthetic hands do indicate a positive correlation. We conclude that all four premises are supported by direct or indirect evidence in the literature, suggesting that embodiment of a remote manipulator may improve dexterous performance in telerobotics. This warrants further implementation testing of embodiment in telerobotics. We formulate a first set of guidelines to apply embodiment in telerobotics and identify some important research topics.

Sensorimotor and Posterior Brain Activations During the Observation of Illusory Embodied Fake Hand Movement

In the rubber hand illusion (RHI), the subject recognizes a fake hand as his or her own. We recently found that the observation of embodied fake hand movement elicited mu-rhythm (8–13 Hz) desynchronization on electroencephalography (EEG), suggesting brain activation in the sensorimotor regions. However, it is known that mu-rhythm desynchronization during action observation is confounded with occipital alpha-rhythm desynchronization, which is modulated by attention. This study examined the independence of brain activities in the sensorimotor and occipital regions relating to the movement observation under the RHI. The invisible participant’s left and fake right hands were stroked simultaneously, which was interrupted by unexpected fake hand movements. A mirror-reversed image of the fake hand was shown on a monitor in front of the participant with a delay of 80, 280, or 480 ms. Illusion strength decreased as a function of the delay. EEG independent component analysis (ICA) and ICA clustering revealed six clusters with observation-induced desynchronization of 8–13 Hz frequency band. In the right sensorimotor cluster, mu-rhythm desynchronization was the greatest under the 80-ms delay, while alpha-rhythm desynchronization of the occipital clusters did not show delay-dependence. These results suggest that brain activation in the sensorimotor areas (i.e., mu-rhythm desynchronization) induced by embodied fake hand movement is independent of that in the occipital areas (alpha-rhythm desynchronization).

The Relationship Between the Virtual Hand Illusion and Motor Performance

Bodily self-consciousness consists of agency (i.e., the feeling of controlling one’s actions and causing external events) and body ownership (i.e., the feeling that one’s body belongs to one’s self). If a visual presentation of a virtual (fake) hand matches the active movement of a real hand, both the agency and body ownership of the virtual hand are induced [i.e., the active virtual hand illusion (VHI)]. However, previous active VHI studies have rarely considered the effects of goal-related movement errors (i.e., motor performance) on the senses of agency and ownership. Hence, the current study aimed to clarify the relationship between the active VHI and motor performance. To induce the VHI, 18 healthy subjects (three men and 15 women; 20.7 ± 7.3 years) were required to continuously move a virtual hand around a circle at a predetermined speed (i.e., spatial and temporal goals) using their active hand movements. While moving the virtual hand actively, five visual feedback delays were introduced: 90, 210, 330, 450, and 570 ms. It was found that the subjective ratings of both the agency and body ownership of the virtual hand decreased as a function of the delay intervals, whereas most of the spatial and temporal movement errors linearly increased. Using multiple regression analyses, we examined whether the agency and ownership ratings could be explained effectively by both the delay and movement errors. The results demonstrated that the agency was determined not only by the delay but also by the movement variability, whereas the body ownership was mostly determined by the delay. These findings suggest a possibility that the goal-related motor performance of the active VHI influences the agency judgment more strongly, while its effect on the ownership judgment is weaker.

Kinesthetic Senses

The kinesthetic senses are the senses of position and movement of the body, senses we are aware of only on introspection. A method used to study kinesthesia is muscle vibration, which engages afferents of muscle spindles to trigger illusions of movement and changed position. When vibrating elbow flexors, it generates sensations of forearm extension, when vibrating extensors, sensations of forearm flexion. Vibrating the elbow joint produces no illusion. Vibrating flexors and extensors together at the same frequency also produces no illusion, because what is perceived is the signal difference between antagonist muscles of each arm and between arms. The size of the illusion depends on how the muscle has been conditioned beforehand, due to a property of muscle called thixotropy. When measuring the illusion, blindfolded subjects may carry out a matching or pointing task. In pointing, signals from muscle spindles are less important than in matching. Afferent signals from kinesthetic receptors project to areas of somatosensory cortex to generate sensations of detection and location. This is referred to the body model, which provides information about size and shape of body parts. Kinesthesia, together with vision and touch, is associated with the sense of body ownership. All three can combine or each, on its own, can generate ownership. Related is the sense of agency, the sense of being responsible for one's own actions. In recent times, much progress has been made using neuroimaging techniques to identify the various areas of the brain likely to be responsible for generating these sensations. © 2017 American Physiological Society. Compr Physiol 8:1157-1183, 2018.

Revisiting the link between body and agency: visual movement congruency enhances intentional binding but is not body-specific

Embodiment and agency are key aspects of how we perceive ourselves that have typically been associated with independent mechanisms. Recent work, however, has suggested that these mechanisms are related. The sense of agency arises from recognising a causal influence on the external world. This influence is typically realised through bodily movements and thus the perception of the bodily self could also be crucial for agency. We investigated whether a key index of agency - intentional binding - was modulated by body-specific information. Participants judged the interval between pressing a button and a subsequent tone. We used virtual reality to manipulate two aspects of movement feedback. First, form: participants viewed a virtual hand or sphere. Second, movement congruency: the viewed object moved congruently or incongruently with the participant's hidden hand. Both factors, form and movement congruency, significantly influenced embodiment. However, only movement congruency influenced intentional binding. Binding was increased for congruent compared to incongruent movement feedback irrespective of form. This shows that the comparison between viewed and performed movements provides an important cue for agency, whereas body-specific visual form does not. We suggest that embodiment and agency mechanisms both depend on comparisons across sensorimotor signals but that they are influenced by distinct factors.

The neuroscience of body memory: From the self through the space to the others

Our experience of the body is not direct; rather, it is mediated by perceptual information, influenced by internal information, and recalibrated through stored implicit and explicit body representation (body memory). This paper presents an overview of the current investigations related to body memory by bringing together recent studies from neuropsychology, neuroscience, and evolutionary and cognitive psychology. To do so, in the paper, I explore the origin of representations of human body to elucidate their developmental process and, in particular, their relationship with more explicit concepts of self. First, it is suggested that our bodily experience is constructed from early development through the continuous integration of sensory and cultural data from six different representations of the body, i.e., the Sentient Body (Minimal Selfhood), the Spatial Body (Self Location), the Active Body (Agency), the Personal Body (Whole Body Ownership - Me); the Objectified Body (Objectified Self - Mine), and the Social Body (Body Satisfaction - Ideal Me). Then, it is suggested that these six representations can be combined in a coherent supramodal representation, i.e. the "body matrix", through a predictive, multisensory processing activated by central, top-down, attentional processes. From an evolutionary perspective, the main goal of the body matrix is to allow the self to protect and extend its boundaries at both the homeostatic and psychological levels. From one perspective, the self extends its boundaries (peripersonal space) through the enactment and recognition of motor schemas. From another perspective, the body matrix, by defining the boundaries of the body, also defines where the self is present, i.e., in the body that is processed by the body matrix as the most likely to be its one, and in the space surrounding it. In the paper I also introduce and discuss the concept of "embodied medicine": the use of advanced technology for altering the body matrix with the goal of improving our health and well-being.