From Embodiment of a Point-Light Display in Virtual Reality to Perception of One's Own Movements

Women show enhanced proprioceptive target estimation through visual-proprioceptive conflict resolution

To form a unified and coherent perception of the organism's state and its relationship with the surrounding environment, the nervous system combines information from various sensory modalities through multisensory integration processes. Occasionally, data from two or more sensory channels may provide conflicting information. This is particularly evident in experiments using the mirror-guided drawing task and the mirror-box illusion, where there is conflict between positional estimates guided by vision and proprioception. This study combined two experimental protocols (the mirror-box and the mirror-guided drawing tasks) to examine whether the learned resolution of visuo-proprioceptive conflicts in the mirror-guided drawing task would improve proprioceptive target estimation of men and women during the mirror-box test. Our results confirm previous findings of visual reaching bias produced by the mirror-box illusion and show that this effect is progressively reduced by improvement in the mirror drawing task performance. However, this was only observed in women. We discuss these findings in the context of possible gender differences in multisensory integration processes as well as in embodiment.

Body ownership and kinaesthetic illusions: Dissociated bodily experiences for distinct levels of body consciousness?

Seeing an embodied humanoid avatar move its arms can induce in the observer the illusion that its own (static) arms are moving accordingly, the kinematic signals emanating from this avatar thus being considered like those from the biological body. Here, we investigated the causal relationship between these kinaesthetic illusions and the illusion of body ownership, manipulated through visuomotor synchronisation. The results of two experiments revealed that the sense of body ownership over an avatar seen from a first-person perspective was intimately linked to visuomotor synchrony. This was not the case for kinaesthetic illusions indicating that when superimposed on the biological body, the avatar is inevitably treated at the sensorimotor level as one's own body, whether consciously considered as such or not. The question of whether these two bodily experiences (body ownership and kinaesthetic illusion) are underpinned by distinct representations, the body image, and the body schema, is discussed.

Does optic flow provide information about actions?

Optic flow, the pattern of light generated in the visual field by motion of objects and the observer's body, serves as information that underwrites perception of events, actions, and affordances. This visual pattern informs the observer about their own actions in relation to their surroundings, as well as those of others. This study explored the limits of action detection for others as well as the role of optic flow. First-person videos were created using camera recordings of the actor's perspective as they performed various movements (jumping jacks, jumping, squatting, sitting, etc.). In three experiments participants attempted to detect the action from first-person video footage using open ended responses (Experiment 1), forced-choice responses (Experiment 2), and a match-to-sample paradigm (Experiment 3). It was discovered that some actions are more difficult to detect than others. When the task was challenging (Experiment 1) athletes were more accurate, but this was not the case in Experiments 2 and 3. All actions were identified above chance level across viewpoints, suggesting that invariant information was detected and used to perform the task.

The impact of visual display of human motion on observers' perception of music performance

In investigating the influence of body movement in multimodal perception, human motion displays are frequently used as a means of visual standardization and control of external confounders. However, no principle is established regarding the selection of an adequate display for specific study purposes. The aim of this study was to evaluate the effects of adopting 4 visual displays (point-light, stick figure, body mass, skeleton) on the observers' perception of music performances in 2 expressive conditions (immobile, projected expressiveness). Two hundred eleven participants rated 8 audio-visual samples in expressiveness, match between movement and music, and overall evaluation. The results revealed significant isolated main effects of visual display and expressive condition on the observers' ratings (in both, p < 0.001), and interaction effects between the two factors (p < 0.001). Displays closer to a human form (mostly skeleton, sometimes body mass) exponentiated the evaluations of expressiveness and music-movement match in the projected expressiveness condition, and of overall evaluation in the immobile condition; the opposite trend occurred with the simplified motion display (stick figure). Projected expressiveness performances were higher rated than immobile performances. Although the expressive conditions remained distinguishable across displays, the more complex ones potentiated the attribution of subjective qualities. We underline the importance of considering the variable display as an influencing factor in perceptual studies.

Effects of virtual body-representation on motor skill learning

Motor learning is often hindered or facilitated by visual information from one's body and its movement. However, it is unclear whether visual representation of the body itself facilitates motor learning. Thus, we tested the effects of virtual body-representation on motor learning through a virtual reality rotary pursuit task. In the task, visual feedback on participants' movements was identical, but virtual body-representation differed by dividing the experimental conditions into three conditions: non-avatar, non-hand avatar, and hand-shaped avatar. We measured the differences in the rate of motor learning, body-ownership, and sense of agency in the three conditions. Although there were no differences in body-ownership and sense of agency between the conditions, the hand-shaped avatar condition was significantly superior to the other conditions in the rate of learning. These findings suggest that visually recognizing one's body shape facilitates motor learning.

Involvement of visual signals in kinaesthesia: A virtual reality study

Body movements are invariably accompanied by various proprioceptive, visual, tactile and/or motor signals. It is therefore difficult to completely dissociate these various signals from each other in order to study their specific involvement in the perception of movement (kinaesthesia). Here, we manipulated visual motion signals in a virtual reality display by using a humanoid avatar. The visual signals of movement could therefore be manipulated freely, relative to the participant's actual movement or lack of movement. After an embodiment phase in which the avatar's movements were coupled to the participant's voluntary movements, kinaesthetic illusions were evoked by moving the avatar's right forearm (flexion or extension) while the participant's right arm remained static. The avatar's left forearm was hidden from view. In parallel, somaesthetic signals could be masked by agonist-antagonist co-vibration or be amplified (by agonist vibration only or antagonist vibration only) so that the real impact of visual cues of movement in kinaesthesia could be studied. In a study of 24 participants, masking the somaesthetic signals (which otherwise provide signals indicating that the arm is static) was associated with a greater intensity and shorter latency of the visually evoked illusions. These results confirm the importance of carefully considering somaesthetic signals when assessing the contribution of vision to kinaesthesia. The use of a combination of virtual reality and somaesthetic signal manipulation might be of clinical value.

Body and the Senses in Spatial Experience: The Implications of Kinesthetic and Synesthetic Perceptions for Design Thinking

Human perception has long been a critical subject of design thinking. While various studies have stressed the link between thinking and acting, particularly in spatial experience, the term "design thinking" seems to disconnect conceptual thinking from physical expression or process. Spatial perception is multimodal and fundamentally bound to the body that is not a mere receptor of sensory stimuli but an active agent engaged with the perceivable environment. The body apprehends the experience in which one's kinesthetic engagement and knowledge play an essential role. Although design disciplines have integrated the abstract, metaphoric, and visual aspects of the body and its movement into conceptual thinking, studies have pointed out that design disciplines have emphasized visuality above the other sensory domains and heavily engaged with the perception of visual configurations, relying on the Gestalt principles. Gestalt psychology must be valued for its attention to a whole. However, the theories of design elements and principles over-empathizing such visuality posit the aesthetics of design mainly as visual value and understate other sensorial and perceptual aspects. Although the visual approach may provide a practical means to represent and communicate ideas, a design process heavily driven by visuality can exhibit weaknesses undermining certain aspects of spatial experience despite the complexity. Grounded in Merleau-Ponty's notion of multisensory perception, this article discusses the relationship between body awareness and spatial perception and its implication for design disciplines concerning built environments. Special attention is given to the concepts of kinesthetic and synesthetic phenomena known as multisensory and cross-sensory, respectively. This discussion integrates the corporeal and spatiotemporal realms of human experience into the discourse of kinesthetic and synesthetic perceptions. Based on the conceptual, theoretical, and precedent analyses, this article proposes three models for design thinking: Synesthetic Translation, Kinesthetic Resonance, and Kinesthetic Engagement. To discuss the concepts rooted in action-based perception and embodied cognition, this study borrows the neurological interpretation of haptic perception, interoception, and proprioception of space. This article suggests how consideration of the kinesthetic or synesthetic body can deepen and challenge the existing models of the perceptual aspects of environmental psychology adopted in design disciplines.

Multisensory integration of visual cues from first- to third-person perspective avatars in the perception of self-motion

In the perception of self-motion, visual cues originating from an embodied humanoid avatar seen from a first-person perspective (1^st-PP) are processed in the same way as those originating from a person's own body. Here, we sought to determine whether the user's and avatar's bodies in virtual reality have to be colocalized for this visual integration. In Experiment 1, participants saw a whole-body avatar in a virtual mirror facing them. The mirror perspective could be supplemented with a fully visible 1^st-PP avatar or a suggested one (with the arms hidden by a virtual board). In Experiment 2, the avatar was viewed from the mirror perspective or a third-person perspective (3^rd-PP) rotated 90° left or right. During an initial embodiment phase in both experiments, the avatar's forearms faithfully reproduced the participant's real movements. Next, kinaesthetic illusions were induced on the static right arm from the vision of passive displacements of the avatar's arms enhanced by passive displacement of the participant's left arm. Results showed that this manipulation elicited kinaesthetic illusions regardless of the avatar's perspective in Experiments 1 and 2. However, illusions were more likely to occur when the mirror perspective was supplemented with the view of the 1st-PP avatar's body than with the mirror perspective only (Experiment 1), just as they are more likely to occur in the latter condition than with the 3^rd-PP (Experiment 2). Our results show that colocalization of the user's and avatar's bodies is an important, but not essential, factor in visual integration for self-motion perception.

Functional properties of extended body representations in the context of kinesthesia

A person's internal representation of his/her body is not fixed. It can be substantially modified by neurological injuries and can also be extended (in healthy participants) to incorporate objects that have a corporeal appearance (such as fake body segments, e.g. a rubber hand), virtual whole bodies (e.g. avatars), and even objects that do not have a corporeal appearance (e.g. tools). Here, we report data from patients and healthy participants that emphasize the flexible nature of body representation and question the extent to which incorporated objects have the same functional properties as biological body parts. Our data shed new light by highlighting the involvement of visual motion information from incorporated objects (rubber hands, full body avatars and hand-held tools) in the perception of one's own movement (kinesthesia). On the basis of these findings, we argue that incorporated objects can be treated as body parts, especially when kinesthesia is involved.

The impact of embodying an "elderly" body avatar on motor imagery

When an individual embodies an avatar, the latter's characteristics or stereotype can change the individual's behavior and attitudes; this is known as the Proteus effect. Here, we looked at whether the embodiment of an avatar resembling an elderly adult (seen from a first-person perspective and facing a virtual mirror) changed mentally represented physical activity in a motor imagery task performed by young adult participants (N = 52). To ensure that the impact of embodiment of an elderly avatar on the motor imagery task was not influenced by a potentially confounded stereotype assimilation effect (due to the mere presence of an avatar), a "young" avatar and an "elderly" avatar were always present together in the virtual environment-even though only one (the self-avatar) was embodied at a given time. We found that it took longer for the participants to perform the motor imagery task with the elderly self-avatar than with the young self-avatar. The more negative the participant's beliefs about motor activity in the elderly, the greater the observed effect of the avatar on motor imagery performance. We conclude that knowledge about the characteristics of an embodied avatar can modify the subject's level of mentally represented physical activity.

Exploring Visual Perceptions of Spatial Information for Wayfinding in Virtual Reality Environments

Human cognitive processes in wayfinding may differ depending on the time taken to accept visual information in environments. This study investigated users’ wayfinding processes using eye-tracking experiments, simulating a complex cultural space to analyze human visual movements in the perception and the cognitive processes through visual perception responses. The experiment set-up consisted of several paths in COEX Mall, Seoul—from the entrance of the shopping mall Starfield to the Star Hall Library to the COEX Exhibition Hall—using visual stimuli created by virtual reality (four stimuli and a total of 60 seconds stimulation time). The participants in the environment were 24 undergraduate or graduate students, with an average age of 24.8 years. Participants’ visual perception processes were analyzed in terms of the clarity and the recognition of spatial information and the activation of gaze fixation on spatial information. That is, the analysis of the visual perception process was performed by extracting “conscious gaze perspective” data comprising more than 50 consecutive 200 ms continuous gaze fixations; “visual understanding perspective” data were also extracted for more than 300 ms of continuous gaze fixation. The results show that the methods for analyzing the gaze data may vary in terms of processing, analysis, and scope of the data depending on the purpose of the virtual reality experiments. Further, they demonstrate the importance of what purpose statements are given to the subject during the experiment and the possibility of a technical approach being used for the interpretation of spatial information.

Studying the visual brain in its natural rhythm

How the brain fluidly orchestrates visual behavior is a central question in cognitive neuroscience. Researchers studying neural responses in humans and nonhuman primates have mapped out visual response profiles and cognitive modulation in a large number of brain areas, most often using pared down stimuli and highly controlled behavioral paradigms. The historical emphasis on reductionism has placed most studies at one pole of an inherent trade-off between strictly controlled experimental variables and open designs that monitor the brain during its natural modes of operation. This bias toward simplified experiments has strongly shaped the field of visual neuroscience, with little guarantee that the principles and concepts established within that framework will apply more generally. In recent years, a growing number of studies have begun to relax strict experimental control with the aim of understanding how the brain responds under more naturalistic conditions. In this article, we survey research that has explicitly embraced the complexity and rhythm of natural vision. We focus on those studies most pertinent to understanding high-level visual specializations in brains of humans and nonhuman primates. We conclude that representationalist concepts borne from conventional visual experiments fall short in their ability to capture the real-life visual operations undertaken by the brain. More naturalistic approaches, though fraught with experimental and analytic challenges, provide fertile ground for neuroscientists seeking new inroads to investigate how the brain supports core aspects of our daily visual experience.