Adapting Virtual Embodiment Through Reinforcement Learning

Examining Effects of Technique Awareness on the Detection of Remapped Hands in Virtual Reality

Input remapping techniques have been widely explored to allow users in virtual reality to exceed both their own physical abilities, the limitations of physical space, or to facilitate interactions with real-world objects. Often considered is how these techniques can be applied to achieve maximum utility, but still be undetectable to users to maintain a sense of immersion and presence. Existing psychophysical methods used to determine these detection thresholds have known limitations: they are highly conservative lower bounds for detection and do not account for complex usage of the technique. Our work describes and evaluates a method for estimating detection that reduces these limitations and yields meaningful upper bounds. We present the findings of our work where we apply this method to a well-explored hand motion scaling technique. In wholly unaware cases, we determined that users may detect their hand speed as abnormal at around 3.37 times the normal speed, compared to a scale factor of 1.47 that was estimated using traditional methods when users knew the motion scaling was occurring. A considerable number of participants in unaware cases (12 of 56) never detected their hand speed increasing at all, even at the maximum scale factor of 5.0. The study demonstrates just how conservative the thresholds generated by traditional psychophysical methods can be compared to detection during naive usage, and our method can be modified and applied easily to other techniques.

In Case of Doubt, One Follows One's Self: The Implicit Guidance of the Embodied Self-Avatar

The sense of embodiment in virtual reality (VR) is commonly understood as the subjective experience that one's physical body is substituted by a virtual counterpart, and is typically achieved when the avatar's body, seen from a first-person view, moves like one's physical body. Embodiment can also be experienced in other circumstances (e.g., in third-person view) or with imprecise or distorted visuo-motor coupling. It was moreover observed, in various cases of small or progressive temporal and spatial manipulations of avatars' movements, that participants may spontaneously follow the movement shown by the avatar. The present work investigates whether, in some specific contexts, participants would follow what their avatar does even when large movement discrepancies occur, thereby extending the scope of understanding of the self-avatar follower effect beyond subtle changes of motion or speed manipulations. We conducted an experimental study in which we introduced uncertainty about which movement to perform at specific times and analyzed participants' movements and subjective feedback after their avatar showed them an incorrect movement. Results show that, when in doubt, participants were influenced by their avatar's movements, leading them to perform that particular error twice more often than normal. Importantly, results of the embodiment score indicate that participants experienced a dissociation with their avatar at those times. Overall, these observations not only demonstrate the possibility of provoking situations in which participants follow the guidance of their avatar for large motor distortions, despite their awareness about the avatar movement disruption and on the possible influence it had on their choice, and, importantly, exemplify how the cognitive mechanism of embodiment is deeply rooted in the necessity of having a body.

Customizing the human-avatar mapping based on EEG error related potentials

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

Advances in neuroRehabilitation of TREM2-related dementia: A case report on a novel multimodal approach using virtual reality

RATIONABLE

The aim of this study is to investigate the effects of an advanced neuroRehabilitation protocol using virtual reality in the treatment of a patient with fronto- temporal dementia due to TREM2 mutation.

PATIENTS CONCERN

A 41-year-old caucasian male, affected by Nasu-Hakola Disease (NHD), presented a 1-year history of change in behavioral and cognitive functioning, before our observation. The onset of the disease was characterized by severe pain in the lower limbs and knees with limitations in the performance of daily life activities.

DIAGNOSIS

Motor and cognitive deficits in NHD.

INTERVENTIONS

As the patient was in a chronic phase, to manage his cognitive and motor status, we decided to treat him by using a specific rehabilitation protocol, including 2 different types of training: conventional cognitive and motor treatment and a combined advanced approach using the virtual reality rehabilitation system (VRRS). The two protocols were separated by 4 weeks of rest, to avoid/reduce a cumulative effect. The patient's cognitive and motor profile was assessed three times: that is before (at T0) and after (at T1) the conventional training as well as at the beginning (T2) and at the end of the combined experimental approach (T3).

OUTCOMES

After the combined therapeutic approach with the virtual reality rehabilitation system, we observed a significant reduction in anxiety, apathy, indifference and depressive symptoms with a more evident motor improvement involving the head and the trunk control.

LESSONS

Virtual reality can be considered a promising tool for the motor and cognitive rehabilitation of rare neurodegenerative disorders, including patients with NHD.