The Neurological Traces of Look-Alike Avatars

The contribution of body perception to self-identity: an event-related potential study

This study used electroencephalography (EEG) and personalized avatars to investigate the neural mechanisms underlying personal identity perception. Compound avatar images combining participants' own faces and bodies, as well as those of others, were generated from photographs. Participants underwent an embodiment training for each avatar type in a virtual reality environment, where they controlled the avatar's actions during physical exercise tasks. Subjective assessments by participants confirmed a stronger identification with avatars representing their own identity compared to those representing others. Analysis of event-related potentials (ERPs) evoked by viewing the avatar revealed that avatars representing the participants' self-identity elicited weaker N2 and P1 responses compared to avatars representing other identities. No significant effects on N170 responses were observed. Control conditions utilizing avatars with modified body characteristics confirmed that the reduction in N2 amplitude was specifically related to identity perception rather than variations in visual body size. These findings suggest that the perception of self-identity occurs rapidly, within ∼200 ms, indicating the integration of visual face and body information into identity representation at an early stage.

The promise of the metaverse in mental health: the new era of MEDverse

Since Mark Zuckerberg's announcement about the development of new three-dimensional virtual worlds for social communication, a great debate has been raised about the promise of such a technology. The metaverse, a term formed by combining meta and universe, could open a new era in mental health, mainly in psychological disorders, where the creation of a full-body illusion via digital avatar could promote healthcare and personal well-being. Patients affected by body dysmorphism symptoms (i.e., eating disorders), social deficits (i.e. autism) could greatly benefit from this kind of technology. However, it is not clear which advantage the metaverse would have in treating psychological disorders with respect to the well-known and effective virtual reality (VR) exposure therapy. Indeed, in the last twenty years, a plethora of studies have demonstrated the effectiveness of VR technology in reducing symptoms of pain, anxiety, stress, as well as, in improving cognitive and social skills. We hypothesize that the metaverse will offer more opportunities, such as a more complex, virtual realm where sensory inputs, and recurrent feedback, mediated by a "federation" of multiple technologies - e.g., artificial intelligence, tangible interfaces, Internet of Things and blockchain, can be reinterpreted for facilitating a new kind of communication overcoming self-body representation. However, nowadays a clear starting point does not exist. For this reason, it is worth defining a theoretical framework for applying this new kind of technology in a social neuroscience context for developing accurate solutions to mental health in the future.

Using Facial Animation to Increase the Enfacement Illusion and Avatar Self-Identification

Through avatar embodiment in Virtual Reality (VR) we can achieve the illusion that an avatar is substituting our body: the avatar moves as we move and we see it from a first person perspective. However, self-identification, the process of identifying a representation as being oneself, poses new challenges because a key determinant is that we see and have agency in our own face. Providing control over the face is hard with current HMD technologies because face tracking is either cumbersome or error prone. However, limited animation is easily achieved based on speaking. We investigate the level of avatar enfacement, that is believing that a picture of a face is one's own face, with three levels of facial animation: (i) one in which the facial expressions of the avatars are static, (ii) one in which we implement lip-sync motion and (iii) one in which the avatar presents lip-sync plus additional facial animations, with blinks, designed by a professional animator. We measure self-identification using a face morphing tool that morphs from the face of the participant to the face of a gender matched avatar. We find that self-identification on avatars can be increased through pre-baked animations even when these are not photorealistic nor look like the participant.

Avatar Embodiment Enhances Haptic Confidence on the Out-of-Body Touch Illusion

In the real world, our bodies influence how we perceive ourselves and how others perceive us. Our body can also affect estimations of object sizes and distances. But how does our body affect our haptic experience? Here, we examined the modulation of a visuo-haptic illusion of touch on a virtual stick in virtual reality, when participants were embodied in an avatar and when they were not. During the experiments participants (n = 49) received successions of three taps delivered from two independent controllers while they saw visual stimuli presented sequentially along the virtual stick. The stimulation pattern resulted in a robust illusion of tapping directly on the virtual stick. After each trial, participants were asked to report where they perceived the taps. We found that participants in both the body and no-body conditions displaced the second tap toward the center of the stick, and reported similar levels of certainty about their reported location. However, the illusion of touch on the stick, as measured by the reported location of the tap, was significantly stronger for those who had a virtual body than those who did not. Therefore, our study shows that avatar embodiment can change haptic perception.

From Robot to Virtual Doppelganger: Impact of Visual Fidelity of Avatars Controlled in Third-Person Perspective on Embodiment and Behavior in Immersive Virtual Environments

This study presents the second phase of a series of experiments investigating the impact of avatar visual fidelity on the sense of embodiment and users' behavior in immersive virtual environments. Our main focus concerns the similarity between users and avatars, a factor known as truthfulness. Our experiment requires the participants to control three avatars using a third-person perspective: a robot, a suit and their virtual doppelganger (virtual representation of the self). In order to analyze users' reactions and strategies, each task of the scenario of the virtual reality application can potentially affect the integrity of their characters. Our results revealed that ownership, one of the three factors of the sense of embodiment, is higher for the participants controlling their self-representation than with abstract representations. Furthermore, avatar visual fidelity seems to affect users' subjective experience, half of the panel reported having different behavior depending on the controlled character. Abstract representations allow the users to adopt more risky behaviors, while self-representations maintain a connection with the real world and encourage users to preserve the integrity of their avatar.

Malleability of the self: electrophysiological correlates of the enfacement illusion

Self-face representation is fundamentally important for self-identity and self-consciousness. Given its role in preserving identity over time, self-face processing is considered as a robust and stable process. Yet, recent studies indicate that simple psychophysics manipulations may change how we process our own face. Specifically, experiencing tactile facial stimulation while seeing similar synchronous stimuli delivered to the face of another individual seen as in a mirror, induces 'enfacement' illusion, i.e. the subjective experience of ownership of the other's face and a bias in attributing to the self, facial features of the other person. Here we recorded visual Event-Related Potentials elicited by the presentation of self, other and morphed faces during a self-other discrimination task performed immediately after participants received synchronous and control asynchronous Interpersonal Multisensory Stimulation (IMS). We found that self-face presentation after synchronous as compared to asynchronous stimulation significantly reduced the late positive potential (LPP; 450-750 ms), a reliable electrophysiological marker of self-identification processes. Additionally, enfacement cancelled out the differences in LPP amplitudes produced by self- and other-face during the control condition. These findings represent the first direct neurophysiological evidence that enfacement may affect self-face processing and pave the way to novel paradigms for exploring defective self-representation and self-other interactions.

Eyewitness Memory in Face-to-Face and Immersive Avatar-to-Avatar Contexts

Technological advances offer possibilities for innovation in the way eyewitness testimony is elicited. Typically, this occurs face-to-face. We investigated whether a virtual environment, where interviewer and eyewitness communicate as avatars, might confer advantages by attenuating the social and situational demands of a face-to-face interview, releasing more cognitive resources for invoking episodic retrieval mode. In conditions of intentional encoding, eyewitnesses were interviewed 48 h later, either face-to-face or in a virtual environment (N = 38). Participants in the virtual environment significantly outperformed those interviewed face-to-face on all episodic performance measures - improved correct reporting reduced errors, and increased accuracy. Participants reported finding it easier to admit not remembering event information to the avatar, and finding the avatar easier to talk to. These novel findings, and our pattern of retrieval results indicates the potential of avatar-to-avatar communication in virtual environments, and provide impetus for further research investigating eyewitness cognition in contemporary retrieval contexts.

Model of Illusions and Virtual Reality

In Virtual Reality (VR) it is possible to induce illusions in which users report and behave as if they have entered into altered situations and identities. The effect can be robust enough for participants to respond “realistically,” meaning behaviors are altered as if subjects had been exposed to the scenarios in reality. The circumstances in which such VR illusions take place were first introduced in the 80's. Since then, rigorous empirical evidence has explored a wide set of illusory experiences in VR. Here, we compile this research and propose a neuroscientific model explaining the underlying perceptual and cognitive mechanisms that enable illusions in VR. Furthermore, we describe the minimum instrumentation requirements to support illusory experiences in VR, and discuss the importance and shortcomings of the generic model.