Quantitative measures of presence in virtual environments: the roles of attention and spatial comprehension

Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

The effects of age on objective and subjective recollection after visiting a virtual apartment

While aging has been associated with decreased retrieval of episodic memory details, subjective ratings about memory quality seem to remain stable. This suggests that subjective memory judgments are based on different information according to age. Here, we tested the hypothesis that older people would rather base their subjective judgments on the retrieval of personal elements (such as emotions and thoughts), whereas younger people would rather base their judgments on the retrieval of event-related elements (such as time, place, and perceptual details). Sixty participants (20 to 79 years old) performed eight actions in a virtual apartment and were then asked to verbally recall each action with a maximum of associated elements and to rate the subjective quality of their memories. The elements reported were classified into "person-related" and "event-related" categories. Executive functions, memory performance on traditional memory tasks, and subjects' perception of memory functioning were also evaluated. Results revealed that aging was associated with reduced retrieval of event-related elements, which was explained by decreasing executive resources. However, age did not affect the retrieval of person-related elements, and the subjective memory judgments of older people were not based on these elements to a greater extent than those of younger people. Finally, our results highlight the value of virtual reality (VR) in memory evaluations since subjects' perception of memory functioning was associated with their performance in the VR task but not in traditional memory tasks.

Enriched environments enhance the development of explicit memory in an incidental learning task

Learning, rendered in an implicit (unconscious) or explicit (conscious) way, is a crucial part of our daily life. Different factors, like attention or motivation, influence the transformation from implicit to explicit memory. Via virtual reality a lively and engaging surrounding can be created, whereby motivational processes are assumed to be a vital part of the transition from implicit to explicit memory. In the present study, we tested the impact of an enriched virtual reality compared to two conventional, non-enriched 2D-computer-screen based tasks on implicit to explicit memory transformation, using an audio-visual sequential association task. We hypothesized, that the immersive nature of the VR surrounding enhances the transfer from implicit to explicit memory. Notably, the overall amount of learned sequence pairs were not significantly different between experimental groups, but the degree of awareness was affected by the different settings. However, we observed an increased level of explicitly remembered pairs within the VR group compared to two screen-based groups. This finding clearly demonstrates that a near-natural experimental setting affects the transformation process from implicit to explicit memory.

Eye Movement Patterns Reflecting Cybersickness: Evidence from Different Experience Modes of a Virtual Reality Game

This study investigated whether the degree of cybersickness varies depending on different virtual reality experience modes (playing vs. watching) and whether specific eye movement parameters reflect changes in cybersickness. Simulator Sickness Questionnaire results from 20 participants (10 playing and 10 watching) showed that cybersickness was much more severe in the watching mode, particularly during the second of the three total trials. Moreover, cybersickness' changing pattern was reflected in the center gaze ratio and scan-path length. These findings imply the importance of physiological measurements for a deeper understanding of cybersickness in theoretical and practical respects.

Immersion, presence, and episodic memory in virtual reality environments

Although virtual reality (VR) represents a promising tool for psychological research, much remains unknown about how properties of VR environments affect episodic memory. Two closely related characteristics of VR are immersion (i.e., the objective degree to which VR naturalistically portrays a real-world environment) and presence (i.e., the subjective sense of being "mentally transported" to the virtual world). Although some research has demonstrated benefits of increased immersion on VR-based learning, it is uncertain how broadly and consistently this benefit extends to individual components of immersion. Moreover, it is unclear whether presence may mediate the effect of immersion on memory. Three experiments assessed how presence and memory were affected by three manipulations of immersion: field of view, unimodal (visual only) vs. bimodal (audiovisual) environments, and the realism of lighting effects (e.g., the occurrence or absence of shadows). Results indicated that effects of different manipulations of immersion are heterogeneous, affecting memory in some instances and presence in others, but not necessarily both. Importantly, no evidence for a mediating effect of presence emerged in any of these experiments, nor in a combined cross-experimental analysis. This outcome demonstrates a degree of independence between immersion and presence with regard to their influence on episodic memory performance.

Car drivers coping with hazardous events in real versus simulated situations: Declarative, behavioral and physiological data used to assess drivers' feeling of presence

More than 1.3 million people lose their lives every year in traffic accidents. Improving road safety requires designing better vehicles and investigating drivers’ abilities more closely. Driving simulators are constantly being used for this purpose, but the question which often arises as to their validity tends to be a barrier to developments in this field. Here we studied the validity of a simulator, defined as how closely users’ behavior under simulated conditions resembles their behavior on the road, based on the concept of drivers’ feeling of presence. For this purpose, the driving behavior, physiological state and declarative data of 41 drivers were tested in the Sherpa2 simulator and in a real vehicle on a track while driving at a constant speed. During each trial, drivers had to cope with an unexpected hazardous event (a one-meter diameter gym ball crossing the road right in front of the vehicle), which occurred twice. During the speed-maintenance task, the simulator showed absolute validity, in terms of the driving and physiological parameters recorded. During the first hazardous event, the physiological parameters showed that the level of arousal (Low Heart Rate/High Heart Rate ratio x10) increased up to the end of the drive. On the other hand, the drivers’ behavioral (braking) responses were 20% more frequent in the simulator than in the real vehicle, and the physiological state parameters showed that stress reactions occurred only in the real vehicle (+5 beats per minute, +2 breaths per minute and the phasic skin conductance increased by 2). In the subjects’ declarative data, several feeling of presence sub-scales were lower under simulated conditions. These results suggest that the validity of motion based simulators for testing drivers coping with hazards needs to be questioned.

Exploration of a novel virtual environment improves memory consolidation in ADHD

Experimental evidence in rodents and humans suggests that long-term memory consolidation can be enhanced by the exploration of a novel environment presented during a vulnerable early phase of consolidation. This memory enhancing effect (behavioral tagging) is caused by dopaminergic and noradrenergic neuromodulation of hippocampal plasticity processes. In translation from animal to human research, we investigated whether behavioral tagging with novelty can be used to tackle memory problems observed in children and adolescents with attention-deficit/hyperactivity disorder (ADHD). 34 patients with ADHD and 34 typically developing participants (age 9–15 years) explored either a previously familiarized or a novel virtual environment 45 min after they had learned a list of 20 words. Participants took a free recall test both immediately after learning the word list and after 24 h. Patients who explored a familiar environment showed significantly impaired memory consolidation compared to typically developing peers. Exploration of a novel environment led to significantly better memory consolidation in children and adolescents with ADHD. However, we did not observe a beneficial effect of novel environment exploration in typically developing participants. Our data rather suggested that increased exploration of a novel environment as well as higher feelings of virtual immersion compromised memory performance in typically developing children and adolescents, which was not the case for patients with ADHD. We propose that behavioral tagging with novel virtual environments is a promising candidate to overcome ADHD related memory problems. Moreover, the discrepancy between children and adolescents with and without ADHD suggests that behavioral tagging might only be able to improve memory consolidation for weakly encoded information.

Virtual reality in episodic memory research: A review

Although virtual reality (VR) is a promising tool for the investigation of episodic memory phenomena, to date there has been relatively little examination of how learning mechanisms operate in VR and how these processes might compare (or contrast) with learning that occurs in real life. Moreover, the existing literature on this topic is spread across several disciplines and uses various distinct apparatuses, thus obscuring whether the differences that exist between studies might be due to genuine theoretical discrepancies or may be more simply explained by accounting for methodological variations. The current review is designed to address and elucidate several issues relevant to psychological researchers interested in understanding and/or using this technological approach to study episodic memory phenomena. The principle objectives of the review are as follows: (a) defining and discussing the various VR systems currently used for research purposes, (b) compiling research of episodic memory effects in VR as they have been studied across several disciplines, and (c) surveying major topics in this body of literature (e.g., how virtual immersion has an impact on memory; transfer effects from VR to the real world). The content of this review is designed to serve as a resource for psychologists interested in learning more about the current state of research in this field and is intended to highlight the capabilities (and constraints) associated with using this technological approach in episodic memory research.

"Being there" and remembering it: Presence improves memory encoding

Few studies have investigated the link between episodic memory and presence: the feeling of "being there" and reacting to a stimulus as if it were real. We collected data from 244 participants after they had watched the movie Avengers: Age of Ultron. They answered questions about factual (details of the movie) and temporal memory (order of the scenes) about the movie, as well as their emotion experience and their sense of presence during the projection. Both higher emotion experience and sense of presence were related to better factual memory, but not to temporal order memory. Crucially, the link between emotion and factual memory was mediated by the sense of presence. We interpreted the role of presence as an external absorption of the attentional focus toward the stimulus, thus enhancing memory encoding. Our findings could shed light on the cognitive processes underlying memory impairments in psychiatric conditions characterized by an altered sense of reality.

Exploring a novel environment improves motivation and promotes recall of words

Active exploration of novel environments is known to increase plasticity in animals, promoting long-term potentiation in the hippocampus and enhancing memory formation. These effects can occur during as well as after exploration. In humans novelty’s effects on memory have been investigated with other methods, but never in an active exploration paradigm. We therefore investigated whether active spatial exploration of a novel compared to a previously familiarized virtual environment promotes performance on an unrelated word learning task. Exploration of the novel environment enhanced recall, generally thought to be hippocampus-dependent, but not recognition, believed to rely less on the hippocampus. Recall was better for participants that gave higher presence ratings for their experience in the virtual environment. These ratings were higher for the novel compared to the familiar virtual environment, suggesting that novelty increased attention for the virtual rather than real environment; however, this did not explain the effect of novelty on recall.