Does virtual reality help to cut the Gordian knot between ecological validity and experimental control?

Dynamic and/or multimodal assessments for social cognition in neuropsychology: Results from a systematic literature review

Objective: Despite the prevalence of socio-cognitive disturbances, and their important diagnostic/therapeutic implications, the assessment of these disturbances remains scarce. This systematic review aims to identify available social cognition tools for adult assessment that use multimodal and/or dynamic social cues, specifying their strengths and limitations (e.g. from a methodological, psychometric, ecological, and clinical perspective). Method: An electronic search was conducted in Pubmed, PsychINFO, Embase and Scopus databases for articles published up to the 3th of January 2023 and the first 200 Google Scholar results on the same date. The PRISMA methodology was applied, 3884 studies were screened based on title and abstract and 329 full texts were screened. Articles using pseudo-dynamic methodologies (e.g. morphing), reported only subjective or self-reported measures, or investigated only physiological or brain activity responses were excluded. Results: In total, 149 works were included in this review, representing 65 assessment tools (i.e. 48% studying emotion recognition (n = 31), 32% Theory of Mind (n = 21), 5% empathy (n = 3), 1.5% moral cognition/social reasoning (n = 1), and 14% being multimodal (n = 9)). For each study, the tool's main characteristics, psychometric properties, ecological validity indicators and available norms are reported. The tools are presented according to social-cognitive process assessed and communication channels used. Conclusions: This study highlights the lack of validated and standardized tools. A few tools appear to partially meet some clinical needs. The development of methodologies using a first-person paradigm and taking into account the multidimensional nature of social cognition seems a relevant research endeavour for greater ecological validity.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

Induced oscillatory brain responses under virtual reality conditions in the context of repetition priming

In the human electroencephalogram (EEG), induced oscillatory responses in various frequency bands are regarded as valuable indices to examine the neural mechanisms underlying human memory. While the advent of virtual reality (VR) drives the investigation of mnemonic processing under more lifelike settings, the joint application of VR and EEG methods is still in its infancy (e.g., due to technical limitations impeding the signal acquisition). The objective of the present EEG study was twofold. First, we examined whether the investigation of induced oscillations under VR conditions yields equivalent results compared to standard paradigms. Second, we aimed at obtaining further insights into basic memory-related brain mechanisms in VR. To these ends, we relied on a standard implicit memory design, namely repetition priming, for which the to-be-expected effects are well-documented for conventional studies. Congruently, we replicated a suppression of the evoked potential after stimulus onset. Regarding the induced responses, we observed a modulation of induced alphaband in response to a repeated stimulus. Importantly, our results revealed a repetition-related suppression of the high-frequency induced gammaband response (>30 Hz), indicating the sharpening of a cortical object representation fostering behavioral priming effects. Noteworthy, the analysis of the induced gammaband responses required a number of measures to minimize the influence of external and internal sources of artefacts (i.e., the electrical shielding of the technical equipment and the control for miniature eye movements). In conclusion, joint VR-EEG studies with a particular focus on induced oscillatory responses offer a promising advanced understanding of mnemonic processing under lifelike conditions.

Performance on a virtual environment shopping task and adaptive functioning among older adults

Neuropsychologists are often asked to evaluate patients' functional capacities, yet traditional neuropsychological tests have limited correspondence with real-world outcomes. The Virtual Environment Grocery Store (VEGS) is a virtual environment that simulates shopping tasks. The purpose of this study was to investigate the relationship between older adults' performance on the VEGS in relation to their self-reported adaptive functioning as well as performance on a performance-based adaptive functioning measure. Older adults (n = 98; age 65-90, M = 75.82, SD = 6.27) were administered the VEGS, the Instrumental Activities of Daily Living Scale (IADLS), and the Texas Functional Living Scale (TFLS). Neither premorbid functioning nor the VEGS variables were associated with self-rated adaptive functioning. However, all three VEGS variables were associated with performance-based adaptive functioning (i.e., the TFLS). Performance on the VEGS measure of correct items in the shopping cart explained 13.9% of the variance in performance on the performance-based adaptive functioning task. Whether the participant picked up the VEGS prescription explained 12.6% of the variance in performance on the performance-based adaptive functioning task. Performance on VEGS long delay free recall explained 35.1% of the variance in performance on the performance-based adaptive functioning task. These results suggest that the VEGS demonstrates value in predicting older adults' functional capacities.

Feeling Virtually Present Makes Me Happier: The Influence of Immersion, Sense of Presence, and Video Contents on Positive Emotion Induction

Immersive technologies, such as Virtual Reality (VR), have great potential for enhancing users' emotions and wellbeing. However, how immersion, Virtual Environment contents, and sense of presence (SoP) influence emotional responses remains to be clarified to efficiently foster positive emotions. Consequently, a total of 26 participants (16 women, 10 men, 22.73 ± 2.69 years old) were exposed to 360-degree videos of natural and social contents on both a highly immersive Head-Mounted Display and a low immersive computer screen. Subjective emotional responses and SoP were assessed after each video using self-reports, while a wearable wristband collected continuously electrodermal activity and heart rate to record physiological emotional responses. Findings supported the added value of immersion, as more positive emotions and greater subjective arousal were reported after viewing the videos in the highly immersive setting, regardless of the video contents. In addition to usually employed natural contents, the findings also provide initial evidence for the effectiveness of social contents in eliciting positive emotions. Finally, structural equation models shed light on the indirect effect of immersion, through spatial and spatial SoP on subjective arousal. Overall, these are encouraging results about the effectiveness of VR for fostering positive emotions. Future studies should further investigate the influence of user characteristics on VR experiences to foster efficiently positive emotions among a broad range of potential users.

The reality of virtual reality

Virtual reality (VR) has become a popular tool for investigating human behavior and brain functions. Nevertheless, it is unclear whether VR constitutes an actual form of reality or is more like an advanced simulation. Determining the nature of VR has been mostly achieved by self-reported presence measurements, defined as the feeling of being submerged in the experience. However, subjective measurements might be prone to bias and, most importantly, do not allow for a comparison with real-life experiences. Here, we show that real-life and VR height exposures using 3D-360° videos are mostly indistinguishable on a psychophysiological level (EEG and HRV), while both differ from a conventional 2D laboratory setting. Using a fire truck, three groups of participants experienced a real-life (N = 25), a virtual (N = 24), or a 2D laboratory (N = 25) height exposure. Behavioral and psychophysiological results suggest that identical exogenous and endogenous cognitive as well as emotional mechanisms are deployed to process the real-life and virtual experience. Specifically, alpha- and theta-band oscillations in line with heart rate variability, indexing vigilance, and anxiety were barely indistinguishable between those two conditions, while they differed significantly from the laboratory setup. Sensory processing, as reflected by beta-band oscillations, exhibits a different pattern for all conditions, indicating further room for improving VR on a haptic level. In conclusion, the study shows that contemporary photorealistic VR setups are technologically capable of mimicking reality, thus paving the way for the investigation of real-world cognitive and emotional processes under controlled laboratory conditions. For a video summary, see https://youtu.be/fPIrIajpfiA.

Factor analysis of the virtual reality Stroop task

INTRODUCTION

Virtual reality (VR) offers neuropsychologists high dimensional (3D) platforms for administering cognitive tasks that balance experimental control with simulations of naturalistic activities. A virtual reality version of the Stroop task, the Virtual Reality Stroop Task (VRST), was developed that leverages technological advances to enhance the ecological validity of neuropsychological assessments. The high mobility multipurpose wheeled vehicle (HMMWV) version of the VRST includes high arousal (ambush) and low arousal (safe) zones was employed in this study. This version of the VRST contains both cognitive (Stroop) and affective (arousal) components. While the VRST has been shown to have good construct validity, the factor structure has yet to be explored. This study aimed to examine the factor structure of the VRST and compare the results with a lower dimensional (2D) computer-automated Stroop task (i.e., the Automated Neuropsychological Assessment Metrics - ANAM).

METHOD

Data was drawn from college-aged students who completed the VRST and ANAM Stroop tasks (N = 115). Factor analyses utilized principal axis factoring (PAF), and output variables included percent of correct responses and response times the VRST and ANAM Stroop tasks.

RESULTS

Results indicated that both Stroop tasks had two-factor solutions. Factor one measured response times and factor two measured accuracy. While factors respective of speed and accuracy factors were correlated across assessment modalities, within assessment factor correlations were low.

CONCLUSIONS

The factors possibly indicated participants response styles in that they either focus on responding accurately or responding quickly to stimuli. Therefore, including throughput in future research examining either the ANAM Stroop task or VRST may provide useful insight into participant performance. Finally, because similar factor structures were observed for both the VRST and ANAM Stroop task this study provided additional support for the construct validity of a higher dimensional Stroop task, the VRST.

Comparison of Traditional and Virtual Reality-Based Episodic Memory Performance in Clinical and Non-Clinical Cohorts

The California Verbal Learning Test, Second Edition (CVLT-II) and the Virtual Environment Grocery Store (VEGS) use list learning and recognition tasks to assess episodic memory. This study aims to: (1) Replicate prior construct validity results among a new sample of young adults and healthy older adults; (2) Extend this work to a clinical sample of older adults with a neurocognitive diagnosis; (3) Compare CVLT-II and VEGS performance among these groups; and (4) Validate the independence of CVLT and VEGS episodic memory performance measures from executive functioning performance measures. Typically developing young adults (n = 53) and older adults (n = 85), as well as older adults with a neurocognitive diagnosis (n = 18), were administered the CVLT-II, VEGS, and D-KEFS CWIT. Results found that (1) the relationship of the VEGS and CVLT-II measures was highly correlated on all variables, (2) compared to the CVLT-II, participants (particularly older adults) recalled fewer items on the VEGS, and (3) the CVLT-II and VEGS were generally independent of D-KEFS CWIT. It appeared that the VEGS may be more difficult than the CVLT-II, possibly reflecting the word length effect. Performance may have also been impacted by the presence of everyday distractors in the virtual environment.

Virtual Reality Biofeedback in Health: A Scoping Review

Over the past decades, virtual reality (VR) has found its way into biofeedback (BF) therapy programs. Using VR promises to overcome challenges encountered in traditional BF such as low treatment motivation, low attentional focus and the difficulty of transferring learnt abilities to everyday life. Yet, a comprehensive research synthesis is still missing. Hence, this scoping review aims to provide an overview over empirical studies on VR based BF regarding key outcomes, included samples, used soft- and hardware, BF parameters, mode of application and potential limitations. We systematically searched Medline, PsycINFO, Scopus, CINAHL, Google Scholar and Open Grey for empirical research. Eighteen articles met the inclusion criteria. Samples mostly consisted of healthy (44.4%) and/or adult (77.7%) participants. Outcomes were mainly anxiety (44.4%), stress (44.4%) or pain reduction (11.1%), which were reduced by the VR-BF interventions at least as much as by classical BF. Participants in VR-BF interventions showed higher motivation and involvement as well as a better user experience. Heart rate or heart rate variability were the most frequently used BF parameters (50.0%), and most VR-BF interventions (72.2%) employed a natural environment (e.g., island). Currently, there is no clear evidence that VR-BF is more effective than traditional BF. Yet, results indicate that VR-BF may have advantages regarding motivation, user experience, involvement and attentional focus. Further research is needed to assess the specific impact of VR and gamification. Also, testing a broader range of clinical and younger samples would allow more far-reaching conclusions.

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality

Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of "social presence": the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity ("research in the wild") and experimental control ("research in the lab") in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either via human-embodied avatars or computer-controlled agents. The mediation of social musical interactions via VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach via a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.

Using Medical Device Standards for Design and Risk Management of Immersive Virtual Reality for At-Home Therapy and Remote Patient Monitoring

Numerous virtual reality (VR) systems have received regulatory clearance as therapeutic medical devices for in-clinic and at-home use. These systems enable remote patient monitoring of clinician-prescribed rehabilitation exercises, although most of these systems are nonimmersive. With the expanding availability of affordable and easy-to-use head-mounted display (HMD)-based VR, there is growing interest in immersive VR therapies. However, HMD-based VR presents unique risks. Following standards for medical device development, the objective of this paper is to demonstrate a risk management process for a generic immersive VR system for remote patient monitoring of at-home therapy. Regulations, standards, and guidance documents applicable to therapeutic VR design are reviewed to provide necessary background. Generic requirements for an immersive VR system for home use and remote patient monitoring are identified using predicate analysis and specified for both patients and clinicians using user stories. To analyze risk, failure modes and effects analysis, adapted for medical device risk management, is performed on the generic user stories and a set of risk control measures is proposed. Many therapeutic applications of VR would be regulated as a medical device if they were to be commercially marketed. Understanding relevant standards for design and risk management early in the development process can help expedite the availability of innovative VR therapies that are safe and effective.

Ethical Issues with Simulating the Bridge Problem in VR

We aim to generate a dilemma for virtual reality-based research that we motivate through an extended case study of Thomson's (Yale Law J 94(6):1395-1415, 1985) Bridge variant of the trolley problem. Though the problem we generate applies more broadly than the Bridge problem, we believe it makes a good exemplar of the kind of case we believe is problematic. First, we argue that simulations of these thought experiments run into a practicality horn that makes it practically impossible to produce them. These problems revolve around concepts that we call "perspectival fidelity" and "context realism." Moral dilemmas that include features present in the Bridge variant will, as a result, be practically impossible to simulate. We also argue that, should we be wrong about the practical impossibility of creating a VR simulation of Bridge, such a simulation must face an ethical horn which renders these simulations ethically impermissible to develop or use. For these reasons, we argue that it is virtually impossible to simulate the bridge problem (and other thought experiments with similar features) both practically and ethically in VR.