Using Biosignals for Objective Measurement of Presence in Virtual Reality Environments

A Machine Learning Approach to Classifying EEG Data Collected with or without Haptic Feedback during a Simulated Drilling Task

Artificial Intelligence (AI), computer simulations, and virtual reality (VR) are increasingly becoming accessible tools that can be leveraged to implement training protocols and educational resources. Typical assessment tools related to sensory and neural processing associated with task performance in virtual environments often rely on self-reported surveys, unlike electroencephalography (EEG), which is often used to compare the effects of different types of sensory feedback (e.g., auditory, visual, and haptic) in simulation environments in an objective manner. However, it can be challenging to know which aspects of the EEG signal represent the impact of different types of sensory feedback on neural processing. Machine learning approaches offer a promising direction for identifying EEG signal features that differentiate the impact of different types of sensory feedback during simulation training. For the current study, machine learning techniques were applied to differentiate neural circuitry associated with haptic and non-haptic feedback in a simulated drilling task. Nine EEG channels were selected and analyzed, extracting different time-domain, frequency-domain, and nonlinear features, where 360 features were tested (40 features per channel). A feature selection stage identified the most relevant features, including the Hurst exponent of 13-21 Hz, kurtosis of 21-30 Hz, power spectral density of 21-30 Hz, variance of 21-30 Hz, and spectral entropy of 13-21 Hz. Using those five features, trials with haptic feedback were correctly identified from those without haptic feedback with an accuracy exceeding 90%, increasing to 99% when using 10 features. These results show promise for the future application of machine learning approaches to predict the impact of haptic feedback on neural processing during VR protocols involving drilling tasks, which can inform future applications of VR and simulation for occupational skill acquisition.

Research Trends in Virtual Reality Music Concert Technology: A Systematic Literature Review

Advances in virtual reality (VR) technology have sparked novel avenues of growth in the musical domain. Following the COVID-19 pandemic, the rise of VR technology has led to growing interest in VR music concerts as an alternative to traditional live concerts. These virtual settings can provide immersion like attending real concerts for physically distant audiences and performers, and also can offer new creative possibilities. VR music concert research is still in its infancy, and advances in technologies such as multimodal devices are rapidly expanding the diversity of research, requiring a unified understanding of the field. To identify trends in VR music concert technology, we conducted a PRISMA-based systematic literature review covering the period from 2018 to 2023. After a thorough screening process, a total of 27 papers were selected for review. The studies were classified and analyzed based on the research topic (audience, performer, concert venue), interaction type (user-environment, user-user), and hardware used (head-mounted display, additional hardware). Furthermore, we categorized the evaluation metrics into user experience, usability, and performance. Our review contributes to advancing the understanding of recent developments in VR music concert technology, shedding light on the diversification and potential of this emerging field.

Immersive media experience: a survey of existing methods and tools for human influential factors assessment

Virtual reality (VR) applications, especially those where the user is untethered to a computer, are becoming more prevalent as new hardware is developed, computational power and artificial intelligence algorithms are available, and wireless communication networks are becoming more reliable, fast, and providing higher reliability. In fact, recent projections show that by 2022 the number of VR users will double, suggesting the sector was not negatively affected by the worldwide COVID-19 pandemic. The success of any immersive communication system is heavily dependent on the user experience it delivers, thus now more than ever has it become crucial to develop reliable models of immersive media experience (IMEx). In this paper, we survey the literature for existing methods and tools to assess human influential factors (HIFs) related to IMEx. In particular, subjective, behavioural, and psycho-physiological methods are covered. We describe tools available to monitor these HIFs, including the user's sense of presence and immersion, cybersickness, and mental/affective states, as well as their role in overall experience. Special focus is placed on psycho-physiological methods, as it was found that such in-depth evaluation was lacking from the existing literature. We conclude by touching on emerging applications involving multiple-sensorial immersive media and provide suggestions for future research directions to fill existing gaps. It is hoped that this survey will be useful for researchers interested in building new immersive (adaptive) applications that maximize user experience.