A SWOT Analysis of the Field of Virtual Reality for Firefighter Training

Exploring Immersive Multimodal Virtual Reality Training, Affective States, and Ecological Validity in Healthy Firefighters: Quasi-Experimental Study

BACKGROUND

Firefighters face stressful life-threatening events requiring fast decision-making. To better prepare for those situations, training is paramount, but errors in real-life training can be harmful. Virtual reality (VR) simulations provide the desired realism while enabling practice in a secure and controlled environment. Firefighters' affective states are also crucial as they are a higher-risk group.

OBJECTIVE

To assess the impact on affective states of 2 simulated immersive experiences in a sample of healthy firefighters (before, during, and after the simulation), we pursued a multivariate approach comprising cognitive performance, situational awareness, depression, anxiety, stress, number of previous adverse events experienced, posttraumatic stress disorder (PTSD) severity, and emotions. The efficacy and ecological validity of an innovative VR haptic system were also tested, exploring its impact on performance.

METHODS

In collaboration with the Portuguese National Fire Service School, we exposed 22 healthy firefighters to 2 immersive scenarios using the FLAIM Trainer VR system (neutral and arousing scenarios) while recording physiological data in a quasi-experimental study. Baseline cognitive performance, depression, anxiety, stress, number of adverse events, and severity of PTSD symptoms were evaluated. Positive and negative affective states were measured before, between, and after each scenario. Situational awareness, sense of presence, ecological validity, engagement, and negative effects resulting from VR immersion were tested.

RESULTS

Baseline positive affect score was high (mean 32.4, SD 7.2) and increased after the VR tasks (partial η2=0.52; Greenhouse-Geisser F1.82,32.78=19.73; P<.001). Contrarily, mean negative affect score remained low (range 11.0-11.9) throughout the study (partial η2=0.02; Greenhouse-Geisser F2.13,38.4=0.39; P=.69). Participants' feedback on the VR sense of presence was also positive, reporting a high sense of physical space (mean score 3.9, SD 0.8), ecological validity (mean score 3.8, SD 0.6), and engagement (mean score 3.8, SD 0.6). Engagement was related to the number of previously experienced adverse events (r=0.49; P=.02) and positive affect (after the last VR task; r=0.55; P=.02). Conversely, participants reported few negative effects (mean score 1.7, SD 0.6). The negative effects correlated positively with negative affect (after the last VR task; r=0.53; P=.03); and avoidance (r=0.73; P<.001), a PTSD symptom, controlling for relevant baseline variables. Performance related to situational awareness was positive (mean 46.4, SD 34.5), although no relation was found to metacognitively perceived situational awareness (r=-0.12; P=.59).

CONCLUSIONS

We show that VR is an effective alternative to in-person training as it was considered ecologically valid and engaging while promoting positive emotions, with few negative repercussions. This corroborates the use of VR to test firefighters' performance and situational awareness. Further research is needed to ascertain that firefighters with PTSD symptomatology are not negatively affected by VR. This study favors the use of VR training and provides new insights on its emotional and cognitive impact on the trainee.

A Machine Learning Approach to Classifying EEG Data Collected with or without Haptic Feedback during a Simulated Drilling Task

Artificial Intelligence (AI), computer simulations, and virtual reality (VR) are increasingly becoming accessible tools that can be leveraged to implement training protocols and educational resources. Typical assessment tools related to sensory and neural processing associated with task performance in virtual environments often rely on self-reported surveys, unlike electroencephalography (EEG), which is often used to compare the effects of different types of sensory feedback (e.g., auditory, visual, and haptic) in simulation environments in an objective manner. However, it can be challenging to know which aspects of the EEG signal represent the impact of different types of sensory feedback on neural processing. Machine learning approaches offer a promising direction for identifying EEG signal features that differentiate the impact of different types of sensory feedback during simulation training. For the current study, machine learning techniques were applied to differentiate neural circuitry associated with haptic and non-haptic feedback in a simulated drilling task. Nine EEG channels were selected and analyzed, extracting different time-domain, frequency-domain, and nonlinear features, where 360 features were tested (40 features per channel). A feature selection stage identified the most relevant features, including the Hurst exponent of 13-21 Hz, kurtosis of 21-30 Hz, power spectral density of 21-30 Hz, variance of 21-30 Hz, and spectral entropy of 13-21 Hz. Using those five features, trials with haptic feedback were correctly identified from those without haptic feedback with an accuracy exceeding 90%, increasing to 99% when using 10 features. These results show promise for the future application of machine learning approaches to predict the impact of haptic feedback on neural processing during VR protocols involving drilling tasks, which can inform future applications of VR and simulation for occupational skill acquisition.

Studying the Influence of Multisensory Stimuli on a Firefighting Training Virtual Environment

How we perceive and experience the world around us is inherently multisensory. Most of the Virtual Reality (VR) literature is based on the senses of sight and hearing. However, there is a lot of potential for integrating additional stimuli into Virtual Environments (VEs), especially in a training context. Identifying the relevant stimuli for obtaining a virtual experience that is perceptually equivalent to a real experience will lead users to behave the same across environments, which adds substantial value for several training areas, such as firefighters. In this article, we present an experiment aiming to assess the impact of different sensory stimuli on stress, fatigue, cybersickness, Presence and knowledge transfer of users during a firefighter training VE. The results suggested that the stimulus that significantly impacted the user's response was wearing a firefighter's uniform and combining all sensory stimuli under study: heat, weight, uniform, and mask. The results also showed that the VE did not induce cybersickness and that it was successful in the task of transferring knowledge.

Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

Use of virtual reality exercises in disaster preparedness training: A scoping review

Background

The scope and number of disasters have increased over the years. This has called for more robust disaster preparedness training and plans. The use of virtual reality exercises in addition to tabletop exercises is considered a new approach to the preparation of disaster preparedness plans. Virtual reality exercises are being developed to either replace or complement current traditional approaches to disaster preparedness training.

Objectives

To review and summarize the current existing literature regarding the effectiveness, advantages and limitations of using virtual reality exercises in disaster preparedness as a complementary/replacement mechanism for real-time drills and tabletop exercises.

Methods

In this scoping review, we searched PubMed, Cochrane, EMBASE, PLOS, and Google Scholar for research publications involving virtual reality exercises in disaster training from 2008 to 2022 using "AND" and "OR" operators for the keywords "disaster," "preparedness," "virtual reality," and "tabletop." From a total of 333 articles that resulted in our search and were then evaluated by the authors, 55 articles were finally included in this review.

Results

Virtual reality exercises are found to be better in the formulation of disaster preparedness plans compared to tabletop exercises. Virtual reality exercises can be used as the primary means of creating a real-life-like experience in disaster preparedness training and proved at least as better complementary to tabletop exercises. Virtual reality exercises have many advantages over traditional real-life or tabletop exercises and are more cost-effective, but some drawbacks are still identified.

Conclusion

The advantages of virtual reality exercises are remarkable and underline their benefits and uses versus costs. We highly encourage decision-makers and institutions dealing in disaster preparedness to adopt using virtual reality exercises in training for disaster preparedness.

Firefighters' muscle activity change during firefighting training program

BACKGROUND

Research on muscle activity to reduce injuries during firefighting training has getting increasing attention.

OBJECTIVE

The purpose of this study was to assess the activity changes in nine muscles of firefighters during the seven firefighting training programs, and to analyze the influence of different firefighting training programs on muscle activity.

METHODS

Ten healthy male firefighters were recruited to measure the field surface electromyographic activities (including the percentage of Maximum Voluntary Contraction electromyography (% MVC) and the integrated electromyography value (iEMG)) during all the firefighting training programs.

RESULTS

The results showed that the electromyographic activity of gastrocnemius (GA) was stronger in climbing the hooked ladder and climbing the six-meter long ladder training programs. Arms, shoulders, and lower limb muscles were more activated, myoelectric activities were more intense, and fatigue in these areas was more likely to occur during climbing five-story building with loads. Compared with other muscles, erector spine (ES) had a higher degree of activation during different postures of water shooting. The Borg scale scores of shoulders, trunk, thighs and calves were also higher.

CONCLUSION

After completing all training programs, GA, tibialis anterior (TA), trapezius (TR), and ES were strongly activated, and all muscles had obvious force. The % MVC and iEMG analyses correspond well with the Borg Scale score. The results can provide certain reference for reducing the musculoskeletal injury of firefighters, carrying out scientific training and formulating effective injury prevention measures for them.

Modeling Brain Dynamics During Virtual Reality-Based Emergency Response Learning Under Stress

BACKGROUND

Stress affects learning during training, and virtual reality (VR) based training systems that manipulate stress can improve retention and retrieval performance for firefighters. Brain imaging using functional Near Infrared Spectroscopy (fNIRS) can facilitate development of VR-based adaptive training systems that can continuously assess the trainee's states of learning and cognition.

OBJECTIVE

The aim of this study was to model the neural dynamics associated with learning and retrieval under stress in a VR-based emergency response training exercise.

METHODS

Forty firefighters underwent an emergency shutdown training in VR and were randomly assigned to either a control or a stress group. The stress group experienced stressors including smoke, fire, and explosions during the familiarization and training phase. Both groups underwent a stress memory retrieval and no-stress memory retrieval condition. Participant's performance scores, fNIRS-based neural activity, and functional connectivity between the prefrontal cortex (PFC) and motor regions were obtained for the training and retrieval phases.

RESULTS

The performance scores indicate that the rate of learning was slower in the stress group compared to the control group, but both groups performed similarly during each retrieval condition. Compared to the control group, the stress group exhibited suppressed PFC activation. However, they showed stronger connectivity within the PFC regions during the training and between PFC and motor regions during the retrieval phases.

DISCUSSION

While stress impaired performance during training, adoption of stress-adaptive neural strategies (i.e., stronger brain connectivity) were associated with comparable performance between the stress and the control groups during the retrieval phase.

Assessing the Financial Sustainability of High-Fidelity and Virtual Reality Simulation for Nursing Education: A Retrospective Case Analysis

To stimulate classroom discussion and collaboration amid the COVID-19 pandemic, increasingly creative pedological methods for nursing education are necessary. Traditionally, high-fidelity simulation has been the standard for nursing education, but the use of virtual reality simulation is increasing. One of the major evaluative measures of simulation clinical training is the cost associated with each modality. In this retrospective case analysis, budget impact analysis methods were employed to compare high-fidelity simulation with virtual reality simulation. The components of each simulation pedagogy were compared in categorized cost buckets. Overall, virtual reality simulation education was determined to require 22% less time than high-fidelity simulation education. The cost associated with the virtual reality simulation was found to be 40% less expensive than the high-fidelity simulation. Our results demonstrate that virtual reality simulation is a financially advantageous, resource conscious pedagogical option for nursing education.

The Validity of Virtual Courage for Trainees in High-Risk Occupations

Background

Employees in high-risk occupations are exposed to tremendous work stress that hinders organizational effectiveness and personal mental health. Based on positive psychology, courage can be considered a protective factor that buffers the adverse effect of high-risk surroundings on employees. However, little is known about the way courage is simulated or evaluated in response to safety concerns. Virtual reality (VR) is an accessible tool for courage simulation due to its immersive qualities, presence and interactive features and may provide a promising pathway to achieve a scientific, accurate and ecologically valid evaluation of high-risk employees.

Methods

The sample consisted of 51 high-risk employees who were recruited voluntarily. Before and after experiencing the VR courage scenarios, the participants completed the VR features questionnaire, the Physical Courage at Work Scale (PCWS), the Courage Measure (CM), and the Positive and Negative Affect Scale (PANAS). During the process of watching the VR courage scenarios, the participants’ heart rate and skin conductance at resting-state baseline and during virtual courage scenarios were recorded through HTC VIVE Pro Eye and BioGraph Infiniti 8.

Results

The results support the hypothesis and reveal that the interaction, immersion and presence scores of the scenarios were all significantly higher than the median 4 points. The score for the CM in the posttest was significantly higher than that in the pretest. The scared and afraid scores for the posttest were significantly higher than those for the pretest. The heart rate and skin conductance of each scenario showed an increase compared with the baseline. The Pearson’s correlation between physiological indicators and the score of the PCWS was 0.28~0.54.

Conclusion

This study developed virtual courage for high-risk occupations based on well-established theory and VR technology. Experimental data revealed that the paradigm conformed to the requirements of VR features and was able to activate fear and evoke the quality of courage. Thus, the virtual courage paradigms have good validity in simulating scenarios for high-risk employees, which might accelerate organizational effectiveness while buffering working stress.

Building Information Modeling (BIM) Capabilities in the Operation and Maintenance Phase of Green Buildings: A Systematic Review

In recent years, green buildings have gradually become a worldwide trend. Compared with traditional buildings, green buildings have advanced requirements and standards in their operation and maintenance phase. In such a context, some studies proposed that building information modeling (BIM) is an effective method to improve green buildings’ operation and maintenance quality. The aim of this study is to perform a comprehensive review of the BIM capabilities in the operation and maintenance phase of green buildings through a systematic literature review. To achieve this aim, the PRISMA protocol was used to perform this systematic review. The whole systematic review was conducted between January 2022 and April 2022: 128 articles were included. In the process of study, Web of Science (WoS) and Scopus were adopted as bibliographic repositories. Through this study, it can be determined that BIM capabilities can be utilized in the facility management of the green building in the following aspects: safety and emergency management, maintenance and repair, energy management, security, retrofit and renovation, space management, and asset management. Secondly, these BIM capabilities were discussed, and the challenges and shortcomings of BIM capabilities in the operation and maintenance phase of green buildings were reviewed. Finally, a comprehensive overview of BIM capabilities in the facility management of green buildings was developed, and suggestions for future study were provided.

Mathematical Algorithm for Risk Assessment of Police Officer in VR Training Simulation

Conducting safe coaching is essential for training police officers, who very often face a variety of unexpected and dangerous incidents. Their reaction to situations must be rapid and appropriate. To prepare officers for dangerous situations, but those that cannot be practiced in real life due to high costs, danger, time, or effort, virtual training seems to be the obvious choice. This article deals with the development of a calculation algorithm to assess the risk of actions taken on the site of a traffic incident, which was implemented into the training version of a virtual reality (VR) simulation. It includes a number of factors and elements that form a scenario of simulations that affect the degree of its difficulty and the assessment of the performance of each exercise. The different components of the algorithm that make it possible to assess the skills of the students of police specialist courses are presented. The acceptance criterion for the developed algorithm shall be the correct assessment of the student’s skills during the course of the training.

Visual fatigue induced by watching virtual reality device and the effect of anisometropia

The effect of small anisometropia on visual fatigue when using virtual reality (VR) devices was investigated. Participants (n = 34) visited three times. In the first visit, VR exposure (10 min) was conducted with the full correction of the refractive error of both eyes. Experimental anisometropia was induced by adding a + 1.0 dioptre spherical lens either on the dominant eyes in the second visit or on the non-dominant eyes in the third visit. At each visit, the participants played a predetermined video game using a head-mounted display VR for 10 min. Visual fatigue was assessed before and after playing VR game using the Virtual Reality Symptom Questionnaire (VRSQ) and high-frequency component of accommodative microfluctuation. Results showed that watching VR induced significant increase of VRSQ score, significant decrease in the maximum accommodation power and objective increase in visual fatigue. Experimental anisometropia induction either on the dominant or non-dominant eyes did not aggravate visual fatigue. Practitioner summary: Mild differences in refractive error (up to 1.0 dioptre) between both eyes do not significantly increase ocular fatigue by viewing virtual reality device (10 min). The impact of small anisometropia may be limited in developing a virtual reality device. Abbreviations: VR: virtual reality; VRSQ: virtual reality symptom questionnaire; HMD: head-mounted display; HFC: high-frequency component.

The More, the Better? Improving VR Firefighting Training System with Realistic Firefighter Tools as Controllers

A virtual reality (VR) controller plays a key role in supporting interactions between users and the virtual environment. This paper investigates the relationship between the user experience and VR control device modality. We developed a VR firefighting training system integrated with four control devices adapted from real firefighting tools. We iteratively improved the controllers and VR system through a pilot study with six participants and conducted a user study with 30 participants to assess two salient human factor constructs—perceived presence and cognitive load—with three device modality conditions (two standard VR controllers, four real tools, and a hybrid of one real tool and one standard VR controller). We found that having more realistic devices that simulate real tools does not necessarily guarantee a higher level of user experience, highlighting a strategic approach to the development and utilization of VR control devices. Our study gives empirical insights on establishing appropriate combinations of VR control device modality in the context of field-based VR simulation and training.

Remote Virtual Simulation for Incident Commanders—Cognitive Aspects

Due to the COVID-19 restrictions, on-site Incident Commander (IC) practical training and examinations in Sweden were canceled as of March 2020. The graduation of one IC class was, however, conducted through Remote Virtual Simulation (RVS), the first such examination to our current knowledge. This paper presents the necessary enablers for setting up RVS and its influence on cognitive aspects of assessing practical competences. Data were gathered through observations, questionnaires, and interviews from students and instructors, using action-case research methodology. The results show the potential of RVS for supporting higher cognitive processes, such as recognition, comprehension, problem solving, decision making, and allowed students to demonstrate whether they had achieved the required learning objectives. Other reported benefits were the value of not gathering people (imposed by the pandemic), experiencing new, challenging incident scenarios, increased motivation for applying RVS based training both for students and instructors, and reduced traveling (corresponding to 15,400 km for a class). While further research is needed for defining how to integrate RVS in practical training and assessment for IC education and for increased generalizability, this research pinpoints current benefits and limitations, in relation to the cognitive aspects and in comparison, to previous examination formats.

Brain Activity-Based Metrics for Assessing Learning States in VR under Stress among Firefighters: An Explorative Machine Learning Approach in Neuroergonomics

The nature of firefighters` duties requires them to work for long periods under unfavorable conditions. To perform their jobs effectively, they are required to endure long hours of extensive, stressful training. Creating such training environments is very expensive and it is difficult to guarantee trainees' safety. In this study, firefighters are trained in a virtual environment that includes virtual perturbations such as fires, alarms, and smoke. The objective of this paper is to use machine learning methods to discern encoding and retrieval states in firefighters during a visuospatial episodic memory task and explore which regions of the brain provide suitable signals to solve this classification problem. Our results show that the Random Forest algorithm could be used to distinguish between information encoding and retrieval using features extracted from fNIRS data. Our algorithm achieved an F-1 score of 0.844 and an accuracy of 79.10% if the training and testing data are obtained at similar environmental conditions. However, the algorithm's performance dropped to an F-1 score of 0.723 and accuracy of 60.61% when evaluated on data collected under different environmental conditions than the training data. We also found that if the training and evaluation data were recorded under the same environmental conditions, the RPM, LDLPFC, RDLPFC were the most relevant brain regions under non-stressful, stressful, and a mix of stressful and non-stressful conditions, respectively.

Measuring User Experience, Usability and Interactivity of a Personalized Mobile Augmented Reality Training System

Innovative technology has been an important part of firefighting, as it advances firefighters' safety and effectiveness. Prior research has examined the implementation of training systems using augmented reality (AR) in other domains, such as welding, aviation, army, and mathematics, offering significant pedagogical affordances. Nevertheless, firefighting training systems using AR are still an under-researched area. The increasing penetration of AR for training is the driving force behind this study, and the scope is to analyze the main aspects affecting the acceptance of AR by firefighters. The current research uses a technology acceptance model, extended by the external constructs of perceived interactivity and personalization, to consider both the system and individual level. The proposed model was evaluated by a sample of 200 users, and the results show that both the external variables of perceived interactivity and perceived personalization are prerequisite factors in extending the TAM model. The findings reveal that the usability is the strongest predictor of firefighters' behavioral intentions to use the AR system, followed by the ease of use with smaller, yet meaningful, direct and indirect effects on firefighters' intentions. The identified acceptance factors help AR developers enhance the firefighters' experience in training operations.

Progress in Research on Sustainable Urban Renewal Since 2000: Library and Visual Analyses

Urban renewal is an ideal approach to promoting the value of the urban fabric and improving the sustainability of the urban environment. This study, which shows the continuity of research on sustainable urban renewal, aimed to identify sustainable urban renewal literature based on a library analysis of scientific research since 2000. A total of 3971 scientific papers from the SCIE (Science Citation Index Expanded) and SSCI (Social Sciences Citation Index) databases were reviewed to examine how research concerning “sustainable urban renewal” has emerged and developed in the past 20 years. The h-indices and impact factors of the most relevant journals in urban renewal and sustainable development since 2000 were analyzed. The most frequently cited articles were analyzed using analysis of social networks (VOSviewer). The results revealed potential future focuses of research and guidelines that link urban renewal and sustainability: the engagement of all stakeholders in the decision-making process; the involvement of residents in projects; the development of cooperation between towns and cities; the preservation and reuse of built and industrial heritage while respecting environmental law; and, finally, the search for new financing techniques. These potential future research topics were analyzed in four research areas so that sustainable development can easily be integrated into an urban renewal project.

Preliminary Work on a Virtual Reality Interface for the Guidance of Underwater Robots

The need for intervention in underwater environments has increased in recent years but there is still a long way to go before AUVs (Autonomous Underwater Vehicleswill be able to cope with really challenging missions. Nowadays, the solution adopted is mainly based on remote operated vehicle (ROV) technology. These ROVs are controlled from support vessels by using unnecessarily complex human–robot interfaces (HRI). Therefore, it is necessary to reduce the complexity of these systems to make them easier to use and to reduce the stress on the operator. In this paper, and as part of the TWIN roBOTs for the cooperative underwater intervention missions (TWINBOT) project, we present an HRI (Human-Robot Interface) module which includes virtual reality (VR) technology. In fact, this contribution is an improvement on a preliminary study in this field also carried out, by our laboratory. Hence, having made a concerted effort to improve usability, the HRI system designed for robot control tasks presented in this paper is substantially easier to use. In summary, reliability and feasibility of this HRI module have been demonstrated thanks to the usability tests, which include a very complete pilot study, and guarantee much more friendly and intuitive properties in the final HRI-developed module presented here.