User evaluation of clinical simulation-based learning developed by FORCE (Framework for Online Radiographer Clinical Education)

INTRODUCTION

The FORCE initiative, funded by the EU, was created in response to the COVID-19 pandemic's impact on radiography education. It offers a virtual learning environment (VLE) filled with clinical simulation scenarios targeting Diagnostic Imaging, Nuclear Medicine, and Radiotherapy. These scenarios are designed to provide open-access, simulation-based learning (SBL) resources that help radiography undergraduates engage in problem-based learning across relevant clinical knowledge and professional awareness. This study presents the results of an online survey aimed at evaluating the FORCE VLE and SBL resources to guide future development.

METHODS

The survey targeted academic and clinical staff in Radiography and student Radiographers. It was based on the Course Experience Questionnaire (CEQ) and modified to include topics relevant to online learning, using a five-point Likert Scale (LS). The questionnaire consisted of four sections: research information and consent; socio-demographic data; LS-based questions on case structure, content, interactivity, technical aspects, multimedia quality, and overall experience; and open-text responses.

RESULTS

Of the 407 invited participants, 109 (27 %) responded, with 77 % being students and 23 % clinical or academic radiographers from 13 countries. English proficiency was reported by 76 % of respondents. The median LS was consistently 4 across all categories, indicating high satisfaction. Content analysis of 159 open-text comments revealed 139 positive opinions on the SBL resources.

CONCLUSION

The FORCE VLE SBL resources were highly regarded for their interactivity, applicability, and support in consolidating knowledge and enhancing patient care. Suggestions for improvement were minor, mainly focusing on navigation and specific content preferences.

IMPLICATIONS FOR PRACTICE

Expanding the availability of online SBL cases could enhance Radiographer education and promote inclusivity across the field.

Get Over It: Surgical Residents' Responses to Simulated Harassment. A Multi Method Study

OBJECTIVE

This study examined the response strategies of Surgery residents as bystanders to harassment in a simulated clinical environment, their alignment with the bystander intervention model, and the motivations behind their actions.

DESIGN

Participants watched an educational video on harassment and ways to address it prior to undergoing a simulated clinical scenario where they witnessed a senior resident harassing a medical student. The study used audio-video recordings of the simulations to capture and analyze residents' verbal and nonverbal responses to harassment. Frequencies from deductive thematic analysis were used for descriptive analysis, and nonparametric chi-square tests. Audio recordings of debriefings following simulations were also analyzed using deductive thematic analysis to explore residents' motivations behind their responses.

SETTING

The study was conducted in a controlled environment at McGill University's Steinberg Centre for Simulation and Interactive Learning (Montreal, Canada).

PARTICIPANTS

Participants included 32 Surgery residents completing the simulation as part of their Objective Structured Clinical Examinations. 28 residents provided usable data for analysis.

RESULTS

Residents used passive responses significantly more frequently than other responses throughout the simulation, especially in reaction to harassment. Analysis revealed that residents often delayed intervention, opting to remain passive or reassuring the victim rather than confronting the harasser. Debriefing showed that some residents intervened to denounce the SR's behavior or support the MS, while others hesitated due to discomfort with power dynamics, fear of repercussions, or uncertainty about addressing the situation CONCLUSIONS: The study found that Surgery residents exhibit a tendency towards passive bystander responses in the face of harassment highlighting the need for targeted educational strategies to address power dynamics, build confidence in bystander intervention, and promote proactive responses to harassment in clinical training. Future research should explore similar dynamics across different medical professionals and consider intersectional factors to enhance antiharassment initiatives in medical education.

Allied health professionals cognitive workload scale in Malacca, Malaysia: A psychometric properties evaluation

BackgroundCognitive workload has emerged as one of the most important topics that must be understood and addressed because of its impact on errors and work performance that will compromise patient safety. With increasing patient health demands, Allied Health Professionals (AHPs) tend to have heavier cognitive workloads and impact on personal well-being.ObjectiveOur study aims to determine the validity and reliability of a cognitive workload scale for AHP in Malacca, Malaysia and examine its effect on personal well-being.MethodThe instrument comprises items related to human reliability, human-computer interaction, decision-making, skilled performance, training, work stress and cognitive ergonomic effect. The content validation was conducted using the Content Validity Questionnaire with eight expert panels' responses. 120 AHP working in Malacca were recruited using purposive sampling and took part in the study, completing the online questionnaires. To evaluate the validity based on relationships with other measures, the SMEQ scale was also administered. AMOS Version 24 and IBM SPSS Version 26 were used in data analysis.ResultsThe instrument showed good content validity (CVI > 0.74). After the deletion of six items, the instrument has good convergent validity (AVE > 0.5), discriminant validity (HTMT Ratio < 0.85), composite reliability (CR > 0.7), internal consistency (alpha > 0.7), construct validity the ratio (1.865), CFI (0.900), PCFI (0.821), PNFI (0.738), RMSEA (0.085), IFI (0.90) and test-retest reliability (ICC > 0.6). The study found a positive correlation with the Malay version of SMEQ. This study also found that the AHP in the laboratory group have higher work stress and experiences more cognitive load effects, leading to lower personal well-being.ConclusionOverall, the final version of a scale measuring the cognitive workload is valid and reliable in assessing cognitive ergonomics among AHP. Testing cognitive workload on different work populations, including manual and non-manual workers, is recommended for future studies.

Development of a novel global rating scale for objective structured assessment of technical skills in an emergency medical simulation training

BACKGROUND

Medical simulation trainings lead to an improvement in patient care by increasing technical and non-technical skills, procedural confidence and medical knowledge. For structured simulation-based trainings, objective assessment tools are needed to evaluate the performance during simulation and the learning progress. In surgical education, objective structured assessment of technical skills (OSATS) are widely used and validated. However, in emergency medicine and anesthesia there is a lack of validated assessment tools for technical skills. Thus, the aim of the present study was to develop and validate a novel Global Rating Scale (GRS) for emergency medical simulation trainings.

METHODS

Following the development of the GRS, 12 teams of different experience in emergency medicine (4th year medical students, paramedics, emergency physicians) were involved in a pre-hospital emergency medicine simulation scenario and assessed by four independent raters. Subsequently, interrater reliability and construct validity of the GRS were analyzed. Moreover, the results of the GRS were cross-checked with a task specific check list. Data are presented as median (minimum; maximum).

RESULTS

The GRS consists of ten items each scored on a 5-point Likert scale yielding a maximum of 50 points. The median score achieved by novice teams was 22.75 points (17;30), while experts scored 39.00 points (32;47). The GRS overall scores significantly discriminated between student-guided teams and expert teams of emergency physicians (p = 0.005). Interrater reliability for the GRS was high with a Kendall's coefficient of concordance W ranging from 0.64 to 0.90 in 9 of 10 items and 0.88 in the overall score.

CONCLUSION

The GRS represents a promising novel tool to objectively assess technical skills in simulation training with high construct validity and interrater reliability in this pilot study.

The desktop, or the top of the desk? The relative usefulness of household features for personal health information management

Sixty percent of the US population manages at least one chronic illness. For these patients, personal health information management (PHIM) is an integral part of daily life, and largely occurs within the home. However, the way in which the home supports PHIM has not been systematically investigated. The present study examined how members of the diabetic population use features of the home environment to support PHIM. Participants (N = 60) explored a simulated home environment, the VR CAVE, and identified the most useful features for performing three examples of PHIM tasks. The computer was perceived as the most useful feature for PHIM. However, perceived usefulness of features varied based on the PHIM task performed and the rooms in which features appeared. We conclude that a detailed study of the affordances of features is necessary to ease the burden of managing chronic illness, particularly diabetes mellitus, in the sociotechnical system of the home.

An Analysis of the Efficacy of Different Teaching Modalities in Imparting Adult Cardiopulmonary Resuscitation Skills among First-year Medical Students: A Pilot Study

INTRODUCTION

Our current medical curriculum devotes a large percentage of time to knowledge acquisition by means of didactic lectures. Psychomotor skill acquisition takes a back seat. Certain lifesaving skills like basic life support skill training have not even made an appearance in the current curriculum. Equal time distribution to cognitive and psychomotor skills should be allotted for MBBS trainees, which is a very practical subject. Simulation can prove to be a valuable tool in imparting skill training. The present study aims to evaluate the efficacy of different teaching modalities in imparting lifesaving skills among first-year MBBS students.

MATERIALS AND METHODS

This cross-sectional study was conducted among 33 first-year students who consented to participate. Approval was obtained from the institutional ethics committee. The students were divided into three groups, each undergoing either didactic lecture or animation-based videos or simulation studies. Pretest, posttest, and skills tests were administered to them. One-way analysis of variance (ANOVA) and paired t test were the statistical tests employed using SPSS version 21.

RESULTS

The pretest and posttest scores were comparable in the three groups while the improvement in the posttest scores in all the three groups was significant. The skills test was significantly better in the group undergoing simulation training compared to the other groups.

CONCLUSION

Didactic, animation, and simulation are all good methods in imparting cognitive knowledge, but simulation is the method of choice in imparting psychomotor skills.

CLINICAL SIGNIFICANCE

An overhauling of the medical curriculum to include more skills training to the budding doctors using simulation-based techniques is recommended.

HOW TO CITE THIS ARTICLE

Suseel A, Panchu P, Abraham SV, Varghese S, George T, Joy L. An Analysis of the Efficacy of Different Teaching Modalities in Imparting Adult Cardiopulmonary Resuscitation Skills among First-year Medical Students: A Pilot Study. IJCCM 2019;23(11): 509-512.

Registered nurses psychophysiological stress and confidence during high-fidelity emergency simulation: Effects on performance

INTRODUCTION

Simulation has been used extensively to train students and health professionals in the assessment and early intervention of patients with acutely deteriorating conditions. These simulations evoke psychophysiological stress in learners which may affect performance. We examined the relationship between stress variables, confidence, and performance during repeated scenarios in clinically-based emergency simulations.

METHODS

Twenty-six registered nurses completed three simulation scenarios focussing on life-threatening clinical events in a single group pre-test/post-test study design. Trait anxiety was measured at baseline. Visual analogue ratings of anxiety and stress were measured before ('pre'), recalled 'during', and immediately following ('post') each simulation scenario, with a self-rating of confidence completed after each simulation scenario. Heart rate was measured continuously throughout the simulation program. Participants self-rated their clinical performance prior to and following the simulation program ('pre' and 'post').

RESULTS

Participants' trait anxiety was not elevated at baseline (mean: 39.6, SD 6.1). Across the three simulation scenarios, anxiety and stress was elevated 'during' simulation compared to 'pre' and 'post' time points. However, the magnitude of elevation of stress and anxiety during all time points ('pre', 'during' and 'post' simulation) decreased significantly (p < 0.05) with progressive simulations. Heart rate increased significantly during all simulations compared to 'pre'-levels but returned to similar levels following the simulation. The amount of increase in heart rate over progressive simulations was attenuated during simulation 3 compared with 1 and 2 (Sim 1: 103.6 bpm (SD 22.1), Sim 2: 101.9 bpm (SD 18.9), and Sim 3: 99.5 bpm (SD 23.4)). Confidence increased across the three simulations (p < 0.001), with most of the increase observed after the first two simulations. Performance scores increased by 19.0% 'pre-post' simulation program (p < 0.001) and were not confounded by previous ALS or simulation experience.

DISCUSSION

We observed temporal-dependent changes in psychophysiological stress variables across the simulation scenarios, with decreased magnitudes of elevations of psychological (self-reported anxiety and stress) and physiological (heart rate) stress variables during successive simulation scenarios. This study has shown that simulation increased stress, especially before and during scenarios; however, the learning effect decreased the magnitude of the stress response with repeated simulation scenarios. Simulation educators need to create simulations that change stress in a purposeful manner to enhance learning.

Home is where the head is: a distributed cognition account of personal health information management in the home among those with chronic illness

Managing chronic illness requires personal health information management (PHIM) to be performed by lay individuals. Paramount to understanding the PHIM process is understanding the sociotechnical system in which it frequently occurs: the home environment. We combined distributed cognition theory and the patient work system model to investigate how characteristics of the home interact with the cognitive work of PHIM. We used a 3D virtual reality CAVE that enabled participants who had been diagnosed with diabetes (N = 20) to describe how they would perform PHIM in the home context. We found that PHIM is distinctly cognitive work, and rarely performed 'in the head'. Rather, features of the physical environment, tasks, people, and tools and technologies present, continuously shape and are shaped by the PHIM process. We suggest that approaches in which the individual (sans context) is considered the relevant unit of analysis overlook the pivotal role of the environment in shaping PHIM. Practitioner Summary: We examined how Personal Health Information Management (PHIM) is performed in the homes of diabetic patients. We found that approaches to studying cognition that focus on the individual, to the exclusion of their context, overlook the pivotal role of environmental, social, and technological features in shaping PHIM.

Comparison between Simulation-based Training and Lecture-based Education in Teaching Situation Awareness. A Randomized Controlled Study

RATIONALE

Situation awareness has been defined as the perception of the elements in the environment within volumes of time and space, the comprehension of their meaning, and the projection of their status in the near future. Intensivists often make time-sensitive critical decisions, and loss of situation awareness can lead to errors. It has been shown that simulation-based training is superior to lecture-based training for some critical scenarios. Because the methods of training to improve situation awareness have not been well studied in the medical field, we compared the impact of simulation vs. lecture training using the Situation Awareness Global Assessment Technique (SAGAT) score.

OBJECTIVES

To identify an effective method for teaching situation awareness.

METHODS

We randomly assigned 17 critical care fellows to simulation vs. lecture training. Training consisted of eight cases on airway management, including topics such as elevated intracranial pressure, difficult airway, arrhythmia, and shock. During the testing scenario, at random times between 4 and 6 minutes into the simulation, the scenario was frozen, and the screens were blanked. Respondents then completed the 28 questions on the SAGAT scale. Sample items were categorized as Perception, Projection, and Comprehension of the situation. Results were analyzed using SPSS Version 21.

RESULTS

Eight fellows from the simulation group and nine from the lecture group underwent simulation testing. Sixty-four SAGAT scores were recorded for the simulation group and 48 scores were recorded for the lecture group. The mean simulation vs. lecture group SAGAT score was 64.3 ± 10.1 (SD) vs. 59.7 ± 10.8 (SD) (P = 0.02). There was also a difference in the median Perception ability between the simulation vs. lecture groups (61.1 vs. 55.5, P = 0.01). There was no difference in the median Projection and Comprehension scores between the two groups (50.0 vs. 50.0, P = 0.92, and 83.3 vs. 83.3, P = 0.27).

CONCLUSIONS

We found a significant, albeit modest, difference between simulation training and lecture training on the total SAGAT score of situation awareness mainly because of the improvement in perception ability. Simulation may be a superior method of teaching situation awareness.

Virtual reality-based assessment of basic laparoscopic skills using the Leap Motion controller

BACKGROUND

The majority of the current surgical simulators employ specialized sensory equipment for instrument tracking. The Leap Motion controller is a new device able to track linear objects with sub-millimeter accuracy. The aim of this study was to investigate the potential of a virtual reality (VR) simulator for assessment of basic laparoscopic skills, based on the low-cost Leap Motion controller.

METHODS

A simple interface was constructed to simulate the insertion point of the instruments into the abdominal cavity. The controller provided information about the position and orientation of the instruments. Custom tools were constructed to simulate the laparoscopic setup. Three basic VR tasks were developed: camera navigation (CN), instrument navigation (IN), and bimanual operation (BO). The experiments were carried out in two simulation centers: MPLSC (Athens, Greece) and CRESENT (Riyadh, Kingdom of Saudi Arabia). Two groups of surgeons (28 experts and 21 novices) participated in the study by performing the VR tasks. Skills assessment metrics included time, pathlength, and two task-specific errors. The face validity of the training scenarios was also investigated via a questionnaire completed by the participants.

RESULTS

Expert surgeons significantly outperformed novices in all assessment metrics for IN and BO (p < 0.05). For CN, a significant difference was found in one error metric (p < 0.05). The greatest difference between the performances of the two groups occurred for BO. Qualitative analysis of the instrument trajectory revealed that experts performed more delicate movements compared to novices. Subjects' ratings on the feedback questionnaire highlighted the training value of the system.

CONCLUSIONS

This study provides evidence regarding the potential use of the Leap Motion controller for assessment of basic laparoscopic skills. The proposed system allowed the evaluation of dexterity of the hand movements. Future work will involve comparison studies with validated simulators and development of advanced training scenarios on current Leap Motion controller.

Missing mediated interruptions in manual assembly: Critical aspects of breakpoint selection

The factory of the future aims to make manufacturing more effective and easily customisable, using advanced sensors and communications to support information management. In this paper, we examine how breakpoint selection during interruption management can fail, even when using recommendations for interruption management from existing research. We present an experiment based on prior work where mediated interruptions (i.e. smart interruptions that should interrupt at opportune moments) were missed by participants when sent at one of two pre-defined breakpoints. These breakpoints were selected based on existing research to minimise the cost of interruption, which can involve longer times to complete tasks as well as making errors on tasks. Missing mediated interruptions in this way was unexpected, and the prior study was not configured to measure this effect, which has led to the experiment detailed here. We strive to explore whether there is a risk of missing notifications when mediated interruptions are used, and how this is affected by breakpoint selection. This was investigated through an experiment that uses tasks and environments that simulate a manufacturing assembly facility. The results indicate that the effect exists, i.e. that participants miss significantly more notifications when interrupted at fine breakpoints than when interrupted at coarse breakpoints. An embodied cognition perspective was used for analysis of the tasks to understand the cause of the effect. This analysis shows that an overlap between "action" and "anticipation of action" can account for why participants miss notifications at fine breakpoints. Based on these findings, recommendations were developed for designing interruption systems that minimise the costs (errors and time) imposed by interruptions during assembly tasks in manufacturing.

Coordinating the interruption of assembly workers in manufacturing

This paper examines how interruptions from information and communications technology systems affect errors and the time to complete tasks for assembly workers. Interruptions have previously been examined in laboratory experiments and office environments, but not much work has been performed in other authentic environments. This paper contains the results of an experiment that was performed in a simulated manufacturing assembly environment, which tested the effects of interruptions on a manual assembly task. The experiment used existing interruption coordination methods as a basis, and the results showed a difference in the effect of interruptions and interruption coordination between cognitively complex laboratory tasks and manual assembly tasks in an authentic environment. Most notably, the negative effects of interruptions delivered without consideration were smaller in this experiment. Based on these findings, recommendations were developed for designing interruption systems for minimizing the costs (errors and time) imposed by interruptions during assembly tasks in manufacturing.

Achieving Interface and Environment Fidelity in the Virtual Basic Laparoscopic Surgical Trainer

Virtual reality trainers are educational tools with great potential for laparoscopic surgery. They can provide basic skills training in a controlled environment and free of risks for patients. They can also offer objective performance assessment without the need for proctors. However, designing effective user interfaces that allow the acquisition of the appropriate technical skills on these systems remains a challenge. This paper aims to examine a process for achieving interface and environment fidelity during the development of the Virtual Basic Laparoscopic Surgical Trainer (VBLaST). Two iterations of the design process were conducted and evaluated. For that purpose, a total of 42 subjects participated in two experimental studies in which two versions of the VBLaST were compared to the accepted standard in the surgical community for training and assessing basic laparoscopic skills in North America, the FLS box-trainer. Participants performed 10 trials of the peg transfer task on each trainer. The assessment of task performance was based on the validated FLS scoring method. Moreover, a subjective evaluation questionnaire was used to assess the fidelity aspects of the VBLaST relative to the FLS trainer. Finally, a focus group session with expert surgeons was conducted as a comparative situated evaluation after the first design iteration. This session aimed to assess the fidelity aspects of the early VBLaST prototype as compared to the FLS trainer. The results indicate that user performance on the earlier version of the VBLaST resulting from the first design iteration was significantly lower than the performance on the standard FLS box-trainer. The comparative situated evaluation with domain experts permitted us to identify some issues related to the visual, haptic and interface fidelity on this early prototype. Results of the second experiment indicate that the performance on the second generation VBLaST was significantly improved as compared to the first generation and not significantly different from that of the standard FLS box-trainer. Furthermore, the subjects rated the fidelity features of the modified VBLaST version higher than the early version. These findings demonstrate the value of the comparative situated evaluation sessions entailing hands on reflection by domain experts to achieve the environment and interface fidelity and training objectives when designing a virtual reality laparoscopic trainer. This suggests that this method could be used successfully in the future to enhance the value of VR systems as an alternative to physical trainers for laparoscopic surgery skills. Some recommendations on how to use this method to achieve the environment and interface fidelity of a VR laparoscopic surgical trainer are identified.