Using epistemic network analysis to identify targets for educational interventions in trauma team communication

Taking Stock and Looking Ahead: Evolution of Accreditation Feedback for Simulation Centers Over 8 Years Using Epistemic Network Analysis

INTRODUCTION

Since 2005, the American College of Surgeons Accredited Educational Institutes has provided accreditation of surgically focused simulation centers with the added benefit of identifying best practices defined as areas far exceeding the accreditation standards or novel methods of advancing high-quality, impactful education. This study aimed to examine the evolution of the best practices observed by accreditors during site visits over the 8-year period.

METHODS

Accreditation included the completion of an application form followed by a site visit by a simulation expert and review of all materials by an accreditation committee to identify areas out of compliance along with areas far exceeding accreditation standards. These are termed "best practices." To evaluate the evolution of accreditation feedback and embedded associations, the compiled list of 337 best practices identified from all 247 site visits over an 8-year period was analyzed and visualized using epistemic network analysis, a quantitative ethnographic technique for modeling the structure of connections in qualitative data.

RESULTS

The overall association network of the data indicates that the strongest associations were between assessment, curriculum development, faculty development, research, and teaching methods, demonstrating a highly interconnected model of accreditation feedback. Best practices evolved from an early focus on teaching methods, faculty, and curriculum development to more advanced educational topics including assessment, research, resources, and overall center governance. Distribution of associations also increased over the 8-year period with more nuanced and interconnected statements demonstrating higher-level feedback including explanations, contributing factors, impact on other areas, and, in some cases, recommendations to share best practices outside the organization.

CONCLUSIONS

The epistemic network analysis of this 8-year database of simulation center feedback provides a novel perspective on an organization and the evolving field of simulation from an optional to essential modality in healthcare professions education.

A scoping review of nontechnical skill assessment tools to evaluate trauma team performance

BACKGROUND

The objective of this scoping review was to identify assessment tools of trauma team performance (outside of technical skills) and assess the validity and reliability of each tool in assessing trauma team performance.

METHODS

We searched Embase, Cochrane Library, Web of Science, Ovid Medline, and Cumulative Index to Nursing and Allied Health Literature from inception to June 1, 2021. English studies that evaluated trauma team performance using nontechnical skill assessment tools in a simulation or real-world setting were included. Studies were assessed by two independent reviewers for meeting inclusion/exclusion criteria. Data regarding team assessment tools were extracted and synthesized into behavior domains. Each tool was then assessed for validity and reliability.

RESULTS

The literature search returned 4,215 articles with 29 meeting inclusion criteria. Our search identified 12 trauma team performance assessment tools. Most studies were conducted in the United States (n = 20 [69%]). Twenty studies (69%) assessed trauma team performance in a simulation setting; Team Emergency Assessment Measure (TEAM) and Trauma Nontechnical Skills Scale (T-NOTECHS) were the only tools to be applied in a simulation and real-world setting. Most studies assessed trauma team performance using video review technology (n = 17 [59%]). Five overarching themes were designed to encompass behavioral domains captured across the 12 tools: (1) Leadership, (2) Communication, (3) Teamwork, (4) Assessment, and (5) Situation Awareness. The reliability and validity of T-NOTECHS were investigated by the greatest number of studies (n = 13); however, TEAM had the most robust evidence of reliability and validity.

CONCLUSION

We identified 12 trauma team performance tools that assessed nontechnical skills to varying degrees. Trauma Nontechnical Skills Scale and TEAM tools had the most evidence to support their reliability and validity. Considering the limited research in the impact of trauma team performance on patient outcomes, future studies could use video review technology in authentic trauma cases to further study this important relationship.

LEVEL OF EVIDENCE

Systematic reviews and meta-analyses, level IV.

An analysis of trauma team communication using the verbal response mode taxonomy

BACKGROUND

Although 40 years has passed since the Institute of Medicine released its report "To Err Is Human," error counts are still high in healthcare. The understanding and training of nontechnical skills and teamwork thus remains a pertinent area for improvement. Most evaluation of nontechnical skills of trauma teams takes place in simulation rooms. The aim of this study was to determine if real trauma resuscitation communication could be analyzed using the speech classification system of verbal response modes, otherwise known as the verbal response mode taxonomy and, if so, if there is a predominant approach of verbally delivering messages.

METHODS

Video and audio recordings of 5 trauma team resuscitations were transcribed. Communication was coded using the verbal response mode taxonomy for both form and intent. The rate of mixed-mode communication (unmatched form and intent) and pure-mode communication were calculated and compared between the participants roles. Comparisons were made with simulated material published in other research.

RESULTS

The most frequent mixed-mode communication was acknowledgment in service of confirmation. Question in service of a question was the most used pure-mode communication. Six predominant roles were seen, which matched well with the roles in the simulations.

CONCLUSION

The verbal response mode taxonomy can be used to study communication during real trauma resuscitation, and it was found that pure-mode communication was predominant, meaning that the grammatical form matches the intent. Verbal response mode methodology is time consuming and requires analysts with domain knowledge. Comparisons show some differences between simulations and our material indicating that verbal response modes can be used to evaluate differences in communication.

Costs of hospital trauma team simulation training: a prospective cohort study

OBJECTIVES

This study investigated the costs of 2-hour multiprofessional in situ hospital trauma team simulation training and its effects on teams' non-technical skills using the T-NOTECHS instrument.

BACKGROUND

Simulation is a feasible and effective teaching and learning method. Calculating the costs of simulated trauma team training in medical emergency situations can yield valuable information for improving its overall cost-effectiveness.

DESIGN

A prospective cohort study.

SETTING

Trauma resuscitation room in Central Finland Hospital, Finland.

PARTICIPANTS

475 medical professionals in 81 consecutive, simulated trauma teams.

PRIMARY AND SECONDARY OUTCOME MEASURES

Team simulation training costs in 2017 and 2018 were analysed in the following two phases: (1) start-up costs and (2) costs of education. Primary outcome measures were training costs per participant and training costs per team. Secondary outcome measures were non-technical skills, which were measured on a 5-25-point scale using the T-NOTECHS instrument.

RESULTS

The annual mean total costs of trauma team simulation training were €58 000 for 40 training sessions and 238 professionals. Mean cost per participant was €203. Mean cost per team was €1220. The annual costs of simulation training markedly decreased when at least 70-80 teams participated in the training. Mean change in T-NOTECHS score after simulation training was +2.86 points (95% CI 1.97 to 3.75;+14.5%).

CONCLUSIONS

The greater the number of teams trained per year, the lower the costs per trauma team. In this study, we developed an activity-based costing method to calculate the costs of trauma team simulation training to help stakeholders make decisions about whether to initiate or increase existing trauma team simulation training or to obtain these services elsewhere.

Epistemic Network Analyses of Economics Students' Graph Understanding: An Eye-Tracking Study

Learning to solve graph tasks is one of the key prerequisites of acquiring domain-specific knowledge in most study domains. Analyses of graph understanding often use eye-tracking and focus on analyzing how much time students spend gazing at particular areas of a graph-Areas of Interest (AOIs). To gain a deeper insight into students' task-solving process, we argue that the gaze shifts between students' fixations on different AOIs (so-termed transitions) also need to be included in holistic analyses of graph understanding that consider the importance of transitions for the task-solving process. Thus, we introduced Epistemic Network Analysis (ENA) as a novel approach to analyze eye-tracking data of 23 university students who solved eight multiple-choice graph tasks in physics and economics. ENA is a method for quantifying, visualizing, and interpreting network data allowing a weighted analysis of the gaze patterns of both correct and incorrect graph task solvers considering the interrelations between fixations and transitions. After an analysis of the differences in the number of fixations and the number of single transitions between correct and incorrect solvers, we conducted an ENA for each task. We demonstrate that an isolated analysis of fixations and transitions provides only a limited insight into graph solving behavior. In contrast, ENA identifies differences between the gaze patterns of students who solved the graph tasks correctly and incorrectly across the multiple graph tasks. For instance, incorrect solvers shifted their gaze from the graph to the x-axis and from the question to the graph comparatively more often than correct solvers. The results indicate that incorrect solvers often have problems transferring textual information into graphical information and rely more on partly irrelevant parts of a graph. Finally, we discuss how the findings can be used to design experimental studies and for innovative instructional procedures in higher education.