Gathering Validity Evidence to Adapt the Non-technical Skills for Surgeons (NOTSS) Assessment Tool to the United States Context

Nontechnical Skills for Surgeons as a Framework to Evaluate Cardiopulmonary Bypass Management Skills of Resident Trainees

BACKGROUND

Nontechnical skills are critical in cardiac surgery but currently there is no formal paradigm to teach these in residency training. We investigated the use of the Nontechnical skills for surgeons (NOTSS) system as a framework to assess and teach nontechnical skills related to cardiopulmonary bypass (CPB) management.

METHODS

Single-center retrospective analysis of Integrated and Independent pathway thoracic surgery residents who participated in dedicated nontechnical skills evaluation and training. Two CPB management simulation scenarios were utilized. All residents received a lecture on CPB fundamentals and then individually participated in the first simulation ("Pre-NOTSS"). Immediately following this, nontechnical skills were rated by self-assessment and by a NOTSS trainer. All residents then underwent group NOTSS training followed by the second individual simulation ("Post-NOTSS"). Nontechnical skills were rated as before. NOTSS categories assessed included Situation Awareness, Decision Making, Communication and Teamwork, and Leadership.

RESULTS

Nine residents were divided into 2 groups: Junior (n = 4, PGY1-4) and Senior (n = 5, PGY5-8). Pre-NOTSS resident self-ratings were higher for Senior than Junior in the categories of Decision Making, Communication and Teamwork, and Leadership while trainer ratings were similar between the groups. Post-NOTSS, resident self-ratings were higher for Senior than Junior in Situation Awareness and Decision Making while trainer scores were higher for both groups in Communication and Teamwork and Leadership.

CONCLUSIONS

The NOTSS framework in conjunction with simulation scenarios provides a practical framework to evaluate and teach nontechnical skills related to CPB management. NOTSS training can lead to improvements in both subjective and objective ratings of nontechnical skills for all PGY levels.

The development of a tool to assess medical students' non-technical skills - The Norwegian medical students' non-technical skills (NorMS-NTS)

PURPOSE

New physicians need to master non-technical skills (NTS), as high levels of NTS have been shown to increase patient safety. It has also been shown that NTS can be improved through training. This study aimed to establish the necessary NTS for Norwegian medical students to create a tool for formative and summative assessments.

METHODS

Focus group interviews were conducted with colleagues and patients of newly graduated physicians. Interviews were then analyzed using card sort methods, and the identified NTS were used to establish a framework. Focus groups commented on a prototype of an NTS assessment tool. Finally, we conducted a search of existing tools and literature. The final tool was developed based on the combined inputs.

RESULTS

We created Norwegian medical students' non-technical skills (NorMS-NTS) assessment tool containing four main categories; together comprising 13 elements and a rating scale for the NTS of the person observed.

CONCLUSIONS

The NorMS-NTS represents a purpose-made tool for assessing newly graduated physicians' NTS. It is similar to existing assessment tools but based on domain-specific user perspectives obtained through focus group interviews and feedback, integrated with results from a literature search, and with consideration of existing NTS tools.

Integrative Review of Non-Technical Skills Frameworks to Apply for Air Medical Transfer of Pregnant Women

OBJECTIVE

To identify the most suitable non-technical skills framework to adapt and apply to the air medical transfer of pregnant women.

DATA SOURCES

Embase, PsycINFO, PubMed, MEDLINE, Web of Science, CINAHL, Science Direct, and Google Scholar.

STUDY SELECTION

We retrieved potentially relevant articles using a predefined combination of keywords extended with truncation and Boolean operators. Database and manual reference searches yielded 569 peer-reviewed articles. We included articles if they presented empirical data and described non-technical or cognitive competency skills frameworks for health care professionals. We discussed any ambiguities regarding inclusion, and they were resolved by consensus. We retained 71 full-text articles for final review.

DATA EXTRACTION

We coded extracted data under four criteria: non-technical skill categories, context of use, psychometric properties, and rating system. We generated descriptive summary tables of the characteristics of existing non-technical skills frameworks based on publication year, method of development, clinical setting, clinical specialty, routine/crisis-based performance, and team/individual performance.

DATA SYNTHESIS

We identified 42 non-technical skills frameworks from a variety of health care settings. We critically examined context of use and how use in various clinical settings may align with air transfers of pregnant women. Our findings illustrate the importance of team-based and routine performance rather than crisis-focused skills. Maintaining situational awareness throughout all stages of the transfer and communicating effectively with team members, the pregnant woman, and her partner are skills that are particularly important to ensure good outcomes.

CONCLUSION

We selected the Global Assessment of Obstetric Team Performance as the most suitable non-technical skills framework to adapt to the clinical setting of air medical transfer of pregnant women. We considered the clinical specialty, specific non-technical skills required in the setting, the framework's properties, and the requirement to focus on routine team performance.

Team emergency assessment measure (TEAM) of non-technical skills: The Brazilian Portuguese version of the TEAM tool

OBJECTIVES

The aim of this study was to conduct the translation and cross-cultural adaptation of the original Team Emergency Assessment Measure (TEAM) tool into the Brazilian Portuguese language and investigate the internal consistency, inter-rater reliability, and concurrent validity of this new version (bp-TEAM).

METHODS

Independent medical translators performed forward and backward translations of the TEAM tool between English and Portuguese, creating the bp-TEAM. The authors selected 23 videos from final-year medical students during in-situ emergency simulations. Three independent raters assessed all the videos using the bp-TEAM and provided a score for each of the 12 items of the tool. The authors assessed the internal consistency and the inter-rater reliability of the tool.

RESULTS

Raters assessed all 23 videos. Internal consistency was assessed among the 11 items of the bp-TEAM from one rater, yielding a Cronbach's alpha of 0.89. inter-item correlation analysis yielded a mean correlation coefficient rho of 0.46. Inter-rater reliability analysis among the three raters yielded an intraclass correlation coefficient of 0.86 (95% CI 0.83‒0.89), p < 0.001.

CONCLUSION

The Brazilian Portuguese version of the TEAM tool presented acceptable psychometric properties, similar to the original English version.

Teaching and Evaluating Nontechnical Skills for General Surgery

Surgical training programs have long used quantitative measures of knowledge, as well as subjective evaluation of technical skills, to define the competence of trainees. However, a growing body of literature has shown the importance of nontechnical surgical skills as vital components of quality surgical care. Institutions must train nontechnical surgical skills, including leadership, communication, teamwork, situational awareness, and decision making, and incorporate these attributes into their evaluative processes to maximally enhance surgical performance at every career stage.