Do novices display automaticity during simulator training?

Neurocognitive Concepts of Arthroscopic Surgical Training

» Arthroscopy is used to treat a broad variety of orthopaedic conditions.» The technical aspects of arthroscopic surgery are distinct from traditional open surgery and require different approaches to education and training.» There are neurocognitive concepts related to learning, memory, and performance that are recognized and understood in many fields and relevant but not commonly and specifically considered in orthopaedic surgery.» The purpose of this review was to introduce and discuss neurocognitive principles and concepts of visuospatial motor skill acquisition and proficiency to provide a background to support the development of arthroscopy educational curricula and training.

[Mental Workload Assessment of Health Care Staff by NASA-TLX]

The National Aeronautics Space Administration Task Load Index (NASA-TLX) is the most frequently used mental workload assessment method. This article reviews 26 papers which report mental workload evaluation in health care staff, categorizing them into laparoscopic surgery, anesthesia, ICU, electronic record, patient controlled analgesia, emergency, display and others. Although indices other than NASA-TLX were also used in these papers, this review describes the results of NASA-TLX only.

Training with cognitive load improves performance under similar conditions in a real surgical task

BACKGROUND

Enhancing cognitive load while performing a bimanual surgical task affects performance. Whether repeated training under this condition could benefit performance in an operating room was tested using a virtual reality simulator with cognitive load applied through two-digit math multiplication questions.

METHOD

11 subjects were randomized to Control, VR and VR + CL groups. After a pre-test, VR and VR + CL groups repeated the peg transfer task 150 times over 15 sessions with cognitive load applied only for the last 100 trials. After training, all groups took a post-test and two weeks later the retention test with and without cognitive load and the transfer task on a pig intestine of 150 cm long under cognitive load.

RESULTS AND CONCLUSION

Mixed ANOVA analysis showed significant differences between the control and VR and VR + CL groups (p = 0.013, p = 0.009) but no differences between the VR + CL and the VR groups (p = 1.0). GOALS bimanual dexterity score on transfer test show that VR + CL group outperformed both Control and VR groups (p = 0.016, p = 0.03). Training under cognitive load benefitted performance on an actual surgical task under similar conditions.

Development of a fundamentals of endoscopic surgery proficiency-based skills curriculum for general surgery residents

BACKGROUND

Fundamentals of Endoscopic Surgery (FES) has become a board certification requirement for general surgery residents. While the FES program provides a robust didactic curriculum, an endoscopic skills curriculum is lacking for this high stakes assessment. The aims of this study were to develop a proficiency-based endoscopic skills curriculum and assess its effectiveness on success in the FES exam.

METHODS

Endoscopy experts developed a multiple-choice questionnaire based on the FES online didactics. Five training cases from the GI Mentor II simulator were selected, and expert performance defined proficiency levels for each case. Participating surgery residents were required to review online didactics and achieve proficiency twice on selected simulator cases. The multiple-choice questionnaire, simulator-generated metrics of two endoscopy cases, Global Assessment of Gastrointestinal Endoscopic Skills (GAGES), NASA-Task Load Index (TLX), and the manual portion of the FES exam were used for assessment before and after training. The curriculum was implemented either alongside a clinical endoscopy rotation or independent of the rotation. Clinical endoscopic skills of participants with a dedicated rotation were assessed using GAGES.

RESULTS

Twenty-eight general surgery residents (PGY 2-5) participated in the study, of which 25 (89%) completed the curriculum. Scores of the multiple-choice questionnaire and all simulator-generated metrics improved in the post-training assessment, with the exception of the percentage of mucosal surface examined, which was slightly decreased. Simulated and clinical GAGES scores and the NASA-TLX score improved after training. Performance scores on four of five FES exam tasks were significantly improved.

CONCLUSIONS

The proficiency-based endoscopic skills curriculum was successfully implemented both alongside the clinical endoscopy rotation and independent of the rotation. Participating residents acquired skills to pass the FES exam. This curriculum will be valuable to general surgery residency programs.

Attentional selectivity, automaticity, and self-efficacy predict simulator-acquired skill transfer to the clinical environment

INTRODUCTION

Several studies demonstrated that simulator-acquired skill transfer to the operating room is incomplete. Our objective was to identify trainee characteristics that predict the transfer of simulator-acquired skill to the operating room.

METHODS

Trainees completed baseline assessments including intracorporeal suturing (IS) performance, attentional selectivity, self-reported use of mental skills, and self-reported prior clinical and simulated laparoscopic experience and confidence. Residents then followed proficiency-based laparoscopic skills training, and their skill transfer was assessed on a live-anesthetized porcine model. Predictive characteristics for transfer test performance were assessed using multiple linear regression.

RESULTS

Thirty-eight residents completed the study. Automaticity, attentional selectivity, resident perceived ability with laparoscopy and simulators, and post-training IS performance were predictive of IS performance during the transfer test.

CONCLUSIONS

Promoting automaticity, self-efficacy, and attention selectivity may help improve the transfer of simulator-acquired skill. Mental skills training and training to automaticity may therefore be valuable interventions to achieve this goal.

The effect of distributed virtual reality simulation training on cognitive load during subsequent dissection training

BACKGROUND

Complex tasks such as surgical procedures can induce excessive cognitive load (CL), which can have a negative effect on learning, especially for novices.

AIM

To investigate if repeated and distributed virtual reality (VR) simulation practice induces a lower CL and higher performance in subsequent cadaveric dissection training.

METHODS

In a prospective, controlled cohort study, 37 residents in otorhinolaryngology received VR simulation training either as additional distributed practice prior to course participation (intervention) (9 participants) or as standard practice during the course (control) (28 participants). Cognitive load was estimated as the relative change in secondary-task reaction time during VR simulation and cadaveric procedures.

RESULTS

Structured distributed VR simulation practice resulted in lower mean reaction times (32% vs. 47% for the intervention and control group, respectively, p < 0.01) as well as a superior final-product performance during subsequent cadaveric dissection training.

CONCLUSIONS

Repeated and distributed VR simulation causes a lower CL to be induced when the learning situation is increased in complexity. A suggested mechanism is the formation of mental schemas and reduction of the intrinsic CL. This has potential implications for surgical skills training and suggests that structured, distributed training be systematically implemented in surgical training curricula.

Validation of a Visual-Spatial Secondary Task to Assess Automaticity in Laparoscopic Skills

INTRODUCTION

Our objective was to assess reliability and validity of a visual-spatial secondary task (VSST) as a method to measure automaticity on a basic simulated laparoscopic skill model. In motor skill acquisition, expertise is defined by automaticity. The highest level of performance with less cognitive and attentional resources characterizes this stage, allowing experts to perform multiple tasks. Conventional validated parameters as operative time, objective assessment skills scales (OSATS), and movement economy, are insufficient to distinguish if an individual has reached the more advanced learning phases, such as automaticity. There is literature about using a VSST as an attention indicator that correlates with the automaticity level.

METHODS

Novices with completed and approved Fundamentals of Laparoscopic Surgery course, and laparoscopy experts were enrolled for an experimental study and measured under dual tasks conditions. Each participant performed the test giving priority to the primary task while at the same time they responded to a VSST. The primary task consisted of 4 interrupted laparoscopic stitches (ILS) on a bench-model. The VSST was a screen that showed different patterns that the surgeon had to recognize and press a pedal while doing the stitches (PsychoPsy software, Python, MacOS). Novices were overtrained on ILS until they reach at least 100 repetitions and then were retested. Participants were video recorded and then assessed by 2 blinded evaluators who measured operative time and OSATS. These scores were considered indicators of quality for the primary task. The VSST performance was measured by the detectability index (DI), which is a ratio between correct and wrong detections. A reliable evaluation was defined as two measures of DI with less than 10% of difference, maintaining the cutoff scores for performance on the primary task (operative time <110 seg and OSATS >17 points).

RESULTS

Novices (n = 11) achieved reliable measure of the test after 2 (2-5) repetitions on the preassessment and 3.75 (2-5) on the postassessment (p = 0.04); whereas laparoscopy experts (n = 4) did it after 3.5 (3-4) repetitions. Proficiency cutoff scores for the primary task were achieved on every measure for novices (prepost overtraining) and experts. Expert performance on VSST was DI 0.78 (0.69-0.87). Novice performance was significantly better on postassessment (DI-pre 0.48 [0.06-0.71] vs DI-post 0.78 [0.48-0.95], p = 0.003). Overtraining consisted in 140 (100-210) repetitions of ILS for all novices, made in 8 hours (3-15). By categorizing DI based on expert performance, novices with DI-post >0.65 achieved better OSATS score and less operative time than novices with DI-post<0.65 (p = 0.007 y, p = 0.089, respectively).

CONCLUSION

Measuring automaticity is feasible using a VSST. This instrument is reliable and has a face, content and construct validity. A DI over 0.65 may be a cutoff point correlated with high standard performance on the primary task. This instrument measures performance on laparoscopic skills, and along with conventional indicators, would better define advance levels of expertise. More studies are required applying this VSST to achieve external validity by reproducing our results.

Gaming in Nursing Education: A Literature Review

AIM

The aim of this research was to investigate videogame-based learning in nursing education and establish how videogames are currently employed and how they link to the development of decision-making, motivation, and other benefits.

BACKGROUND

Although digital game-based learning potentially offers a safe and convenient environment that can support nursing students developing essential skills, nurse educators are typically slow to adopt such resources.

METHOD

A comprehensive search of electronic databases was conducted, followed by a thematic analysis of the literature.

RESULTS

Evaluations of identified games found generally positive results regarding usability and effectiveness of videogames in nursing education. Analysis of advantages of videogames in nursing education identified potential benefits for decision-making, motivation, repeated exposure, logistical, and financial value.

CONCLUSION

Despite the paucity of games available and the methodological limitations identified, findings provide evidence to support the potential effectiveness of videogames as a learning resource in nursing education.

Evaluation of Procedural Simulation as a Training and Assessment Tool in General Surgery-Simulating a Laparoscopic Appendectomy

BACKGROUND

Laparoscopic appendectomy is a commonly performed surgical procedure, but few training models have been described for it. We examined a virtual reality module for practising a laparoscopic appendectomy.

METHODS

A prospective cohort study with the following 3 groups of surgeons (n = 45): novices (0 procedures), intermediates (10-50 procedures), and experienced (>100 procedures). After being introduced to the simulator and 1 familiarization attempt on the procedural module, the participants practiced the module 20 times. Movements, task time, and procedure-specific parameters were compared over time.

RESULTS

The time and movement parameters were significantly different during the first attempt, and more experienced surgeons used fewer movements and less time than novices (p < 0.01), although only 2 parameters were significantly different between novices and intermediates. All 3 groups improved significantly over 20 attempts (p < 0.0001). The intraclass correlation coefficient varied between 0.55 and 0.68 and did not differ significantly between the 3 groups (p > 0.05). When comparing novices with experienced surgeons, novices had a higher risk of burn damage to cecum (odds ratio [OR] = 3.0 [95% CI: 1.3; 7.0] p = 0.03), pressure damage to appendix (OR = 3.1 [95% CI: 2.0; 4.9] p < 0.0001), and grasping of the appendix (OR = 2.9 [95% CI: 1.8; 4.7] p < 0.0001). The risk of causing a perforation was not significantly different among the different experience levels (OR = 1.9 [95% CI: 0.9; 3.8] p = 0.14). Only 3 out of 5 error parameters differed significantly when comparing novices and experienced surgeons. Similarly, when comparing intermediates and novices, it was only 2 of the parameters that differed.

DISCUSSION

The simulator module for practising laparoscopic appendectomy may be useful as a training tool, but further development is required before it can be used for assessment purposes. Procedural simulation may demonstrate more variation for movement parameters, and future research should focus on developing better procedure-specific parameters.

Laparoscopic Motor Learning and Workspace Exploration

BACKGROUND

Laparoscopic surgery requires operators to learn novel complex movement patterns. However, our understanding of how best to train surgeons' motor skills is inadequate, and research is needed to determine optimal laparoscopic training regimes. This difficulty is confounded by variables inherent in surgical practice, for example, the increasing prevalence of morbidly obese patients presents additional challenges related to restriction of movement because of abdominal wall resistance and reduced intra-abdominal space. The aim of this study was to assess learning of a surgery-related task in constrained and unconstrained conditions using a novel system linking a commercially available robotic arm with specialised software creating the novel kinematic assessment tool (Omni-KAT).

METHODS

We created an experimental tool that records motor performance by linking a commercially available robotic arm with specialized software that presents visual stimuli and objectively measures movement outcome (kinematics). Participants were given the task of generating aiming movements along a horizontal plane to move a visual cursor on a vertical screen. One group received training that constrained movements to the correct plane, whereas the other group was unconstrained and could explore the entire "action space."

RESULTS

The tool successfully generated the requisite force fields and precisely recorded the aiming movements. Consistent with predictions from structural learning theory, the unconstrained group produced better performance after training as indexed by movement duration (p < 0.05).

CONCLUSION

The data showed improved performance for participants who explored the entire action space, highlighting the importance of learning the full dynamics of laparoscopic instruments. These findings, alongside the development of the Omni-KAT, open up exciting prospects for better understanding of the learning processes behind surgical training and investigate ways in which learning can be optimized.

Differences in mental workload between traditional and single-incision laparoscopic procedures measured with a secondary task

INTRO

The mental workload associated with laparoscopic suturing can be assessed with a secondary task that requires the same visual-spatial attentional resources. The purpose of this study was to use a secondary task to measure the incremental workload demands of single-incision laparoscopic surgery (SILS) procedures versus traditional laparoscopic procedures.

METHOD

12 surgery residents and surgical assistants who had met FLS criteria on an FLS and SILS simulator performed one trial each of peg transfer, cutting, and intracorporeal suturing tasks simultaneously with the secondary task and provided subjective workload ratings using the NASA-TLX.

RESULTS

SILS procedures resulted in lower primary and secondary task scores, p < 0.001 and higher workload ratings, p < 0.0001. Suturing resulted in lower primary (p < 0.003) and secondary task scores (p < 0.017) and higher workload ratings (p < 0.017) compared to the other tasks.

CONCLUSIONS

SILS procedures were significantly more mentally demanding than traditional laparoscopic procedures corroborated by primary and secondary tasks scores and subjective ratings.

Multitask training promotes automaticity of a fundamental laparoscopic skill without compromising the rate of skill learning

Background

A defining characteristic of expertise is automated performance of skills, which frees attentional capacity to better cope with some common intraoperative stressors. There is a paucity of research on how best to foster automated performance by surgical trainees. This study examined the use of a multitask training approach to promote automated, robust laparoscopic skills.

Methods

Eighty-one medical students completed training of a fundamental laparoscopic task in either a traditional single-task training condition or a novel multitask training condition. Following training, participants’ laparoscopic performance was tested in a retention test, two stress transfer tests (distraction and time pressure) and a secondary task test, which was included to evaluate automaticity of performance. The laparoscopic task was also performed as part of a formal clinical examination (OSCE).

Results

The training groups did not differ in the number of trials required to reach task proficiency (p = .72), retention of skill (ps > .45), or performance in the clinical examination (p = .14); however, the groups did differ with respect to the secondary task (p = .016). The movement efficiency (number of hand movements) of single-task trainees, but not multitask trainees, was negatively affected during the secondary task test. The two stress transfer tests had no discernable impact on the performance of either training group.

Conclusion

Multitask training was not detrimental to the rate of learning of a fundamental laparoscopic skill and added value by providing resilience in the face of a secondary task load, indicative of skill automaticity. Further work is needed to determine the extent of the clinical utility afforded by multitask training.

Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings

As interest in applying cognitive load theory (CLT) to the study and design of pedagogic and technological approaches in healthcare simulation grows, suitable measures of cognitive load (CL) are needed. Here, we report a two-phased study investigating the sensitivity of subjective ratings of mental effort (SRME) and secondary-task performance (signal detection rate, SDR and recognition reaction time, RRT) as measures of CL. In phase 1 of the study, novice learners and expert surgeons attempted a visual-monitoring task under two conditions: single-task (monitoring a virtual patient's heart-rate) and dual-task (tying surgical knots on a bench-top simulator while monitoring the virtual patient's heart-rate). Novices demonstrated higher mental effort and inferior secondary-task performance on the dual-task compared to experts (RRT 1.76 vs. 0.73, p = 0.012; SDR 0.27 vs. 0.97, p < 0.001; SRME 7.75 vs. 2.80, p < 0.001). Similarly, secondary task performance deteriorated from baseline to dual-task among novices (RRT 0.63 vs. 1.76 s, p < 0.006 and SDR 1.00 vs. 0.27, p < 0.001), but not experts (RRT 0.63 vs. 0.73 s, p = 0.124 and SDR 1.00 vs. 0.97, p = 0.178). In phase 2, novices practiced surgical knot-tying on the bench top simulator during consecutive dual-task trials. A significant increase in SDR (F(9,63) = 6.63, p < 0.001, f = 0.97) and decrease in SRME (F(9,63) = 9.39, p < 0.001, f = 1.04) was observed during simulation training, while RRT did not change significantly (F(9,63) = 1.18, p < 0.32, f = 0.41). The results suggest subjective ratings and dual-task performance can be used to track changes in CL among novices, particularly in early phases of simulation-based skills training. The implications for measuring CL in simulation instructional design research are discussed.

Measuring cognitive load: performance, mental effort and simulation task complexity

CONTEXT

Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation.

OBJECTIVES

We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training.

METHODS

We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task.

RESULTS

Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant.

CONCLUSIONS

Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.

Cognitive load in distributed and massed practice in virtual reality mastoidectomy simulation

OBJECTIVES/HYPOTHESIS

Cognitive load theory states that working memory is limited. This has implications for learning and suggests that reducing cognitive load (CL) could promote learning and skills acquisition. This study aims to explore the effect of repeated practice and simulator-integrated tutoring on CL in virtual reality (VR) mastoidectomy simulation.

STUDY DESIGN

Prospective trial.

METHODS

Forty novice medical students performed 12 repeated virtual mastoidectomy procedures in the Visible Ear Simulator: 21 completed distributed practice with practice blocks spaced in time and 19 participants completed massed practice (all practices performed in 1 day). Participants were randomized for tutoring with the simulator-integrated tutor function. Cognitive load was estimated by measuring reaction time in a secondary task. Data were analyzed using linear mixed models for repeated measurements.

RESULTS

The mean reaction time increased by 37% during the procedure compared with baseline, demonstrating that the procedure placed substantial cognitive demands. Repeated practice significantly lowered CL in the distributed practice group but not in massed practice group. In addition, CL was found to be further increased by 10.3% in the later and more complex stages of the procedure. The simulator-integrated tutor function did not have an impact on CL.

CONCLUSION

Distributed practice decreased CL in repeated VR mastoidectomy training more consistently than was seen in massed practice. This suggests a possible effect of skills and memory consolidation occurring over time. To optimize technical skills learning, training should be organized as time-distributed practice rather than as a massed block of practice, which is common in skills-training courses.

LEVEL OF EVIDENCE

N/A.

Manual Skill Acquisition During Transesophageal Echocardiography Simulator Training of Cardiology Fellows: A Kinematic Assessment

OBJECTIVE

To investigate whether a transesophageal echocardiography (TEE) simulator with motion analysis can be used to impart proficiency in TEE in an integrated curriculum-based model.

DESIGN

A prospective cohort study.

SETTING

A tertiary-care university hospital.

PARTICIPANTS

TEE-naïve cardiology fellows.

INTERVENTIONS

Participants underwent an 8-session multimodal TEE training program. Manual skills were assessed at the end of sessions 2 and 8 using motion analysis of the TEE simulator's probe. At the end of the course, participants performed an intraoperative TEE; their examinations were video captured, and a blinded investigator evaluated the total time and image transitions needed for each view. Results are reported as mean±standard deviation, or median (interquartile range) where appropriate.

MEASUREMENTS AND MAIN RESULTS

Eleven fellows completed the knowledge and kinematic portions of the study. Five participants were excluded from the evaluation in the clinical setting because of interim exposure to TEE or having participated in a TEE rotation after the training course. An increase of 12.95% in post-test knowledge scores was observed. From the start to the end of the course, there was a significant reduction (p<0.001 for all) in the number of probe. During clinical performance evaluation, trainees were able to obtain all the required echocardiographic views unassisted but required a longer time and had more probe transitions when compared with an expert.

CONCLUSION

A curriculum-based approach to TEE training for cardiology fellows can be complemented with kinematic analyses to objectify acquisition of manual skills during simulator-based training.

Towards immersive virtual reality (iVR): a route to surgical expertise

Surgery is characterized by complex tasks performed in stressful environments. To enhance patient safety and reduce errors, surgeons must be trained in environments that mimic the actual clinical setting. Rasmussen’s model of human behavior indicates that errors in surgical procedures may be skill-, rule-, or knowledge-based. While skill-based behavior and some rule-based behavior may be taught using box trainers and ex vivo or in vivo animal models, we posit that multimodal immersive virtual reality (iVR) that includes high-fidelity visual as well as other sensory feedback in a seamless fashion provides the only means of achieving true surgical expertise by addressing all three levels of human behavior. While the field of virtual reality is not new, realization of the goals of complete immersion is challenging and has been recognized as a Grand Challenge by the National Academy of Engineering. Recent technological advances in both interface and computational hardware have generated significant enthusiasm in this field. In this paper, we discuss convergence of some of these technologies and possible evolution of the field in the near term.

Can we become better robot surgeons through simulator practice?

INTRODUCTION

There is significant growth in the use of the robotic surgery platform in the general surgery community. Current pre-requisites for robot surgery training include performing basic tasks on a simulator and achieving a minimum overall score for each task. However, there is limited information about these tasks related to performance and time required to become proficient. We focused on critical tasks that have the highest potential for preventing inadvertent injuries, and constructed models to predict how many attempts would be needed to master the tasks depending on the user's initial attempt.

METHODS AND PROCEDURES

This study was conducted using de-identified data collected over 12 months from the dV-Trainers® simulator at our institution. We analyzed tasks used in institutional surgical robot credentialing that focused on camera manipulation and energy use. Data were extracted from the Camera Targeting, Energy Dissection, and Energy Switching exercises focusing on individual metrics such as Time to Complete Exercise, Economy of Motion, Misapplied Energy Time, and Blood Volume Loss. Mixed linear models looking at sequential attempts and specific performance metrics were constructed using IBM SPSS Statistics version 20.

RESULTS

Over 26,000 overall minutes of recorded use was logged in our simulator by more than 30 unique users across all exercises. An average of 15 users performed each of the analyzed exercises, with an average of eight attempts per exercise. Based on our models, on average most users would need four to five attempts to achieve 80 % proficiency for any given metric.

CONCLUSION

Virtual reality robotic simulators such as the dv-Trainer® can be used by general surgeons to become better robotic surgeons. Our data suggests that it can be used by a surgeon to predict how much time and effort one would need to spend on the simulator in order to become proficient with the robot, especially in critical metrics such as camera manipulation and energy application. Surgeons who require more attempts to successfully complete tasks may want to consider additional training methods, such as proctoring or hands-on laboratories, to improve robot surgery proficiency.

Simulator-based transesophageal echocardiographic training with motion analysis: a curriculum-based approach

BACKGROUND

Transesophageal echocardiography (TEE) is a complex endeavor involving both motor and cognitive skills. Current training requires extended time in the clinical setting. Application of an integrated approach for TEE training including simulation could facilitate acquisition of skills and knowledge.

METHODS

Echo-naive nonattending anesthesia physicians were offered Web-based echo didactics and biweekly hands-on sessions with a TEE simulator for 4 weeks. Manual skills were assessed weekly with kinematic analysis of TEE probe motion and compared with that of experts. Simulator-acquired skills were assessed clinically with the performance of intraoperative TEE examinations after training. Data were presented as median (interquartile range).

RESULTS

The manual skills of 18 trainees were evaluated with kinematic analysis. Peak movements and path length were found to be independent predictors of proficiency (P < 0.01) by multiple regression analysis. Week 1 trainees had longer path length (637 mm [312 to 1,210]) than that of experts (349 mm [179 to 516]); P < 0.01. Week 1 trainees also had more peak movements (17 [9 to 29]) than that of experts (8 [2 to 12]); P < 0.01. Skills acquired from simulator training were assessed clinically with eight additional trainees during intraoperative TEE examinations. Compared with the experts, novice trainees required more time (199 s [193 to 208] vs. 87 s [83 to 16]; P = 0.002) and performed more transitions throughout the examination (43 [36 to 53] vs. 21 [20 to 23]; P = 0.004).

CONCLUSIONS

A simulation-based TEE curriculum can teach knowledge and technical skills to echo-naive learners. Kinematic measures can objectively evaluate the progression of manual TEE skills.

da Vinci skills simulator for assessing learning curve and criterion-based training of robotic basic skills

OBJECTIVE

To answer 2 research questions: what are the learning curve patterns of novices on the da Vinci skills simulator parameters and what parameters are appropriate for criterion-based robotic training.

MATERIALS AND METHODS

A total of 17 novices completed 2 simulator sessions within 3 days. Each training session consisted of a warming-up exercise, followed by 5 repetitions of the "ring and rail II" task. Expert participants (n = 3) performed a warming-up exercise and 3 repetitions of the "ring and rail II" task on 1 day. We analyzed all 9 parameters of the simulator.

RESULTS

Significant learning occurred on 5 parameters: overall score, time to complete, instrument collision, instruments out of view, and critical errors within 1-10 repetitions (P <.05). Economy of motion and excessive instrument force only showed improvement within the first 5 repetitions. No significant learning on the parameter drops and master workspace range was found. Using the expert overall performance score (n = 3) as a criterion (overall score 90%), 9 of 17 novice participants met the criterion within 10 repetitions.

CONCLUSION

Most parameters showed that basic robotic skills are learned relatively quickly using the da Vinci skills simulator, but that 10 repetitions were not sufficient for most novices to reach an expert level. Some parameters seemed inappropriate for expert-based criterion training because either no learning occurred or the novice performance was equal to expert performance.

Measurement of mental workload in clinical medicine: a review study

Background:

Measures of mental workload are now commonly used in industries to identify sources of error and to improve performance.

Objectives:

This study aimed to review the evidence for the use of this technique within medicine.

Patients and Methods:

We used search engines and the internet to identify experimental studies that included a measure of mental workload in medical practitioners or trainees/students. Studies that aimed to measure mental “stress” as a disorder, or “productivity” were excluded. Each abstract and then the full paper were appraised prior to inclusion.

Results:

Thirty-three studies were identified that matched the inclusion criteria. Although these covered a variety of settings, common methods were identifiable. The results support the concept of mental workload measurement as an important factor in medical performance.

Conclusions:

The limited number of studies and the variety of definitions and measurement techniques used in these studies, make direct comparisons difficult. However, the utility of this methodology in medical education appears to have been established, and guidelines for further research methods are proposed.

Hands-free administration of subjective workload scales: acceptability in a surgical training environment

INTRODUCTION

Subjective workload measures are usually administered in a visual-manual format, either electronically or by paper and pencil. However, vocal responses to spoken queries may sometimes be preferable, for example when experimental manipulations require continuous manual responding or when participants have certain sensory/motor impairments. In the present study, we evaluated the acceptability of the hands-free administration of two subjective workload questionnaires - the NASA Task Load Index (NASA-TLX) and the Multiple Resources Questionnaire (MRQ) - in a surgical training environment where manual responding is often constrained.

METHOD

Sixty-four undergraduates performed fifteen 90-s trials of laparoscopic training tasks (five replications of 3 tasks - cannulation, ring transfer, and rope manipulation). Half of the participants provided workload ratings using a traditional paper-and-pencil version of the NASA-TLX and MRQ; the remainder used a vocal (hands-free) version of the questionnaires. A follow-up experiment extended the evaluation of the hands-free version to actual medical students in a Minimally Invasive Surgery (MIS) training facility.

RESULTS

The NASA-TLX was scored in 2 ways - (1) the traditional procedure using participant-specific weights to combine its 6 subscales, and (2) a simplified procedure - the NASA Raw Task Load Index (NASA-RTLX) - using the unweighted mean of the subscale scores. Comparison of the scores obtained from the hands-free and written administration conditions yielded coefficients of equivalence of r=0.85 (NASA-TLX) and r=0.81 (NASA-RTLX). Equivalence estimates for the individual subscales ranged from r=0.78 ("mental demand") to r=0.31 ("effort"). Both administration formats and scoring methods were equally sensitive to task and repetition effects. For the MRQ, the coefficient of equivalence for the hands-free and written versions was r=0.96 when tested on undergraduates. However, the sensitivity of the hands-free MRQ to task demands (η(partial)(2)=0.138) was substantially less than that for the written version (η(partial)(2)=0.252). This potential shortcoming of the hands-free MRQ did not seem to generalize to medical students who showed robust task effects when using the hands-free MRQ (η(partial)(2)=0.396). A detailed analysis of the MRQ subscales also revealed differences that may be attributable to a "spillover" effect in which participants' judgments about the demands of completing the questionnaires contaminated their judgments about the primary surgical training tasks.

CONCLUSION

Vocal versions of the NASA-TLX are acceptable alternatives to standard written formats when researchers wish to obtain global workload estimates. However, care should be used when interpreting the individual subscales if the object is to make comparisons between studies or conditions that use different administration modalities. For the MRQ, the vocal version was less sensitive to experimental manipulations than its written counterpart; however, when medical students rather than undergraduates used the vocal version, the instrument's sensitivity increased well beyond that obtained with any other combination of administration modality and instrument in this study. Thus, the vocal version of the MRQ may be an acceptable workload assessment technique for selected populations, and it may even be a suitable substitute for the NASA-TLX.

Objective assessment, selection, and certification in surgery

Historically, surgical competence has been evaluated subjectively. Fundamental changes in surgical technology and training have focused attention on the use of objective measurement of performance to improve patient safety and reduce errors. Surgical performance can be measured using a variety of tools, both in the clinical and simulated environments. Objective assessments can play a role in training by improving the evaluation and feedback. At the end of training or when a new skill is acquired, objective assessments may be used to ensure that a proficiency level has been reached and potentially as a condition for independent practice. When assessments are used for high-stakes evaluations like certification, they must be demonstrably reliable and valid. The definition of assessment, and the necessary components of a valid instrument, will be summarized. An overview of practical applications of objective assessment as it applies to training, selection, and certification of surgeons will be presented.

Transfer of training in the development of intracorporeal suturing skill in medical student novices: a prospective randomized trial

BACKGROUND

To help optimize the use of limited resources in trainee education, we developed a prospective randomized trial to determine the most effective means of teaching laparoscopic suturing to novices.

METHODS

Forty-one medical students received rudimentary instruction in intracorporeal suturing, then were pretested on a pig enterotomy model. They then were posttested after completion of 1 of 4 training arms: laparoscopic suturing, laparoscopic drills, open suturing, and virtual reality (VR) drills. Tests were scored for speed, accuracy, knot quality, and mental workload (National Aeronautics and Space Administration [NASA] Task Load Index).

RESULTS

Paired t tests were used. Task time was improved in all groups except the VR group. Knot quality improved only in the open or laparoscopic suturing groups. Mental workload improved only for those practicing on a physical laparoscopic trainer.

CONCLUSIONS

For novice trainees, the efficacy of VR training is questionable. In contrast, the other training methods had benefits in terms of time, quality, and perceived workload.

Elective laparoscopic recto-sigmoid resection for diverticular disease is suitable as a training operation

PURPOSE

Some authors state that elective laparoscopic recto-sigmoid resection is more difficult for diverticular disease as compared with malignancy. For this reason, starting laparoscopic surgeons might avoid diverticulitis, making the implementation phase unnecessary long. The aim of this study was to determine whether laparoscopic resection for diverticular disease should be included during the implementation phase.

METHODS

All consecutive patients who underwent an elective laparoscopic recto-sigmoid resection in our hospital for diverticulitis or cancer from 2003 to 2007 were analysed.

RESULTS

A total of 256 consecutive patients were included in this prospective cohort study. One hundred and fifty-one patients were operated on for diverticulitis and 105 for cancer. There was no significant difference in operation time (168 vs. 172 min), blood loss (189 vs. 208 ml), conversion rates (9.9% vs. 11.4%), hospital stay (8 vs. 8 days), total number of peroperative (2.3% vs. 1.6%) or postoperative complications (21.9% vs. 26.9%). The occurrence of anastomotic leakages was associated with higher American Society of Anesthesiologists (ASA) classification, which differed between the groups (86.8% vs. 64.8% ASA I-II, p < 0.001).

CONCLUSION

Since there are no differences in operation time, blood loss, conversion rate and total complications, there is no need to avoid laparoscopic recto-sigmoid resection for diverticular disease early in the learning curve.