The impact of external feedback on computer-assisted learning for surgical technical skill training

Video feedback combined with peer role-playing: a method to improve the teaching effect of medical undergraduates

OBJECTIVE

The purpose of this study was to investigate the effectiveness of implementation of video feedback combined with peer role-playing (PRP) teaching method in medical undergraduates adopting problem-based learning (PBL) teaching mode.

METHODS

The undergraduates of five-year clinical medicine who get enrollment of Wuhan local University from 2016 and 2018 were selected to be the research objects. The same grade level is randomly divided into several groups to carry out PBL, with 6-10 students in each group. Following the principle of voluntary participation, 34 students were enrolled in the study group and 33 students in the control group finally. The research regards group as the unit, and study report in group should be carried out to fulfill the research. In the study group, the students were asked to perform PRP report, and the report videos were used for feedback. At the same time, the control group reported by PPT, and the feedback was carried out according to the PPT. At the end of the study, the "Competency Improvement Satisfaction Questionnaire (CISQ)" was distributed to investigate students' satisfaction with this teaching method to improve their ability, Arizona Clinical Interview Score (ACIR) was administered in Chinese by a trained teacher unrelated using PRP method to assess students' clinical inquiry ability and communication skills, and theory test was performed to assess mastery of theoretical knowledge.

RESULTS

The results show that the study group is superior to the control group in improving the interest of learning and the ability of independent learning, interpersonal communication and active problem solving. Although it is in terms of the confidence in becoming a real doctor and the ability of teamwork, language expression, clinical thinking cultivated, active knowledge acquired and understood that study group are better than the control group, the difference was not statistically significant. ACIR shows that the study group is significantly better than the control group in organization, timeline planning, and transition statements, openly questioning, smooth progress, and avoiding repetition, summarizing, understandable language, documentation and total score. There is no significant difference in eye contact and no interruption. The differences between the two groups are not statistically significant in terms of responsing to concerns, positive feedback, and additional questions. The theoretical test scores of the study group are significantly higher than those of the control group.

CONCLUSION

Video feedback combined with peer role-playing teaching method implemented in medical undergraduates adopting PBL teaching mode is effective, it could stimulate interest in learning actively, improve interpersonal communication ability, improve learning efficiency and clinical knowledge and skills, and improve the confidence of becoming a real doctor. It is worthy of further research and promotion.

Application of mobile-based web app to enhance simple suturing skills of nurse practitioners

BACKGROUND

Suturing is a crucial clinical skill for nurse practitioners (NPs), but the effectiveness of traditional training methods (e.g., physical suture kits combined with video content) is low.

OBJECTIVE

This study compared the effectiveness and usability of a mobile-based web app (MoWa) developed for NPs to learn simple suturing skills with those of traditional instructional video-based training.

METHODS

The MoWa system utilizes mobile devices to simulate hands-on suturing and provides learning guidance and feedback to support self-learning with a physical suturing kit. Fifty-four suturing novices (NPs) were recruited as participants, divided into an experimental group (EG: 28 participants) and a control group (CG: 26 participants), and instructed to self-learn for 3 weeks. Learning effectiveness and system usability were evaluated through a pretest and posttest.

RESULTS

The EG exhibited significant improvements in learning outcomes, self-confidence, self-efficacy, and learning anxiety and expressed satisfaction with the MoWa system. Furthermore, the EG also considerably enhanced learning outcomes, self-efficacy, and learning anxiety compared to the CG, with no significant difference in self-confidence.

CONCLUSION

The MoWa system combined with deliberate practice is an effective strategy for supporting suturing skills training.

Effect of head-mounted displays on students' acquisition of surgical suturing techniques compared to an e-learning and tutor-led course: a randomized controlled trial

BACKGROUND

Although surgical suturing is one of the most important basic skills, many medical school graduates do not acquire sufficient knowledge of it due to its lack of integration into the curriculum or a shortage of tutors. E-learning approaches attempt to address this issue but still rely on the involvement of tutors. Furthermore, the learning experience and visual-spatial ability appear to play a critical role in surgical skill acquisition. Virtual reality head-mounted displays (HMDs) could address this, but the benefits of immersive and stereoscopic learning of surgical suturing techniques are still unclear.

MATERIAL AND METHODS

In this multi-arm randomized controlled trial, 150 novices participated. Three teaching modalities were compared: an e-learning course (monoscopic), an HMD-based course (stereoscopic, immersive), both self-directed and a tutor-led course with feedback. Suturing performance was recorded by video camera both before and after course participation (>26 h of video material) and assessed in a blinded fashion using the Objective Structured Assessment of Technical Skills (OSATS) Global Rating Score (GRS). Furthermore, the optical flow of the videos was determined using an algorithm. The number of sutures performed was counted, the visual-spatial ability was measured with the Mental Rotation Test (MRT), and courses were assessed with questionnaires.

RESULTS

Students' self-assessment in the HMD-based course was comparable to that of the tutor-led course and significantly better than in the e-learning course ( P =0.003). Course suitability was rated best for the tutor-led course ( x̄ =4.8), followed by the HMD-based ( x̄ =3.6) and e-learning ( x̄ =2.5) courses. The median ΔGRS between courses was comparable ( P =0.15) at 12.4 (95% CI 10.0-12.7) for the e-learning course, 14.1 (95% CI 13.0-15.0) for the HMD-based course, and 12.7 (95% CI 10.3-14.2) for the tutor-led course. However, the ΔGRS was significantly correlated with the number of sutures performed during the training session ( P =0.002), but not with visual-spatial ability ( P =0.615). Optical flow ( R2 =0.15, P <0.001) and the number of sutures performed ( R2 =0.73, P <0.001) can be used as additional measures to GRS.

CONCLUSION

The use of HMDs with stereoscopic and immersive video provides advantages in the learning experience and should be preferred over a traditional web application for e-learning. Contrary to expectations, feedback is not necessary for novices to achieve a sufficient level in suturing; only the number of surgical sutures performed during training is a good determinant of competence improvement. Nevertheless, feedback still enhances the learning experience. Therefore, automated assessment as an alternative feedback approach could further improve self-directed learning modalities. As a next step, the data from this study could be used to develop such automated AI-based assessments.

Learning surgical knot tying and suturing technique - effects of different forms of training in a controlled randomized trial with dental students

Objective

The acquisition of surgical skills requires motor learning. A special form of this is intermanual transfer by transferring motor skills from the nondominant hand (NDH) to the dominant hand (DH). The purpose of this study was to determine the learning gains that can be achieved for the DH by training with the DH, the NDH, and by non-surgical alternative training (AT).

Methods

124 preclinical (n=62) and clinical (n=62) dental students completed surgical knot tying and suturing technique training with the DH, with the NDH, and an AT in a controlled randomized trial.

Results

A statistically significant learning gain in knot tying and suture technique with the DH was evident only after training with the DH when compared to training with the NDH (p<0.001 and p=0.004, respectively) and an AT (p=0.001 and p=0.010, respectively). Of those students who achieved a learning gain ≥4 OSATS points, 46.4% (n=32) benefited in their knot tying technique with the DH from training with the DH, 29.0% (n=20) from training with the NDH, and 24.6% (n=17) from an AT while 45.7% (n=32) benefited in their suturing technique with the DH from training with the DH, 31.4% (n=22) from training with the NDH, and 22, 9% (n=16) from an AT.

Conclusions

Training with the DH enabled significantly better learning gains in the surgical knot tying and suturing techniques with the DH.

Learning of Intracorporal Knot Tying in Minimally Invasive Surgery by Video or Expert Instruction

INTRODUCTION

 Minimally invasive surgery skill laboratories are indispensable in training, especially for complex procedural skills such as intracorporal suturing and knot tying (ICKT). However, maintaining a laboratory is expensive, and specially trained teachers are in short supply. During the COVID-19 pandemic, in-person instruction has reduced to almost zero, while model learning via video instruction (VID) has become an integral part of medical education. The aim of this study was to compare the learning effectiveness and efficiency of ICKT by laparoscopically inexperienced medical students through video versus direct expert instruction.

MATERIALS AND METHODS

 A secondary analysis of two randomized controlled trials was performed. We drew data from students who were trained in ICKT with expert instruction (EXP, n = 30) and from students who were trained via VID, n = 30). A laparoscopic box trainer including laparoscope was used for ICKT. Objective Structured Assessment of Technical Skills (OSATS), knot quality, and total ICKT time were the parameters for the assessment in this study. Proficiency criteria were also defined for these parameters.

RESULTS

 Students in the EXP group performed significantly better in OSATS-procedure-specific checklist (PSC) and knot quality compared with students in the VID group, with no difference in task time. Of the students who reached the proficiency criteria for OSATS-PSC and knot quality, those in the EXP group required fewer attempts to do so than those in the VID group. Students in both groups improved significantly in all parameters over the first hour of evaluation.

CONCLUSION

 For the laparoscopically inexperienced, training in ICKT through expert instruction presents an advantage compared with video-based self-study in the form of faster understanding of the procedure and the associated consistent implementation of good knot quality. Both teaching methods significantly improved participants' ICKT skills.

The Effect of Anonymity on Quality of Resident Feedback

OBJECTIVE

Quality feedback is critical to facilitate better performance and quicker learning. However, faculty may be hesitant to provide quality constructive feedback, especially in written form, due to fear of retaliation. We evaluated the impact of faculty anonymity on the quality of faculty-to-resident feedback.

DESIGN

A retrospective review was undertaken of faculty evaluation of resident performance from 2017 to 2018, when evaluations were identifiable, compared to 2018-2019, when evaluations were anonymous. Evaluations included 27 individual items with Likert type scoring and 2 open-ended questions. Open-ended responses and overall performance were de-identified and scored by 2 reviewers independently using the task, performance gap, action scoring model. Comparisons between groups were performed with the Wilcoxon-Mann-Whitney test.

SETTING

Tertiary Care Institution, University of Michigan, Ann Arbor, MI PARTICIPANTS: 415 resident performance evaluations were available for analysis with 251 in the identifiable group and 164 in the anonymous group.

RESULTS

The average composite score for the identifiable group was 105.2 and 103.4 in the anonymous group (p = 0.22). The effect size of the impact on composite score was small (Cohen's d 0.084, 95% CI -0.11-0.28). There was excellent inter-rater reliability. There were no differences between feedback groups for any of the 3 components of task, performance gap, and action model. While average scores for all 3 components were low, action scores were lowest.

CONCLUSIONS

Anonymity did not significantly impact faculty evaluations of resident performance. The quality of open-ended feedback on written evaluations was generally poor, especially in identifying actions for continued performance improvement. Additional mechanisms to improve feedback quality should be sought.

Performance measures for the SACRED team-centered approach to advanced gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative

The European Society of Gastrointestinal Endoscopy and United European Gastroenterology have defined performance measures for upper and lower gastrointestinal, pancreaticobiliary, and small-bowel endoscopy. Quality indicators to guide endoscopists in the growing field of advanced endoscopy are also underway. We propose that equal attention is given to developing the entire advanced endoscopy team and not the individual endoscopist alone.We suggest that the practice of teams intending to deliver high quality advanced endoscopy is underpinned by six crucial principles concerning: selection, acceptance, complications, reconnaissance, envelopment, and documentation (SACRED).

Learning in the Operating Theatre: A Thematic Analysis of Opportunities Lost and Found

OBJECTIVE

The operating theatre (OT) is an important learning environment. Trainees face barriers to learning in the OT that may reduce meaningful educational interactions. The impact of these barriers on the intraoperative learning experience of trainees and the strategies that they employ to overcome them are not known. This qualitative study aimed to describe the intraoperative learning experiences of senior general surgery trainees in Australia and their strategies to optimize learning in the OT.

DESIGN, SETTING, PARTICIPANTS

The authors developed a semi-structured interview guide based on published literature. Purposive sampling was used to identify a representative group of general surgery trainees in Australia, who were interviewed in a private setting with audio recordings deidentified for verbatim transcription and analysis. Thematic analysis was conducted using an interpretivist approach to produce a coding framework.

RESULTS

Ten trainees participated in the study. Themes were divided into external and internal barriers to learning, promoters of effective learning and actions to facilitate learning. External barriers included cultural neglect of an important issue, with inadequate prioritization of teaching and a lack of structure for intraoperative learning. From this, we identified the theme of missed opportunities. Internal barriers included difficulties in developing assertiveness required to address these issues and a failure to adequately plan for learning, with reliance on the mentor to initiate. Actions to facilitate learning were rarely employed by trainees, as most were unaware of strategies to maximize intraoperative learning.

CONCLUSIONS

Trainees find the barriers to learning in the OT difficult to address and are not well acquainted with strategies that may allow them to maximize their learning.

Maximizing Learning in the Operating Room: Residents' Perspectives

BACKGROUND

Few studies examine how residents can optimize their educational experience in the OR on their terms. This study aimed to examine residents' perceptions of how learners can maximize their education in the OR.

METHOD

Using constructivist grounded theory methodology, the authors conducted focus groups with general surgery residents, PGY1-5, followed by semi-structured interviews with attending surgeons from a single, academic medical center. Constant comparison was used to identify themes and explore their relationships. Theoretical sampling was used until saturation was achieved.

RESULTS

Residents and attendings participated. Two phases of OR learning were identified, intra-operative and inter-operative. Characters that made optimized learning included control, struggling, and reflection. Residents who practiced self-reflection with their experiences, and were able to articulate this awareness to attendings, felt the OR was an ideal learning environment. Attendings echoed similar findings.

CONCLUSIONS

Providing residents with a method of maximizing OR learning is critical to postgraduate clinical education. Currently, observation passively morphs into active learning and eventually independent operating in the OR. However, residents who practice self-regulated learning, and are able to discuss their educational goals with attendings, seem to find the OR a better learning environment and progress to independence more quickly. This was echoed by practicing attendings. Providing residents with a generalizable, self-regulated learning framework specific to operative educational experiences could maximize learning potential and expedite resident progression in the OR.

Computer-based video training is effective in teaching basic surgical skills to novices without faculty involvement using a self-directed, sequential and incremental program

Introduction

Computer-based video training (CBVT) of surgical skills overcomes limitations of 1:1 instruction. We hypothesized that a self-directed CBVT program could teach novices by dividing basic surgical skills into sequential, easily-mastered steps.

Methods

We developed a 12 video program teaching basic knot tying and suturing skills introduced in discrete, incremental steps. Students were evaluated pre- and post-course with a self-assessment, a written exam and a skill assessment.

Results

Students (n = 221) who completed the course demonstrated significant improvement. Their average pre-course product quality score and assessment of technique using standard Global Rating Scale (GRS) were <0.4 for 6 measured skills (scale 0–5) and increased post-course to ≥3.25 except for the skill tying on tension whose GRS = 2.51. Average speed increased for all skills. Students’ self-ratings (scale 1–5) increased from an average of 1.4 ± 0.7 pre-elective to 3.9 ± 0.9 post-elective across all skills (P < 0.01).

Conclusion

Self-directed, incremental and sequential video training is effective teaching basic surgical skills and may be a model to teach other skills or to play a larger role in remote learning.

•

Video based training can teach medical student basic knot tying and suturing skills.

•

This training is effective despite no faculty involvement.

•

Students showed marked improvement in technique, speed and the end product.

•

Remote, computer-based learning may be able to play a larger role in medical education.

Video-based coaching in surgical education: a systematic review and meta-analysis

BACKGROUND

In the era of competency-based surgical education, VBC has gained increased attention and may enhance the efficacy of surgical education. The objective of this systematic review was to summarize the existing evidence of video-based coaching (VBC) and compare VBC to traditional master-apprentice-based surgical education.

METHODS

We performed a systematic review and meta-analysis of randomized controlled trials (RCT) assessing VBC according to the PRISMA and Cochrane guidelines. The MEDLINE, EMBASE, and COCHRANE and Researchgate databases were searched for eligible manuscripts. Standard mean difference (SMD) of performance scoring scales was used to assess the effect of VBC versus traditional training without VBC (control).

RESULTS

Of 627 studies identified, 24 RCTs were eligible and evaluated. The studies included 778 surgical trainees (n = 386 VBC vs. n = 392 control). 13 performance scoring scales were used to assess technical competence; OSATS-GRS was the most common (n = 15). VBC was provided preoperative (n = 11), intraoperative (n = 1), postoperative (n = 10), and perioperative (n = 2). The majority of studies were unstructured, where identified coaching frameworks were PRACTICE (n = 1), GROW (n = 2) and Wisconsin Coaching Framework (n = 1). There was an effect on performance scoring scales in favor of VBC coaching (SMD 0.87, p < 0.001). In subgroup analyses, the residents had a larger relative effect (SMD 1.13; 0.61-1.65, p < 0.001) of VBC compared to medical students (SMD 0.43, 0.06-0.81, p < 0.001). The greatest source of potential bias was absence of blinding of the participants and personnel (n = 20).

CONCLUSION

Video-based coaching increases technical performance of medical students and surgical residents. There exist significant study and intervention heterogeneity that warrants further exploration, showing the need to structure and standardize video-based coaching tools.

To the Point: The expanding role of simulation in obstetrics and gynecology medical student education

This article, from the "To the Point" series prepared by the Association of Professors of Gynecology and Obstetrics (APGO) Undergraduate Medical Education Committee (UMEC), provides educators with an overview of the use of simulation in undergraduate medical education in the field of obstetrics and gynecology. Simulation plays an important role in the education of medical students. Students are increasingly serving as clinical observers and providing less direct patient care. Simulation can help standardize education and ensure quality and comparability across an enlarging educational environment. This article summarizes the expanding role of simulation in undergraduate medical education in obstetrics and gynecology and its effect on important learner outcomes such as confidence, knowledge, skills, workplace behaviors, and translation to patient care.

The Application of Observational Practice and Educational Networking in Simulation-Based and Distributed Medical Education Contexts

INTRODUCTION

Research has revealed that individuals can improve technical skill performance by viewing demonstrations modeled by either expert or novice performers. These findings support the development of video-based observational practice communities that augment simulation-based skill education and connect geographically distributed learners. This study explores the experimental replicability of the observational learning effect when demonstrations are sampled from a community of distributed learners and serves as a context for understanding learner experiences within this type of training protocol.

METHODS

Participants from 3 distributed medical campuses engaged in a simulation-based learning study of the elliptical excision in which they completed a video-recorded performance before being assigned to 1 of 3 groups for a 2-week observational practice intervention. One group observed expert demonstrations, another observed novice demonstrations, and the third observed a combination of both. Participants returned for posttesting immediately and 1 month after the intervention. Participants also engaged in interviews regarding their perceptions of the usability and relevance of video-based observational practice to clinical education.

RESULTS

Checklist (P < 0.0001) and global rating (P < 0.0001) measures indicate that participants, regardless of group assignment, improved after the intervention and after a 1-month retention period. Analyses revealed no significant differences between groups. Qualitative analyses indicate that participants perceived the observational practice platform to be usable, relevant, and potentially improved with enhanced feedback delivery.

CONCLUSIONS

Video-based observational practice involving expert and/or novice demonstrations enhances simulation-based skill learning in a group of geographically distributed trainees. These findings support the use of Internet-mediated observational learning communities in distributed and simulation-based medical education contexts.

Impact of Video Technology for Improving Success of Medial Canthus Episcleral Anesthesia in Ophthalmology

BACKGROUND AND OBJECTIVES

Efficient learning of regional anesthesia in ophthalmology remains challenging because trainees are afforded limited opportunity to practice ocular anesthesia. The aim of this prospective, randomized, blinded study was to determine whether teaching with video improves regional anesthesia skills of residents in ophthalmology.

METHODS

From January to October 2016, 32 novice anesthesiology residents were evaluated while performing medial canthus episcleral procedures during a 5-day rotation. Residents were randomly assigned to either receive or not receive a video review of their performance at day 3. The primary outcome was a comparison of akinesia using a 12-point scale before incision assessed by the blinded surgeon.

RESULTS

A total of 288 blocks were performed by 32 residents and were assessed by 3 surgeons before the intervention (144 blocks) and after the intervention (144 blocks). Residents in the review group improved to a greater degree compared with residents in the no-review group. The median overall akinesia scores for the review and no-review groups were similarly low (6; interquartile range [IQR], 2-11; and 6 [IQR, 2-9], respectively) on day 1 of the rotation, whereas anesthesia performed by residents in the video group provided a better akinesia score (12 [IQR, 10-12] vs 8 [IQR, 6-10]; P < 0.001) on day 5 of the rotation.

CONCLUSIONS

Video-assisted teaching significantly improves performance of medial canthus episcleral anesthesia performed by novice trainees.

Self-Directed Interactive Video-Based Instruction Versus Instructor-Led Teaching for Myanmar House Surgeons: A Randomized, Noninferiority Trial

OBJECTIVE

To compare self-directed interactive video-based instruction (IVBI) with instructor-led teaching in the acquisition of basic surgical skills by House Surgeons at University of Medicine 1, Yangon.

DESIGN

A prospective, 1:1 randomized controlled trial was conducted. Participants were randomized into 2 teaching arms: (1) self-directed IVBI or (2) instructor-led teaching. Self-directed IVBI participants were provided with a portable DVD player that could play, fast forward, rewind, and skip through skills modules. Participants in the instructor-led teaching group were taught in small groups by standardized instructors. Pretesting and posttesting of 1-handed knot tie, 2-handed knot tie, vertical mattress suture, and instrument tie was performed using the Objective Structured Assessment of Technical Skills (OSATS). Students randomized to self-directed IVBI completed an exit survey to assess satisfaction. Demographic data were collected of all participants.

SETTING

University of Medicine 1, Yangon, Myanmar.

PARTICIPANTS

Fifty participants were randomly selected from 78 eligible House Surgeons who were enrolled in their basic surgery rotation.

RESULTS

Demographic characteristics and baseline skills were comparable in participants randomized to IVBI and instructor-led teaching. Mean OSATS score increased from pretest to posttest in both groups (p < 0.001). The mean posttest OSATS score of the IVBI group was 0.72 points below that of the instructor-led teaching group (90% CI: -3.8 to 5.2), with the 90% CI falling below the a priori noninferiority margin, satisfying criteria for noninferiority. More than 90% of students marked either "agree" or "strongly agree" to the following statements on the exit survey: further expansion of IVBI into other skills modules and integration of IVBI into training curriculum.

CONCLUSION

IVBI is noninferior to instructor-led teaching of surgical skills based on OSATS scores. House Surgeons highly rated self-directed IVBI. Self-directed IVBI has the potential to significantly reduce the personnel required for skills teaching and may serve as a long-term learning adjunct in low-resource settings.

Self-assessment in laparoscopic surgical skills training: Is it reliable?

BACKGROUND

The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education.

METHODS

Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed.

RESULTS

A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s > 0.5; p < 0.001) between the expert- and self-assessment in all categories with only the 'tissue handling' having a weaker correlation (r s = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03).

CONCLUSION

Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Justify Your Answer: The Role of Written Think Aloud in Script Concordance Testing

Construct: Clinical reasoning assessment is a growing area of interest in the medical education literature. Script concordance testing (SCT) evaluates clinical reasoning in conditions of uncertainty and has emerged as an innovative tool in the domain of clinical reasoning assessment. SCT quantifies the degree of concordance between a learner and an experienced clinician and attempts to capture the breadth of responses of expert clinicians, acknowledging the significant yet acceptable variation in practice under situations of uncertainty.

BACKGROUND

SCT has been shown to be a valid and reliable clinical reasoning assessment tool. However, as SCT provides only quantitative information, it may not provide a complete assessment of clinical reasoning.

APPROACH

Think aloud (TA) is a qualitative research tool used in clinical reasoning assessment in which learners verbalize their thought process around an assigned task. This study explores the use of TA, in the form of written reflection, in SCT to assess resident clinical reasoning, hypothesizing that the information obtained from the written TA would enrich the quantitative data obtained through SCT. Ninety-one pediatric postgraduate trainees and 21 pediatricians from 4 Canadian training centers completed an online test consisting of 24 SCT cases immediately followed by retrospective written TA. Six of 24 cases were selected to gather TA data. These cases were chosen to allow all phases of clinical decision making (diagnosis, investigation, and treatment) to be represented in the TA data. Inductive thematic analysis was employed when systematically reviewing TA responses.

RESULTS

Three main benefits of adding written TA to SCT were identified: (a) uncovering instances of incorrect clinical reasoning despite a correct SCT response, (b) revealing sound clinical reasoning in the context of a suboptimal SCT response, and (c) detecting question misinterpretation.

CONCLUSIONS

Written TA can optimize SCT by demonstrating when correct examinee responses are based on guessing or uncertainty rather than robust clinical rationale. TA can also enhance SCT by allowing examinees to provide justification for responses that otherwise would have been considered incorrect and by identifying questions that are frequently misinterpreted to avoid including them in future examinations. TA also has significant value in differentiating between acceptable variations in expert clinician responses and deviance associated with faulty rationale or question misinterpretation; this could improve SCT reliability. A written TA protocol appears to be a valuable tool to assess trainees' clinical reasoning and can strengthen the quantitative assessment provided by SCT.

An Innovative, No-cost, Evidence-Based Smartphone Platform for Resident Evaluation

PURPOSE

Timely performance evaluation and feedback are critical to resident development. However, formulating and delivering this information disrupts physician workflow, leading to low participation. This study was designed to determine if a locally developed smartphone platform would integrate regular evaluation into daily processes and thus increase faculty participation in timely resident evaluation.

METHODS

Formal, documented resident operative and patient interaction evaluations were compiled over an 8-month study period. The study was divided into two 4-month phases. No changes to the existing evaluation methods were made during Phase 1. Phase 2 began after a washout period of 2 weeks and coincided with the launch of a smartphone-based platform. The platform uses a combination of Likert scale questions and the Dreyfus model of skill acquisition to describe competence levels in technical and nontechnical skills. The instrument inflicts minimal effect on surgeon workflow, with the aim of integrating resident evaluation into daily processes. The number of different faculty members performing evaluations, resident level (postgraduate year), type of interaction or procedure, and competency data were compiled. All evaluations were tracked by the program director as they were automatically uploaded into a database. Faculty members were introduced to the new platform at the beginning of Phase 2, and previous methods of evaluation continued to be encouraged and were considered valid throughout both phases of the study. Data were analyzed using Fisher exact test for specific PGY level, and chi-square test was used for overall program analysis. Statistical significance was set at p < 0.05.

RESULTS

Total faculty engagement, that is, number of faculty members completing evaluations, increased from 13% (5/38) in Phase 1 to 53% (20/38) in Phase 2. During Phase 1, all evaluations consisted of online forms through the department's established system or e-mails to the program director. Evaluations were completed in 0.9% (15/1599) of cases residents completed in Phase 1 versus 12% (217/1812) of those in Phase 2. During Phase 2, evaluations were conducted exclusively using the new platform. This was done based on participant's choice. Total numbers of residents and core faculty members did not change between Phases 1 and 2.

CONCLUSIONS

A smartphone-based platform can be created with existing technology at no cost. It is adaptable and can be updated in real-time and can employ validated scales to build an evaluation portfolio for learners assessing technical and nontechnical skills. Furthermore, and perhaps most importantly, it can be designed to integrate into existing workflow patterns to increase faculty participation.

Filling the gaps between theory and daily clinical procedural skills training in family medicine

Performance of procedures is an integral part of any family physician/general practitioner's practice. Unfortunately, discrepancy occurs between the existing theoretical methods of procedural teaching and the training imparted during real daily practice, which creates gaps that need to be overcome. This article identifies and reviews teaching gaps in family medicine training and presents suggestions to overcome them with a view to forming holistic psychomotor skills based on the learner's characteristics within the patient-centred philosophy of family medicine.

Video-based on-ward supervision for final year medical students

BACKGROUND

Constructive feedback is an essential element of the educational process, helping trainees reach their maximum potential and increasing their skill level. Video-based feedback has been described as highly effective in various educational contexts. The present study aimed to evaluate the feasibility and acceptability of video-based, on-ward supervision for final year students in a clinical context with real patients.

METHODS

Nine final year medical students (three male, six female; aged 25.1 ± 0.7 years) and eight patients (five male, three female; aged 59.3 ± 16.8 years) participated in the pilot study. Final year students performed routine medical procedures at bedside on internal medicine wards at the University of Heidelberg Medical Hospital. Students were filmed and were under supervision. After performing the procedures, an oral feedback loop was established including student, patient and supervisor feedback on communicative and procedural aspects of skills performed. Finally, students watched their video, focusing on specific teachable moments mentioned by the supervisor. Written evaluations and semi-structured interviews were conducted that focused on the benefits of video-based, on-ward supervision. Interviews were analysed qualitatively, using open coding to establish recurring themes and overarching categories to describe patients' and students' impressions. Descriptive, quantitative analysis was used for questionnaire data.

RESULTS

Supervised, self-chosen skills included history taking (n = 6), physical examination (n = 1), IV cannulation (n = 1), and ECG recording (n = 1). The video-based, on-ward supervision was well accepted by patients and students. Supervisor feedback was rated as highly beneficial, with the video material providing an additional opportunity to focus on crucial aspects and to further validate the supervisor's feedback. Students felt the video material would be less beneficial without the supervisor's feedback. The setting was rated as realistic, with filming not influencing behaviour.

CONCLUSION

Video-based, on-ward supervision may be a powerful tool for improving clinical medical education. However, it should be regarded as an additional tool in combination with supervisors' oral feedback. Acceptance was high in both students and patients. Further research should address possibilities of efficiently combining and routinely establishing these forms of feedback in medical education.

Training of deep microsurgical skill: Establishment of a high-volume intracranial carotid bypass model

OBJECTIVE

To establish a model for a high-volume intracranial carotid bypass operation.

METHODS

High-volume internal carotid-middle cerebral artery bypass was performed on 9 cadaver heads with arteries of porcine forearms as grafts by 6 residents with no previous experience in vascular anastomosis on cadavers. The intima was dissected immediately after the anastomoses were completed to observe the patency of anastomosis.

RESULTS

After different duration periods of training using this model, 36 vascular anastomoses on 18 sides were successfully performed by the 6 residents with a self-made difficulty regulation device. As the difficulty level increased, the time needed for anastomosis lengthened and patency rate showed a decreasing trend. As the amount of training increased, the residents were able to decrease the amount of time to complete the operation with increasing patency rates.

CONCLUSIONS

The model of high-volume internal carotid-middle cerebral artery bypass with arteries of porcine forearms has the advantages of material similarity, easy access of grafts, better simulation of intraoperative conditions, and adjustable difficulties. Our results suggest that this new procedure has a better simulation-training platform which is closer to the real surgical procedure for surgeons willing to master the technique of a high-volume bypass operation.

Randomized trial to assess the effect of supervised and unsupervised video feedback on teaching practical skills

BACKGROUND

Feedback is a vital component of the learning process; however, great variation exists in the quality, quantity, and method of delivery. Video feedback is not commonly used in the teaching of surgical skills. The aim of this trial was to evaluate the benefit of 2 types of video feedback-individualized video feedback (IVF), with the student reviewing their performance with an expert tutor, and unsupervised video-enhanced feedback (UVF), where the student reviews their own performance together with an expert teaching video-to determine if these improve performance when compared with a standard lecture feedback.

METHODS

A prospective blinded randomized control trial comparing lecture feedback with IVF and UVF was carried out. Students were scored by 2 experts directly observing the performance and 2 blinded experts using a validated pro forma. Participants were recorded on video when performing a suturing task. They then received their feedback via any of the 3 methods before being invited to repeat the task.

RESULTS

A total of 32 students were recruited between the 3 groups. There was no significant difference in suturing skill performance scores given by those directly observing the students and those blinded to the participant. There was no statistically significant difference between the 2 video feedback groups (p = 1.000), but there was significant improvement between standard lecture feedback and UVF (p = 0.047) and IVF (p = 0.001).

CONCLUSION

Video feedback can facilitate greater learning of clinical skills. Students can attain a similar level of surgical skills improvement with UVF as with teacher-intensive IVF.

Mentor-guided self-directed learning affects resident practice

OBJECTIVE

Self-directed learning (SDL) can be as effective as instructor-led training. It employs less instructional resources and is potentially a more efficient educational approach. Although SDL is encouraged among residents in our surgical training program via 24-hour access to surgical task trainers and online modules, residents report that they seldom practice. We hypothesized that a mentor-guided SDL approach would improve practice habits among our residents.

DESIGN

From 2011 to 2013, 12 postgraduate year (PGY)-2 general surgery residents participated in a 6-week minimally invasive surgery (MIS) rotation. At the start of the rotation, residents were asked to practice laparoscopic skills until they reached peak performance in at least 3 consecutive attempts at a task (individual proficiency).

SETTING

Trainees met with the staff surgeon at weeks 3 and 6 to evaluate progress and review a graph of their individual learning curve. All trainees subsequently completed a survey addressing their practice habits and suggestions for improvement of the curriculum.

RESULTS

By the end of the rotation, 100% of participants improved in all practiced tasks (p < 0.05), and each reported that they practiced more in this rotation than during rotations without mentor-guided SDL. Additionally, 6 (50%) residents reported that their skill level had improved relative to their peers. Some residents (n = 3) felt that the curriculum could be improved by including task-specific goals and additional practice sessions with the staff surgeon.

CONCLUSIONS

Mentor-guided SDL stimulated surgical residents to practice with greater frequency. This repeated deliberate practice led to significantly improved MIS skills without significantly increasing the need for faculty-led instruction. Some residents preferred more discrete goal setting and increased mentor guidance.

The impact of feedback of intraoperative technical performance in surgery: a systematic review

OBJECTIVES

Increasing patient demands, costs and emphasis on safety, coupled with reductions in the length of time surgical trainees spend in the operating theatre, necessitate means to improve the efficiency of surgical training. In this respect, feedback based on intraoperative surgical performance may be beneficial. Our aim was to systematically review the impact of intraoperative feedback based on surgical performance.

SETTING

MEDLINE, Embase, PsycINFO, AMED and the Cochrane Database of Systematic Reviews were searched. Two reviewers independently reviewed citations using predetermined inclusion and exclusion criteria. 32 data-points per study were extracted.

PARTICIPANTS

The search strategy yielded 1531 citations. Three studies were eligible, which comprised a total of 280 procedures by 62 surgeons.

RESULTS

Overall, feedback based on intraoperative surgical performance was found to be a powerful method for improving performance. In cholecystectomy, feedback led to a reduction in procedure time (p=0.022) and an improvement in economy of movement (p<0.001). In simulated laparoscopic colectomy, feedback led to improvements in instrument path length (p=0.001) and instrument smoothness (p=0.045). Feedback also reduced error scores in cholecystectomy (p=0.003), simulated laparoscopic colectomy (p<0.001) and simulated renal artery angioplasty (p=0.004). In addition, feedback improved balloon placement accuracy (p=0.041), and resulted in a smoother learning curve and earlier plateau in performance in simulated renal artery angioplasty.

CONCLUSIONS

Intraoperative feedback appears to be associated with an improvement in performance, however, there is a paucity of research in this area. Further work is needed in order to establish the long-term benefits of feedback and the optimum means and circumstances of feedback delivery.

Feasibility of real-time workflow segmentation for tracked needle interventions

Computer-assisted training systems promote both training efficacy and patient health. An important component for providing automatic feedback in computer-assisted training systems is workflow segmentation: the determination of what task in the workflow is being performed. Our objective was to develop a workflow segmentation algorithm for needle interventions using needle tracking data. Needle tracking data were collected from ultrasound-guided epidural injections and lumbar punctures, performed by medical personnel. The workflow segmentation algorithm was tested in a simulated real-time scenario: the algorithm was only allowed access to data recorded at, or prior to, the time being segmented. Segmentation output was compared to the ground-truth segmentations produced by independent blinded observers. Overall, the algorithm was 93% accurate. It automatically segmented the ultrasound-guided epidural procedures with 81% accuracy and the lumbar punctures with 82% accuracy. Given that the manual segmentation consistency was only 84%, the algorithm's accuracy was 93%. Using Cohen's d statistic, a medium effect size (0.5) was calculated. Because the algorithm segments needle-based procedures with such high accuracy, expert observers can be augmented by this algorithm without a large decrease in ability to follow trainees in a workflow. The proposed algorithm is feasible for use in a computer-assisted needle placement training system.

Self-regulated learning in simulation-based training: a systematic review and meta-analysis

CONTEXT

Self-regulated learning (SRL) requires an active learner who has developed a set of processes for managing the achievement of learning goals. Simulation-based training is one context in which trainees can safely practise learning how to learn.

OBJECTIVES

The purpose of the present study was to evaluate, in the simulation-based training context, the effectiveness of interventions designed to support trainees in SRL activities. We used the social-cognitive model of SRL to guide a systematic review and meta-analysis exploring the links between instructor supervision, supports or scaffolds for SRL, and educational outcomes.

METHODS

We searched databases including MEDLINE and Scopus, and previous reviews, for material published until December 2011. Studies comparing simulation-based SRL interventions with another intervention for teaching health professionals were included. Reviewers worked independently and in duplicate to extract information on learners, study quality and educational outcomes. We used random-effects meta-analysis to compare the effects of supervision (instructor present or absent) and SRL educational supports (e.g. goal-setting study guides present or absent).

RESULTS

From 11,064 articles, we included 32 studies enrolling 2482 trainees. Only eight of the 32 studies included educational supports for SRL. Compared with instructor-supervised interventions, unsupervised interventions were associated with poorer immediate post-test outcomes (pooled effect size: -0.34, p = 0.09; n = 19 studies) and negligible effects on delayed (i.e. > 1 week) retention tests (pooled effect size: 0.11, p = 0.63; n = 8 studies). Interventions including SRL supports were associated with small benefits compared with interventions without supports on both immediate post-tests (pooled effect size: 0.23, p = 0.22; n = 5 studies) and delayed retention tests (pooled effect size: 0.44, p = 0.067; n = 3 studies).

CONCLUSIONS

Few studies in the simulation literature have designed SRL training to explicitly support trainees' capacity to self-regulate their learning. We recommend that educators and researchers shift from thinking about SRL as learning alone to thinking of SRL as comprising a shared responsibility between the trainee and the instructional designer (i.e. learning using designed supports that help prepare individuals for future learning).

A Comprehensive Methodology for Examining the Impact of Surgical Team Briefings and Debriefings on Teamwork

The adoptions of briefing and debriefing protocols have evolved from the Joint Commission’s initiative to improve communication and safety in the operating room. Briefing normally occurs prior to incision and is used to discuss and confirm critical information, while debriefing occurs during or after surgery. Debriefing provides a unique opportunity for individuals and teams to immediately reflect on their performance, allowing them to more easily identify errors and develop plans to improve their next performance. Studies have shown that using briefings and debriefings improve communication and teamwork. However, there is still much to learn about the value of both for surgical teams. This paper presents a robust methodology for examining and measuring the impacts of surgical team briefings and debriefings on teamwork. The methodology includes (1) audio/video recording the surgical care process, (2) prospective observations using a validated electronic data collection tool, (3) pre- and post-surgery surveys, and (4) individual surgical team member interviews. The current paper describes the methodology to obtain a robust and comprehensive data set for analyzing the impacts of briefing and debriefing on teamwork; the results of the surgeries recorded using this methodology will be presented in subsequent papers.

A computerised third molar surgery simulator--results of supervision by different professionals

The purpose of the study was to investigate which supervisory approach afforded the most efficient learning method for undergraduate students in oral and maxillofacial surgery (OMS) using a computerised third molar surgery simulator. Fifth year dental students participated voluntarily in a randomised experimental study using the simulator. The amount of time required and the number of trials used by each student were evaluated as a measure of skills development. Students had the opportunity to practise the procedure until no further visible improvements were achieved. The study assessed four different types of supervision to guide the students. The first group was where they were supported by a teacher/specialist in OMS, the second by a teaching assistant, the third group practised without any supervision and the fourth received help from a simulator technician/engineer. A protocol describing assessment criteria was designed for this purpose, and a questionnaire was completed by all participating students after the study. The average number of attempts required to virtually remove a third molar tooth in the simulator was 1.44 times for the group supervised by an OMS teacher; 1.5 times for those supervised by a teaching assistant; 2.8 times for those who had no supervision; and 3.6 times when support was provided only by a simulator technician. The results showed that the most efficient experience of the students was when they were helped by an OMS teacher or a teaching assistant. In a time and cost-effective perspective, supervision by a teaching assistant for a third molar surgery simulator would be the optimal choice.

Operation debrief: a SHARP improvement in performance feedback in the operating room

OBJECTIVES

To explore the current status of performance feedback (debriefing) in the operating room and to develop and evaluate an evidence-based, user-informed intervention termed "SHARP" to improve debriefing in surgery.

BACKGROUND

Effective debriefing is a key educational technique for optimizing learning in surgical settings. However, there is a lack of a debriefing culture within surgery. Few studies have prospectively evaluated educational interventions to improve the quality and quantity of performance feedback in surgery.

METHODS

This was a prospective pre- and post-study of 100 cases involving 22 trainers (attendings) and 30 surgical residents (postgraduate years 3-8). A trained researcher assessed the quality of debriefings provided to the trainee using the validated Objective Structured Assessment of Debriefing (OSAD) tool alongside ethnographic observation. Following the first 50 cases, an educational intervention termed "SHARP" was introduced and measures repeated for a further 50 cases. User satisfaction with SHARP was assessed via questionnaire. Twenty percent of the cases were observed independently by a second researcher to test interrater reliability.

RESULTS

Interrater reliability for OSAD was excellent (ICC = 0.994). Objective scores of debriefing (OSAD) improved significantly after the SHARP intervention: median pre = 19 (range, 8-31); median post = 33 (range, 26-40), P < 0.001. Strong correlations between observer (OSAD) and trainee rating of debriefing were obtained (median ρ = 0.566, P < 0.01). Ethnographic observations also supported a significant improvement in both quality and style of debriefings. Users reported high levels of satisfaction in terms of usefulness, feasibility, and comprehensiveness of the SHARP tool.

CONCLUSIONS

SHARP is an effective and efficient means of improving performance feedback in the operating room. Its routine use should be promoted to optimize workplace-based learning and foster a positive culture of debriefing and performance improvement within surgery.

Randomized trial of the effect of video feedback on the acquisition of surgical skills

Background

Constructive feedback provides a mechanism for reinforcing learning during the acquisition of surgical skills. Feedback is usually given verbally, and sometimes documented, after direct observation by a trained assessor. The aim was to evaluate video recording as an effective modality for enhancing feedback, in comparison with standard verbal feedback alone.

Methods

This was a prospective, blinded, randomized clinical trial comparing standard verbal feedback plus video with standard verbal feedback alone. Validated pro formas for assessment were used and quality control was performed by independent expert assessors. Trial participants were recorded on video performing the surgical skill, and returned the next day to perform the skill again following video and standard verbal feedback (group 1) or standard verbal feedback alone (group 2).

Results

Forty-eight participants were divided equally between the two groups. There was a significant improvement in the mean overall procedure score for group 1 of 2·875 from a maximum achievable score of 20 (P = 0·003), but not for group 2. There were significant improvements in the specific domains of instrument familiarity, needle handling, skin handling and accurate apposition, again all in group 1. The only significant improvement in group 2 was in an organized approach to the task, also observed in group 1. Knot-tying security deteriorated after feedback in group 2 but not in group 1.

Conclusion

The addition of video feedback can improve the acquisition of surgical skills, and could be incorporated into formal surgical curricula.

Computer-assisted teaching of skin flap surgery: validation of a mobile platform software for medical students

The purpose of this study was to develop and validate a multimedia software application for mobile platforms to assist in the teaching and learning process of design and construction of a skin flap. Traditional training in surgery is based on learning by doing. Initially, the use of cadavers and animal models appeared to be a valid alternative for training. However, many conflicts with these training models prompted progression to synthetic and virtual reality models. Fifty volunteer fifth- and sixth-year medical students completed a pretest and were randomly allocated into two groups of 25 students each. The control group was exposed for 5 minutes to a standard text-based print article, while the test group used multimedia software describing how to fashion a rhomboid flap. Each group then performed a cutaneous flap on a training bench model while being evaluated by three blinded BSPS (Brazilian Society of Plastic Surgery) board-certified surgeons using the OSATS (Objective Structured Assessment of Technical Skill) protocol and answered a post-test. The text-based group was then tested again using the software. The computer-assisted learning (CAL) group had superior performance as confirmed by checklist scores (p<0.002), overall global assessment (p = 0.017) and post-test results (p<0.001). All participants ranked the multimedia method as the best study tool. CAL learners exhibited better subjective and objective performance when fashioning rhomboid flaps as compared to those taught with standard print material. These findings indicate that students preferred to learn using the multimedia method.

Three-step model course to teach intracorporeal laparoscopic suturing

BACKGROUND

Laparoscopy requires a set of skills such as intracorporeal stitching and knotting. The aim of this study is to present an effective specialized training course for the laparoscopic suturing technique.

MATERIALS AND METHODS

We designed a specialized 5-day training course for laparoscopic suturing skills with theoretical and practical sessions on inanimate pelvic training. The "gladiator rule" was the method used to teach intracorporeal suturing using the right and left hand from a lateral and suprapubic access. Data on sense of depth, coordination, dexterity, traction power, and posture at the beginning and at the end of the course were compiled. Three practical evaluations were performed by each course participant. Follow-up on subsequent live laparoscopic application of intracorporeal suturing was obtained.

RESULTS

We enrolled 44 consecutive trainees: 33 men and 11 women. We found a significant statistical improvement during the course in coordination (P=.001), dexterity (P=.000), traction power (P=.002), and posture (P=.003). Men were better than women in coordination (P=.002), dexterity (P=.000), and traction power (P=.014). No significant statistical difference in suturing skill was found in relation to age, gender, previous courses, surgical training (surgeon or resident), and dominant hand. Twenty-nine of 40 (72.5%) trainees after the course began to apply intracorporeal sutures in vivo.

CONCLUSIONS

The present study demonstrates the utility of a 5-day suturing course in teaching laparoscopic suturing technique. The "gladiator rule" is a useful and reproducible theory to teach intracorporeal knotting. The three-step model allows the majority of the trainees to apply laparoscopic suturing in vivo.

Voluntary autonomous simulator based training in minimally invasive surgery, residents' compliance and reflection

BACKGROUND

Knot tying and suturing skills in minimally invasive surgery (MIS) differ markedly from those in open surgery. Appropriate MIS training is mandatory before implementation into practice. The Advanced Suturing Course (ASC) is a structured simulator based training course that includes a 6-week autonomous training period at home on a traditional laparoscopic box trainer. Previous research did not demonstrate a significant progress in laparoscopic skills after this training period. This study aims to identify factors determining autonomous training on a laparoscopic box trainer at home.

METHODS

Residents (n = 97) attending 1 of 7 ASC courses between January 2009 and June 2011 were consecutively included. After 6 weeks of autonomous, training a questionnaire was completed. A random subgroup of 30 residents was requested to keep a time log. All residents received an online survey after attending the ASC. We performed outcome comparison to examine the accuracy of individual responses.

RESULTS

Out of 97 residents, the main motives for noncompliant autonomous training included a lack of (training) time after working hours (n = 80, 83.3%), preferred practice time during working hours (n = 76, 31.6%), or another surgical interest than MIS (n = 79, 15.2%). Previously set training goals would encourage autonomous training according to 27.8% (n = 18) of residents. Thirty participants submitted a time log and reported an average 76.5-minute weekly training time. All residents confirmed that autonomous home practice on a laparoscopic box trainer is valuable.

CONCLUSIONS

Autonomous practice should be structured and inclusive of adequate and sufficient feedback points. A minimally required practice time should be set. An obligatory assessment, including corresponding consequence should be conducted. Compliance herewith may result in increased voluntary (autonomous) simulator based (laparoscopic) training by residents.

The role of collaborative interactivity in the observational practice of clinical skills

CONTEXT

  Video-based observational practice can extend simulation-based learning outside the training space. This study explores the value of collaborative feedback provided during observational practice to the acquisition of clinical skills.

METHODS

  Nursing students viewed a video demonstrating the proper ventrogluteal injection technique before performing a videotaped pre-test trial on a simulator. They were then assigned randomly to one of three observational practice groups: a group that observed the expert demonstration (EO group); a group that viewed the expert demonstration, self-assessed their individual pre-test and contrasted their self-assessments with expert feedback (ESO group), and a group that observed the expert demonstration, self-assessed and contrasted their assessments with those of an expert, and formed a community that engaged in peer-to-peer feedback (ESPO group). The observation of all videos, the provision of assessments and all networking occurred via an Internet-mediated network. After 2 weeks, participants returned for post-tests and transfer tests.

RESULTS

  The pre-test-post-test analyses revealed significant interactions (global rating scale: F((2,22)) =4.00 [p =0.033]; checklist: F((2,22)) =4.31 [p =0.026]), which indicated that post-test performance in the ESPO group was significantly better than pre-test performance. The transfer analyses revealed main effects for both the global rating scale (F((2,23)) =6.73; p =0.005) and validated checklist (F((2,23)) =7.04; p =0.004) measures. Participants in the ESPO group performed better on the transfer test than those in the EO group.

CONCLUSIONS

  The results suggest that video-based observational practice can be effective in extending simulation-based learning, but its effectiveness is mediated by the amount of time the learner spends engaged in the practice and the type of learning activities the learner performs in the observational practice environment. We speculate that increasing collaborative interactivity supports observational learning by increasing the extent to which the educational environment can accommodate learners' specific needs.

The impact of nontechnical skills on technical performance in surgery: a systematic review

BACKGROUND

Failures in nontechnical and teamwork skills frequently lie at the heart of harm and near-misses in the operating room (OR). The purpose of this systematic review was to assess the impact of nontechnical skills on technical performance in surgery.

STUDY DESIGN

MEDLINE, EMBASE, PsycINFO databases were searched, and 2,041 articles were identified. After limits were applied, 341 articles were retrieved for evaluation. Of these, 28 articles were accepted for this review. Data were extracted from the articles regarding sample population, study design and setting, measures of nontechnical skills and technical performance, study findings, and limitations.

RESULTS

Of the 28 articles that met inclusion criteria, 21 articles assessed the impact of surgeons' nontechnical skills on their technical performance. The evidence suggests that receiving feedback and effectively coping with stressful events in the OR has a beneficial impact on certain aspects of technical performance. Conversely, increased levels of fatigue are associated with detriments to surgical skill. One article assessed the impact of anesthesiologists' nontechnical skills on anesthetic technical performance, finding a strong positive correlation between the 2 skill sets. Finally, 6 articles assessed the impact of multiple nontechnical skills of the entire OR team on surgical performance. A strong relationship between teamwork failure and technical error was empirically demonstrated in these studies.

CONCLUSIONS

Evidence suggests that certain nontechnical aspects of performance can enhance or, if lacking, contribute to deterioration of surgeons' technical performance. The precise extent of this effect remains to be elucidated.

Teaching subcuticular suturing to medical students: video versus expert instructor feedback

OBJECTIVE

Given limitations in surgical educational resources, more efficient teaching methods are needed. We sought to evaluate 3 strategies for improving skills in subcuticular suturing-practice with an instructional video, practice with expert instructor supervision, and independent practice.

DESIGN

Fifty-eight medical students volunteered for this research. Students viewed a video on subcuticular suturing then completed a pretest requiring closure of an incision in a plastic model. Students were randomized among 3 groups: practice with an instructional video (group A), practice with supervision by an expert instructor (group B), and independent practice (group C). After instruction, students completed a posttest, then a retention test 1 week later. Their performances were video recorded and evaluated using a validated scoring instrument composed of global and task-specific subscales.

RESULTS

Performances measured using both subscales improved significantly from pretest to post-test only for group B. However, when comparing student performances between pretest and retention posttest, significant improvements on both subscales were seen only in group A.

CONCLUSION

These results suggest that practice with an instructional video is an effective method for acquiring skill in subcuticular suturing.