Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains

ADEA-ADEE Shaping the Future of Dental Education III: Assessment in competency-based dental education: Ways forward

Assessment in competency-based dental education continues to be a recognized area for growth and development within dental programs around the world. At the joint American Dental Education Association (ADEA) and the Association for Dental Education in Europe (ADEE) 2019 conference, Shaping the Future of Dental Education III, the workshop on assessment was designed to continue the discussion started in 2017 at the ADEA-ADEE Shaping the Future of Dental Education II.¹ The focus of the 2019 conference involved examining the potential of entrustable professional activities (EPAs) and current thinking about workplace-based assessment (WBA) within competency-based education in the 21st century. Approximately 30 years ago, George Miller wrote about the assessment of competence in medical education and challenged faculty to reach for higher levels of assessment than knowledge or skill.² Acknowledging that no one assessment method can result in a valid assessment of competence, Miller proposed a four-level framework for assessment. The lowest level involves measuring what students know ("knows"), followed by assessment of the skill with which knowledge is applied in relevant tasks or problems ("knows how"). Next is an assessment of task performance in standardized settings ("shows how"), and finally, the highest level assesses the student's performance in the unstandardized clinical workplace ("does"). The 2019 assessment workshop focused on advances in the assessment of learners in the unstandardized workplace-the highest level of Miller's assessment pyramid ("does"). Research has shown that dental education has struggled to implement assessment strategies that meet this level.³ The workshop brought together individuals from around the world, with an interest in assessment in dental education, to consider how assessment in the "does" level, specifically EPAs and WBA, factors into competence assessment in dentistry/dental education.

Role of Peer Learning in Students' Skill Acquisition and Interest in Plastic Surgery

Supplemental Digital Content is available in the text.

Although the number of plastic surgery residency positions increased over the past decade, interest among Canadian medical students experienced the opposite trajectory. The aim of this study was to assess the effect of a low intensity, basic surgical skills workshop on medical students’ confidence and interest in surgery in general, and plastic surgery in particular.

Virtual reality in spinal endoscopy: a paradigm shift in education to support spine surgeons

BACKGROUND

Minimally invasive spine surgery (MISS) and endoscopic spine surgery have continually evolving indications in the cervical, thoracic, and lumbar spine. Endoscopic spine surgery entails treatment of disc disease, stenosis, spondylolisthesis, radiculopathy, and deformity. MISS involves complex motor skills in regions of variable anatomy. Simulator use has been proposed to aid in training and skill retention, preoperative planning, and intraoperative use.

METHODS

A systematic review of five databases was performed for publications pertaining to the use of virtual (VR), augmented (AR), and mixed (MR) reality in MISS and spinal endoscopic surgery. Qualitative data analysis was undertaken with focus of study design, quality, and reported outcomes. Study quality was assessed using the Medical Education Research Quality Instrument (MERSQI) score and level of evidence (LoE) by a modified Oxford Centre for Evidence-Based Medicine (OCEBM) level for simulation in medicine.

RESULTS

Thirty-eight studies were retained for data collection. Studies were of intervention-control, clinical application, and pilot or cross-sectional design. Identified articles illustrated use of VR, AR, and MR in all study designs. Procedures included pedicle cannulation and screw insertion, vertebroplasty, kyphoplasty, percutaneous transforaminal endoscopic discectomy (PTED), lumbar puncture and facet injection, transvertebral anterior cervical foraminotomy (TVACF) and posterior cervical laminoforaminotomy. Overall MERSQI score was low-to-medium [M =9.71 (SD =2.60); range, 4.5-13.5], and LoE was predominantly low given the number of purely descriptive articles, or low-quality randomized studies.

CONCLUSIONS

The current scope of VR, AR, and MR surgical simulators in MISS and spinal endoscopic surgery was described. Studies demonstrate improvement in technical skill and patient outcomes in short term follow-up. Despite this, overall study quality and levels of evidence remain low. Cohesive study design and reporting with focus on transfer validity in training scenarios, and patient derived outcome measures in clinical studies are required to further advance the field.

A Preoperative Educational Time-Out is Associated with Improved Resident Goal Setting and Strengthens Educational Experiences

OBJECTIVE

The purpose of this study was to assess the impact of a preoperative Educational Time-Out (ETO) with structured postoperative feedback on resident preoperative goal-setting and the educational experience of a clinical rotation.

DESIGN

A preoperative ETO was developed during which trainees and faculty jointly identified an operative goal and discussed the trainee's operative autonomy. Postoperative feedback with a smartphone application was encouraged. From November 2016 to October 2017, the intervention was piloted with 1 surgical service. Outcomes included ETO completion rate, goal setting rate, and subjects' perception of the impact of the ETO on identification of performance deficits, trainee autonomy, and receipt of feedback. Data were analyzed using descriptive statistics.

SETTING

This study was performed in an institutional hospital setting.

PARTICIPANTS

Third-year general surgery residents and surgical faculty in the Department of Hepatobiliary Surgery and Liver Transplantation at Vanderbilt University Medical Center took part in the intervention.

RESULTS

Seven residents and 7 attending surgeons participated in this study. Residents performed a median of 15 procurements during an average of 6.5 weeks each on service. The ETO completion rate was 83%. Resident-reported preoperative goal setting increased after the intervention (from 36% to 78%, p = 0.015). Subjects reported a positive impact of the intervention, with high resident agreement that the ETO helped identify deficits (82% median agreement), increased autonomy (82% median agreement), and increased receipt of feedback (84% median agreement). Residents and attendings agreed that the educational experience was stronger due to the ETO (median 81% and 77%, respectively).

CONCLUSIONS

The ETO intervention improved rates of resident preoperative goal setting and strengthened perceived educational experiences. Resident participants also reported improvements in autonomy and rates of postoperative feedback. Broader implementation of this brief preoperative pause is an easy way to emphasize procedural education in the operating room.

Exploring Use of Endoscopy Simulation in North American Pediatric Gastroenterology Fellowship Training Programs

OBJECTIVES

Increasing evidence supports simulation-based training; however, limited data exist regarding its use in pediatric gastroenterology (GI). We explored the use of simulation-based endoscopy training in pediatric GI fellowship programs across North America.

METHODS

GI fellowship program directors (PDs) from the United States and Canada were surveyed between August to November 2018. The pretested, electronic survey comprised 3 sections: program demographics; details of current simulation-based training; and PDs' perceptions of endoscopy simulation. Responses were analyzed using descriptive statistics.

RESULTS

Forty-three of 71 (61%) PDs responded (6 Canadian, 37 US). Programs were predominantly academic (95%) and enrolled 1.87 ± 1.01 fellows/yr. Twenty-four programs (56%) reported using simulation for endoscopy training, whereas 8 (19%) used simulation for nonprocedural education. Only 2 programs (5%) used endoscopy simulation for assessment. Of those using simulation (n = 24), upper endoscopy and colonoscopy were trained most frequently, and mechanical simulators were used most commonly. Eight programs (33%) required simulation training prior to clinical performance. Although 10 programs (42%) provided protected training time, only 2 (8%) tracked hours. Three programs (13%) reported having an organized curriculum and 6 (25%) train their endoscopic trainers. Cost, time constraints, and lack of a standardized curriculum were perceived as key barriers to integration. Most PDs reported a need for endoscopy simulation to train both technical and nontechnical skills; however, they felt simulation cannot replace clinical experience.

CONCLUSION

PDs recognize the potential importance of endoscopy simulation, particularly for novices; however, only 56% report using it. Perceived barriers indicate the need for inexpensive portable simulators and a validated pediatric simulation curriculum to promote uptake.

Using the Modified Delphi Method to Develop a Tool to Assess Pediatric Residents Supervising on Inpatient Rounds

BACKGROUND

Consensus about behaviors that define effective supervision by residents of more junior trainees on pediatric inpatient rounds is lacking.

OBJECTIVE

Use modified Delphi method to develop a checklist of essential supervisory behaviors pediatric residents demonstrate while leading inpatient, non-ICU, nonspecialty teaching rounds and pilot the checklist.

DESIGN/METHODS

One hundred and forty-one initial candidate behaviors were identified through literature review and narrowed by local stakeholders. Forty-one behaviors were submitted to national experts identified through purposive sampling to be refined using the modified Delphi method. Participants indicated agreement with behaviors' inclusion in the checklist and whether any were nonobservable. Measures of central tendency and dispersion were used to identify consensus with a behavior's inclusion in the next cycle. A criterion was eliminated if >25% of experts felt it was not observable. Cycles continued until consensus was reached on ≤20 behaviors. The resulting checklist was piloted at 2 hospitals.

RESULTS

After 2 modified Delphi cycles, consensus was obtained on 18 behaviors for inclusion in the final checklist. These were spread across 3 subcategories: teaching, leadership, and patient safety. In the pilot, the checklist of behaviors discriminated between residents with differing levels of competency in supervising trainees on inpatient rounds. For residents who had the checklist completed by 2 faculties, there was 75% agreement in behaviors observed.

CONCLUSION(S)

Based on literature review, local stakeholder input, and consensus of national experts through the modified Delphi method, we created and piloted a checklist of observable behaviors characteristic of effective clinical supervision by pediatric residents leading trainee teams on inpatient, non-ICU, nonspecialty rounds.

Transfer of Automated Performance Feedback Models to Different Specimens in Virtual Reality Temporal Bone Surgery

Virtual reality has gained popularity as an effective training platform in many fields including surgery. However, it has been shown that the availability of a simulator alone is not sufficient to promote practice. Therefore, simulator-based surgical curricula need to be developed and integrated into existing surgical training programs. As practice variation is an important aspect of a surgical curriculum, surgical simulators should support practice on multiple specimens. Furthermore, to ensure that surgical skills are acquired, and to support self-guided learning, automated feedback on performance needs to be provided during practice. Automated feedback is typically provided by comparing real-time performance with expert models generated from pre-collected data. Since collecting data on multiple specimens for the purpose of developing feedback models is costly and time-consuming, methods of transferring feedback from one specimen to another should be investigated. In this paper, we discuss a simple method of feedback transfer between specimens in virtual reality temporal bone surgery and validate the accuracy and effectiveness of the transfer through a user study.

Understanding the effects of structured self-assessment in directed, self-regulated simulation-based training of mastoidectomy: A mixed methods study

Objective

Self-directed training represents a challenge in simulation-based training as low cognitive effort can occur when learners overrate their own level of performance. This study aims to explore the mechanisms underlying the positive effects of a structured self-assessment intervention during simulation-based training of mastoidectomy.

Methods

A prospective, educational cohort study of a novice training program consisting of directed, self-regulated learning with distributed practice (5x3 procedures) in a virtual reality temporal bone simulator. The intervention consisted of structured self-assessment after each procedure using a rating form supported by small videos. Semi-structured telephone interviews upon completion of training were conducted with 13 out of 15 participants. Interviews were analysed using directed content analysis and triangulated with quantitative data on secondary task reaction time for cognitive load estimation and participants’ self-assessment scores.

Results

Six major themes were identified in the interviews: goal-directed behaviour, use of learning supports for scaffolding of the training, cognitive engagement, motivation from self-assessment, self-assessment bias, and feedback on self-assessment (validation). Participants seemed to self-regulate their learning by forming individual sub-goals and strategies within the overall goal of the procedure. They scaffolded their learning through the available learning supports. Finally, structured self-assessment was reported to increase the participants’ cognitive engagement, which was further supported by a quantitative increase in cognitive load.

Conclusions

Structured self-assessment in simulation-based surgical training of mastoidectomy seems to promote cognitive engagement and motivation in the learning task and to facilitate self-regulated learning.

Deliberate Practice: Applying the Expert Performance Approach to Gynecologic Surgical Training

Surgical training is shifting from the historical Halstedian apprenticeship model to outcomes-based methods. Surgical residents can reach a higher level of performance when utilizing deliberate practice and the expert performance approach. This article discusses methods for implementing deliberate practice and the expert performance approach into gynecologic surgical training programs.

Professionals' perspectives on factors affecting GP trainees' patient mix: results from an interview and focus group study among professionals working in Dutch general practice

INTRODUCTION

Seeing and treating patients in daily practice forms the basis of general practitioner (GP) training. However, the types of patients seen by GP trainees do not always match trainees' educational needs. Knowledge about factors that shape the mix of patient types is limited, especially with regard to the role of the professionals who work in the GP practice.

AIM

We investigated factors affecting the mix of patients seen by GP trainees from the perspective of professionals.

DESIGN AND SETTING

This qualitative study involved GP trainees, GP supervisors, medical receptionists and nurse practitioners affiliated with a GP Specialty Training Institute in the Netherlands.

METHODS

Twelve focus groups and seven interviews with 73 participants were held. Data collection and analysis were iterative, using thematic analysis with a constant comparison methodology.

RESULTS

The characteristics of patients' health problems and the bond between the doctor and patient are important determinants of GP trainees' patient mix. Because trainees have not yet developed bonds with patients, they are less likely to see patients with complex health problems. However, trainees can deliberately influence their patient mix by paying purposeful attention to bonding with patients and by gaining professional trust through focused engagement with their colleagues.

CONCLUSION

Trainees' patient mix is affected by various factors. Trainees and team members can take steps to ensure that this mix matches trainees' educational needs, but their success depends on the interaction between trainees' behaviour, the attitudes of team members and the context. The findings show how the mix of patients seen by trainees can be influenced to become more trainee centred and learning oriented.

Virtual coach: the next tool in functional endoscopic sinus surgery education

BACKGOUND

Functional endoscopic sinus surgery (FESS) can be challenging as novices become accustomed to handling endoscopes and instruments while navigating complex sinonasal anatomy. Experts demonstrate fluid and efficient motions when addressing pathology. The training process from novice to expert relies on hands-on experience in cadaveric laboratories and preceptorship models that require significant time and expense. This study aims to validate the use of a virtual coach to guide users step-by-step through a basic FESS.

METHODS

Seventeen surgeons were grouped into novice (n = 10) and expert (n = 7) based on self-reported levels of surgical experience. Users were trained using the maxillary antrostomy module in the Neurorhinological Surgery (NRS) simulator combining the physical craniofacial model with virtual reality (VR)-tracked surgical instruments in the VR operating room. The virtual coach guided the user using surgical videos, auditory, and visual cues. The coach recorded data for each subject including the number of times borders of the nasal cavity were encountered and time to completion. Users graded the usefulness of the virtual coach on a Likert questionnaire.

RESULTS

Face validity of our NRS simulator was replicated by user questionnaires, and construct validity replicated by differentiation between novice and expert level surgeons (p < 0.01). Novices contacted a significantly higher number of anatomic borders (n = 17) and had a longer operative time (t = 370 seconds). All users reported high scores on the benefit and usefulness of the virtual coach.

CONCLUSION

The virtual coach provides a useful tool to enhance FESS education by providing objective real-time data in a novel mixed-reality surgical environment.

Medical Students' Experiences and Outcomes Using a Virtual Human Simulation to Improve Communication Skills: Mixed Methods Study

BACKGROUND

Attending to the wide range of communication behaviors that convey empathy is an important but often underemphasized concept to reduce errors in care, improve patient satisfaction, and improve cancer patient outcomes. A virtual human (VH)-based simulation, MPathic-VR, was developed to train health care providers in empathic communication with patients and in interprofessional settings and evaluated through a randomized controlled trial.

OBJECTIVE

This mixed methods study aimed to investigate the differential effects of a VH-based simulation developed to train health care providers in empathic patient-provider and interprofessional communication.

METHODS

We employed a mixed methods intervention design, involving a comparison of 2 quantitative measures-MPathic-VR-calculated scores and the objective structured clinical exam (OSCE) scores-with qualitative reflections by medical students about their experiences. This paper is a secondary, focused analysis of intervention arm data from the larger trial. Students at 3 medical schools in the United States (n=206) received simulation to improve empathic communication skills. We conducted analysis of variance, thematic text analysis, and merging mixed methods analysis.

RESULTS

OSCE scores were significantly improved for learners in the intervention group (mean 0.806, SD 0.201) compared with the control group (mean 0.752, SD 0.198; F1,414=6.09; P=.01). Qualitative analysis revealed 3 major positive themes for the MPathic-VR group learners: gaining useful communication skills, learning awareness of nonverbal skills in addition to verbal skills, and feeling motivated to learn more about communication. Finally, the results of the mixed methods analysis indicated that most of the variation between high, middle, and lower performers was noted about nonverbal behaviors. Medium and high OSCE scorers most often commented on the importance of nonverbal communication. Themes of motivation to learn about communication were only present in middle and high scorers.

CONCLUSIONS

VHs are a promising strategy for improving empathic communication in health care. Higher performers seemed most engaged to learn, particularly nonverbal skills.

Overcoming the Shadow of Expertise: How Humility and Learning Goal Orientation Help Knowledge Leaders Become More Flexible

Although experts are valuable assets to organizations, they suffer from the curse of knowledge and cognitive entrenchment, which prevents them from being able to adapt to changing situational demands. In this study, I propose that experts' performance goal orientation resulting from pressures to perform contributes to their flexibility, but this mechanism can be moderated by learning goal orientation and humility. Data from a small sample of healthcare professionals suggested that performance goal orientation partially explained the mechanism of why experts may be inflexible. Humility, both as self-report and other-report measures, was found to be the most consistent moderator of this indirect effect. Experts with low levels of humility suffered from the negative effects of performance goal orientation, leading them to be less flexible compared to their counterparts with higher levels of humility. Experts who reported high levels of humility, on the other hand, were perceived to be more flexible as their expertise increased. Meanwhile, learning goal orientation partially moderated the indirect effect of expertise on flexibility through performance goal orientation. These findings lead to new conversations on how to get experts unstuck and highlight the importance of developing humility as both a personal virtue and a strategic advantage for organizations.

Assessment of communication skills

OBJECTIVE

This paper addresses how communication skills can best be assessed. Since assessment and learning are strongly connected, the way communication skills are best learned is also described.

RESULTS

Communication skills are best learned in a longitudinal fashion with ample practice in an authentic setting. Confrontation of behavior initiates the learning process and should be supported by meaningful feedback through direct observation. When done appropriately a set of (learned) communication skills become integrated skilled communication, being versatilely used in purposeful goal-oriented clinical communication. The assessment of communication skills should follow a modern approach to assessment where the learning function of assessment is considered a priority. Individual assessments are feedback-oriented to promote further learning and development. The resulting rich information may be used to make progression decisions, usually in a group or committee decision.

CONCLUSION

This modern programmatic approach to assessment fits the learning of skilled communication well.

PRACTICE IMPLICATIONS

Implementation of a programmatic assessment approach to communication will entail a major innovation to education.

AHRQ EPC Series on Improving Translation of Evidence: Use of a Clinical Pathway for C. Difficile Treatment to Facilitate the Translation of Research Findings into Practice

BACKGROUND

In 2018 the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program issued a call for strategies to disseminate AHRQ EPC systematic reviews. In this pilot, findings from the 2016 AHRQ EPC report on Clostridioides difficile infection were translated into a treatment pathway and disseminated via a cloud-based platform and electronic health record (EHR).

METHODS

An existing 10-step framework was used for developing and disseminating evidence-based clinical pathways. The development of the EHR intervention was informed by the Five Rights model for clinical decision support and human-computer interaction design heuristics. The researchers used observations and time measurements to describe the impact of the EPC report on pathway development and examined provider adoption using counts of pathway views.

RESULTS

Two main themes emerged: (1) discrepancies between the EPC report and existing guidelines prompted critical discussions about available treatments, and (2) lack of guideline and pathway syntheses in the EPC report necessitated a rapid literature review. Pathway development required 340 hours: 205 for the rapid literature review, 63 for pathway development and EHR intervention design, and 5 for technical implementation of the intervention. Pathways were viewed 1,069 times through the cloud-based platform and 47 times through a hyperlink embedded in key EHR ordering screens.

CONCLUSION

Pathways can be an approach for disseminating AHRQ EPC report findings within health care systems; however, reports should include guideline and pathway syntheses to meet their full potential. Embedding hyperlinks to pathway content within the EHR may be a viable and low-effort solution for promoting awareness of evidence-based resources.

[Entrustable professional activities : Promising concept in postgraduate medical education]

Entrustable professional activities (EPAs) are characterized as self-contained units of work in a given typical clinical context, which may be entrusted to a trainee for independent execution at a certain point of training. An example could be the intraoperative anesthesia management of an ASA 1 patient for an uncomplicated surgical intervention as an EPA in early postgraduate anesthesia training. The EPAs can be described as an evolution of a competency-based medical educational concept, applying the concept of the competencies of a person to specific workplace contexts. In this way the expected level of skills and supervision at a certain stage of training have a more practical meaning and the danger of fragmentation of individual competencies in the competence-based model is avoided. It is a more holistic view of a trainee. Experience with this new concept is so far limited, therefore, further studies are urgently needed to determine whether and how EPAs can contribute to improvements in further training.

Is surgery more about doing than thinking?

Detailed thought, knowledge, complex analysis, reasoned judgment and professionalism all fundamentally underpin a surgeon's work and training, yet there is a popularly held view that accomplished surgeons are primarily concerned with performing procedures. A review of pedagogical, social and medical literature, together with personal reflections from the authors, shows that a surgeon's work is multi-faceted. This article discusses the technical skills of operating as a reflection of the 'tip of the iceberg' of a surgeon's cognition, the increasingly multidisciplinary strategic approach of surgeons today, the importance of surgical decision making, the influence of robotics, the role of non-medically trained staff, surgeons' role in postoperative care, adaptive expertise and the formation of professional identity. In so doing, a much wider view of a surgeon than simply 'doing' or 'thinking' is presented with implications for surgical training.

A multiple-trainee, multiple-level, multiple-competency (multi-TLC) simulation-based approach to training obstetrical emergencies

Competency-based education requires that programs increase the breadth of direct observation and assessment to improve resident training. To achieve these goals, the authors developed and executed a multiple-trainee, multiple-level, multiple-competency (Multi-TLC) obstetrical emergencies simulation curriculum. Depending upon their training level (PGY1-PGY5), obstetrics and gynaecology residents participated in various roles (i.e., first responder, second responder, confederates, and evaluators) within four simulation scenarios designed to provide opportunities for education, direct observation, and assessment across a number of competencies (i.e., medical expert, communicator, collaborator, leader, advocate, and scholar). The curriculum was carried out over 8 h spread evenly across 2 days (i.e., 4 h/day) and involved periods of pre-briefing, live simulation, and debriefing. An evaluation of the Multi-TLC was operationalised via a context-input-process-product model. This report presents the outcomes of that evaluation derived from quasi-experimental comparisons of the new and previous curricula across four priorities for simulation-based education identified by the Department of Obstetrics and Gynecology at McMaster University (Hamilton, ON, Canada): increasing learning opportunities, maintaining or improving resident learning, maintaining or reducing program costs, and improving resident satisfaction. The evaluation revealed that the Multi-TLC curriculum permitted a greater breadth of direct observation and assessment across competencies, maintained the previous learning objectives while also addressing additional ones, and was done so in a way that reduced the overall financial and human resource costs associated with the department's obstetrical emergency simulation curriculum. A Multi-TLC organisation of simulation curricula can facilitate efficient application of competency-based education principles.

Advanced airway training in the UK: A national survey of senior anesthetic trainees

Background and Aims:

High-quality training in advanced airway skills is imperative to ensure safe anesthetic care and develop future airway specialists. Modern airway management skills are continually evolving in response to advancing technology and developing research. Therefore, it is of concern that training provisions and trainee competencies remain current and effective.

Material and Methods:

A survey questionnaire based on the airway competencies described in the Royal College of Anaesthetists’ curriculum and Difficult Airway Society guidelines was posted to all United Kingdom (UK) National Health Service hospitals to be completed by the most senior anesthetic trainee (ST 5–7, resident).

Results:

A total of 149 responses were analyzed from 237 hospitals with eligible anesthetic trainees (response rate 63%), including 53 (36%) and 39 (26%) respondents who had completed higher and advanced level airway training respectively. Although clinical experience with videolaryngoscopy was satisfactory, poor confidence and familiarity was identified with awake fiberoptic intubation, high frequency jet ventilation, at risk extubation techniques, and airway ultrasound assessment. Only 26 (17%) respondents had access to an airway skills room or had regular airway emergency training with multidisciplinary theater team participation. Reported barriers to training included lack of training lists, dedicated teaching time, experienced trainers, and availability of equipment.

Conclusions:

This national survey identified numerous deficiencies in airway competencies and training amongst senior anesthetic trainees (residents) in the UK. Restructuring of the airway training program and improvements in access to training facilities are essential to ensure effective airway training and the capability to produce future airway specialists.

Acquiring and Maintaining Technical Skills Using Simulation: Initial, Maintenance, Booster, and Refresher Training

Simulation-based education has been shown to be an effective tool to mitigate skill decay. However, many of the strategies reported in the literature have overlapping terminology with little consensus on the timing of the strategy to prevent skill decay. In this review, we propose and provide a standardized nomenclature and framework for simulation strategies used to obtain, maintain, or regain skills that are decaying. This framework delineates four types of training: initial, maintenance, booster, and refresher. The framework differentiates these training types based on the learner competency at the time of the training, as well as the frequency and intensity of the training. Initial training is aimed at “novice” learners with the goal to achieve competency. Once competency is achieved, maintenance training prevents skill deterioration through low-dose high-frequency (LDHF) training. Booster training is used when the learner is still proficient, but competency begins to wane. Booster training occurs less frequently than maintenance training but with greater intensity to overcome the skill decay that occurs over time. Refresher training is aimed at re-establishing skill levels after competency has reached unsatisfactory levels. Refresher training is higher intensity than booster and maintenance training. We describe simulation-based strategies reported in the literature that can be used for each type of training. We conclude that there should be an increased emphasis in medical education towards maintenance and booster training in order to preserve skills before competency is lost. 

Recommendations on the Use of Ultrasound Guidance for Central and Peripheral Vascular Access in Adults: A Position Statement of the Society of Hospital Medicine

PREPROCEDURE

1)We recommend that providers should be familiar with the operation of their specific ultrasound machine prior to initiation of a vascular access procedure. 2)We recommend that providers should use a high-frequency linear transducer with a sterile sheath and sterile gel to perform vascular access procedures. 3)We recommend that providers should use two-dimensional ultrasound to evaluate for anatomical variations and absence of vascular thrombosis during preprocedural site selection. 4)We recommend that providers should evaluate the target blood vessel size and depth during preprocedural ultrasound evaluation.

TECHNIQUES

General Techniques 5) We recommend that providers should avoid using static ultrasound alone to mark the needle insertion site for vascular access procedures. 6)We recommend that providers should use real-time (dynamic), two-dimensional ultrasound guidance with a high-frequency linear transducer for central venous catheter (CVC) insertion, regardless of the provider's level of experience. 7)We suggest using either a transverse (short-axis) or longitudinal (long-axis) approach when performing real-time ultrasound-guided vascular access procedures. 8)We recommend that providers should visualize the needle tip and guidewire in the target vein prior to vessel dilatation. 9)To increase the success rate of ultrasound-guided vascular access procedures, we recommend that providers should utilize echogenic needles, plastic needle guides, and/or ultrasound beam steering when available. Central Venous Access Techniques 10) We recommend that providers should use a standardized procedure checklist that includes the use of real-time ultrasound guidance to reduce the risk of central line-associated bloodstream infection (CLABSI) from CVC insertion. 11)We recommend that providers should use real-time ultrasound guidance, combined with aseptic technique and maximal sterile barrier precautions, to reduce the incidence of infectious complications from CVC insertion. 12)We recommend that providers should use real-time ultrasound guidance for internal jugular vein catheterization, which reduces the risk of mechanical and infectious complications, the number of needle passes, and time to cannulation and increases overall procedure success rates. 13)We recommend that providers who routinely insert subclavian vein CVCs should use real-time ultrasound guidance, which has been shown to reduce the risk of mechanical complications and number of needle passes and increase overall procedure success rates compared with landmark-based techniques. 14)We recommend that providers should use real-time ultrasound guidance for femoral venous access, which has been shown to reduce the risk of arterial punctures and total procedure time and increase overall procedure success rates. Peripheral Venous Access Techniques 15) We recommend that providers should use real-time ultrasound guidance for the insertion of peripherally inserted central catheters (PICCs), which is associated with higher procedure success rates and may be more cost effective compared with landmark-based techniques. 16)We recommend that providers should use real-time ultrasound guidance for the placement of peripheral intravenous lines (PIV) in patients with difficult peripheral venous access to reduce the total procedure time, needle insertion attempts, and needle redirections. Ultrasound-guided PIV insertion is also an effective alternative to CVC insertion in patients with difficult venous access. 17)We suggest using real-time ultrasound guidance to reduce the risk of vascular, infectious, and neurological complications during PIV insertion, particularly in patients with difficult venous access. Arterial Access Techniques 18)We recommend that providers should use real-time ultrasound guidance for arterial access, which has been shown to increase first-pass success rates, reduce the time to cannulation, and reduce the risk of hematoma development compared with landmark-based techniques. 19)We recommend that providers should use real-time ultrasound guidance for femoral arterial access, which has been shown to increase first-pass success rates and reduce the risk of vascular complications. 20)We recommend that providers should use real-time ultrasound guidance for radial arterial access, which has been shown to increase first-pass success rates, reduce the time to successful cannulation, and reduce the risk of complications compared with landmark-based techniques.

POSTPROCEDURE

21) We recommend that post-procedure pneumothorax should be ruled out by the detection of bilateral lung sliding using a high-frequency linear transducer before and after insertion of internal jugular and subclavian vein CVCs. 22)We recommend that providers should use ultrasound with rapid infusion of agitated saline to visualize a right atrial swirl sign (RASS) for detecting catheter tip misplacement during CVC insertion. The use of RASS to detect the catheter tip may be considered an advanced skill that requires specific training and expertise.

TRAINING

23) To reduce the risk of mechanical and infectious complications, we recommend that novice providers should complete a systematic training program that includes a combination of simulation-based practice, supervised insertion on patients, and evaluation by an expert operator before attempting ultrasound-guided CVC insertion independently on patients. 24)We recommend that cognitive training in ultrasound-guided CVC insertion should include basic anatomy, ultrasound physics, ultrasound machine knobology, fundamentals of image acquisition and interpretation, detection and management of procedural complications, infection prevention strategies, and pathways to attain competency. 25)We recommend that trainees should demonstrate minimal competence before placing ultrasound-guided CVCs independently. A minimum number of CVC insertions may inform this determination, but a proctored assessment of competence is most important. 26)We recommend that didactic and hands-on training for trainees should coincide with anticipated times of increased performance of vascular access procedures. Refresher training sessions should be offered periodically. 27)We recommend that competency assessments should include formal evaluation of knowledge and technical skills using standardized assessment tools. 28)We recommend that competency assessments should evaluate for proficiency in the following knowledge and skills of CVC insertion: (a) Knowledge of the target vein anatomy, proper vessel identification, and recognition of anatomical variants; (b) Demonstration of CVC insertion with no technical errors based on a procedural checklist; (c) Recognition and management of acute complications, including emergency management of life-threatening complications; (d) Real-time needle tip tracking with ultrasound and cannulation on the first attempt in at least five consecutive simulation. 29)We recommend a periodic proficiency assessment of all operators should be conducted to ensure maintenance of competency.

In Search of Characterizing Surgical Skill

OBJECTIVE

This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill.

DESIGN

Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency.

STRUCTURE

Each term is discussed in turn, drawing from existing literature and colloquial uses.

IMPLICATIONS

A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.

A multi-disciplinary model of life-course canalization and agency

This article integrates life-course sociological insights and perspectives with the conceptions of agency and individual motivation formulated as the motivational theory of life-span development. We use Waddington's epigenetic landscape as a metaphor for how life courses are shaped jointly by societal structure and individual agency. Social structure imposes constraints and institutions provide the transitions and pathways that together constitute critical scaffolding for life-course timing and path dependency ("canalization"). The building blocks from developmental and motivational psychology as well as from life-course sociology are introduced first. Then we address the dynamic interplay of individual agent and society in terms of life-span timing and life-course canalization (i.e., path-dependency) effects. The proposed conceptual framework moves beyond previous accounts of agent-society interplay in two distinct ways. First, we develop a systematically organized set of specific phenomena of developmental canalization on the one hand, and of institutionalized or social-structure based canalization on the other. Second, we offer a discussion of a set of scenarios that show how these specific psychological and society-generated processes may play together to shape individuals' life courses and life-span development.

Stepwise Training in Laparoscopic Surgery for Complex Ileocolonic Crohn's Disease: Analysis of 127 Training Episodes

INTRODUCTION

The inflammation encountered in Crohn's disease makes a minimally invasive approach challenging due to a thickened mesentery, fistulas, abscesses, and large phlegmons with high reported rates of conversion and septic complications. Aim of this study was to evaluate the feasibility of a stepwise approach to training in laparoscopic surgery for complex Crohn's disease.

METHODS

Every surgical procedure was divided in 4 different training tasks: access and exposure, bowel mobilization, division of the mesentery, anastomosis. Extensive adhesiolysis and division and repair of fistulae were considered as additional tasks when present. The laparoscopic competence assessment tool was used to evaluate the safety and proficiency of the surgical performance. The primary outcome was the rate of training tasks successfully completed by surgical trainees.

RESULTS

One hundred and twenty seven training episodes were included and 86 were performed by trainees (67.7%). Fistula division was the less commonly performed training task (25%), while mobilisation and anastomosis were performed by the supervised trainee in 90% and 85% of the cases. Safety and proficiency scores were significantly higher for senior trainees compared to junior trainees.

CONCLUSIONS

Laparoscopic surgery for complex Crohn's disease can be safely performed in a supervised setting with acceptable operating time, postoperative length of hospital stay, and 30 day morbidity.

Ophthalmic simulated surgical competency assessment rubric (Sim-OSSCAR) for trabeculectomy

Background/aims

To develop, test and determine whether a surgical-competency assessment tool for simulated glaucoma surgery is valid.

Methods

The trabeculectomy ophthalmic simulated surgical competency assessment rubric (Sim-OSSCAR) was assessed for face and content validity with a large international group of expert eye surgeons. Cohorts of novice and competent surgeons were invited to perform anonymised simulation trabeculectomy surgery, which was marked using the Sim-OSSCAR in a masked fashion by a panel of four expert surgeons. Construct validity was assessed using a Wilcoxon rank-sum test. Krippendorff’s alpha was calculated for interobserver reliability.

Results

For the Sim-OSSCAR for trabeculectomy, 58 of 67 surgeons (86.6%) either agreed or strongly agreed that the Sim-OSSCAR is an appropriate way to assess trainees’ surgical skill. Face validity was rated as 4.04 (out of 5.00). Fifty-seven of 71 surgeons (80.3%) either agreed or strongly agreed that the Sim-OSSCAR contents represented the surgical technique of surgical trabeculectomy. Content validity was rated as 4.00. Wilcoxon rank-sum test showed that competent surgeons perform better than novices (p=0.02). Interobserver reliability was rated >0.60 (Krippendorff’s alpha) in 19 of 20 steps of the Sim-OSSCAR.

Conclusion

The Sim-OSSCAR for trabeculectomy, a newly developed and validated assessment tool for simulation glaucoma surgery, has validity and reliability. It has the potential to play a useful role in ophthalmic surgical education.

Quantifying the medical student learning curve for ECG rhythm strip interpretation using deliberate practice

Objectives: Obtaining competency in medical skills such as interpretation of electrocardiograms (ECGs) requires repeated practice and feedback. Structured repeated practice and feedback for ECGs is likely not provided to most medical students, so skill development is dependent on opportunistic training during clinical rotations. Our aim was to describe: the amount of deliberate practice completed for learning ECG rhythm strip diagnoses in first year medical students, the learning curve for rhythm strip diagnosis, and student experiences with deliberate practice. Methods: First year medical students from two medical schools were provided with online rhythm strip practice cases. Diagnostic accuracy was measured throughout practice, and students were provided feedback for every case they completed. Total cases practiced and time spent practicing were correlated with their performance during practice and on an exam. Results: 314 of 384 (82%) students consented. The mean number of ECGs each student practiced was 59 (range 0-280), representing 18,466 total instances of deliberate practice. We generated mathematical models that accurately correlated both the number of cases practiced and time spent practicing, with diagnostic accuracy on an exam (p<0.001). For example, students would need to spend on average of 112 minutes and complete 34 practice cases to obtain 75% on an ECG rhythm strip exam. Student satisfaction was high using the online cases. Conclusions: We succeeded in delivering deliberate practice for ECG rhythm strip interpretation to a large cohort of students at 2 medical schools. We quantified a learning curve that estimates the number of cases and practice time required to achieve pre-determined levels of diagnostic accuracy. This data can help inform a competency-based approach to curriculum development.

Complex Motor Learning and Police Training: Applied, Cognitive, and Clinical Perspectives

The practices surrounding police training of complex motor skills, including the use of force, varies greatly around the world, and even over the course of an officer’s career. As the nature of policing changes with society and the advancement of science and technology, so should the training practices that officers undertake at both central (i.e., police academy basic recruit training) and local (i.e., individual agency or precinct) levels. The following review is intended to bridge the gap between scientific knowledge and applied practice to inform best practices for training complex motor skills that are unique and critical to law enforcement, including the use of lethal force. We begin by providing a basic understanding of the fundamental cognitive processes underlying motor learning, from novel skill acquisition to complex behaviors including situational awareness, and decision-making that precede and inform action. Motor learning, memory, and perception are then discussed within the context of occupationally relevant stress, with a review of evidence-based training practices that promote officer performance and physiological responses to stress during high-stakes encounters. A lack of applied research identifying the neurophysiological mechanisms underlying motor learning in police is inferred from a review of evidence from various clinical populations suffering from disorders of cognitive and motor systems, including Alzheimer’s and Parkinson’s disease and stroke. We conclude this review by identifying practical, organizational, and systemic challenges to implementing evidence-based practices in policing and provide recommendations for best practices that will promote training effectiveness and occupational safety of end-users (i.e., police trainers and officers).

Self-directed learning and practice of Italian osteopathic students during summer break: a cross-sectional survey

Background

Self-directed learning is a student-centered approach based on the students’ evaluation about their own learning needs. Self-directed practice is a component of the self-directed learning which focus on achieving manual and practical skills. Given the abundance of manual skills that students need to develop in the osteopathic curriculum, self-directed practice has become an attractive methodology. Most of the Italian osteopathic educational institutions recognize two types of educational curriculum: T1, for students without a healthcare background; and T2 for students who are already healthcare professionals. The aim of the present study is to investigate the attitudes of Italian osteopathic students toward self-directed practice during the summer break highlighting the possible differences between T1 and T2 students.

Methods

A web-based closed-ended survey was administered to the students attending one of the Italian osteopathic educational institutions which accepted to participate in the research.

Results

Two hundred sixty-eight students on 2549 participated to the survey. 64.92% of the students reported to have performed self-directed practice during the summer break. The main difficulty encountered by students in approaching self-directed practice was ‘lack of people to exercise with’. Most of the students performed self-directed practice between 1 to 5 h per week. The most common clinical condition encountered was Low Back Pain. The anatomical area most approached was lumbar spine. The most commonly simulated tests were the articular mobility tests. The techniques in which students trained the most were muscle energy techniques. Significant difference was found in self-engagement to the self-directed practice between T1 and T2 students (p = 0.026), and in the reasons to perform self-directed practice (p = 0.027).

Conclusions

The results of this study could be useful to reveal some aspects of students’ academic education and future clinical practice. Therefore this paper can be a useful tool for the institutions to develop strategies to overcome the reported problems students have towards self-directed practice. For example it should be discussed the possibility for the students to observe some senior osteopaths during their practice or to plan to maintain an open tutored didactic environment during the summer break as an extracurricular activity.

A Content Analysis of the ACGME Specialty Milestones to Identify Performance Indicators Pertaining to the Development of Residents as Educators

Construct: For curriculum development purposes, this study examined how the development of residents as educators is reflected in the Accreditation Council for Graduate Medical Education (ACGME) Milestones. Background: Residents teach patients, families, medical students, physicians, and other health professionals during and beyond their training. Despite this expectation, it is unclear how the development of residents as educators is reflected in the specialty-specific Milestones. Approach: We performed a textual content analysis of 25 specialty Milestone documents available as downloads from the ACGME website in December 2015. Syntactical units of interest included developmental progressions that describe the development of educators over the course of residency training and 16 key terms identified during the analysis. We then categorized the terms by associated Milestone level, ACGME core competency, and targeted learner(s). Results: We identified 10 developmental progressions and 546 instances of the 16 key terms that describe the development of physician educators. The frequency of terms among specialties was quite variable (5-46 terms per specialty, Mdn = 21). The majority of education-related terms appeared at advanced Milestone levels; there were 139 (26%) such instances in Level 4 and 296 (54%) in Level 5. Education-related terms were identified in all six ACGME core competencies, with greatest frequency in Patient Care (157, 29%). Other residents were the learners most frequently targeted by education-related Milestones (211, 40%). Conclusions: The current ACGME Milestones largely imply that resident teaching is a high-level or aspirational goal, achieved without a clear or consistently assessed developmental progression. These findings run counter to the theoretical basis that underlies the development of the Milestones. Wide variation among specialties indicates lack of consensus around the ideal skill set of the resident educator and limits the utility of these documents for curriculum development in this domain.

A Survey Assessment of Perceived Importance and Methods of Maintenance of Critical Procedural Skills in Pediatric Emergency Medicine

OBJECTIVE

The aim of this study was to delineate pediatric emergency medicine provider opinions regarding the importance of, and to ascertain existing processes by which practitioners maintain, the following critical procedural skills: oral endotracheal intubation, intraosseous line placement, pharmacologic and electrical cardioversion, tube thoracostomy, and defibrillation.

METHODS

A customized survey was administered to all members of the Listserv for the American Academy of Pediatrics Section on Emergency Medicine. Perceived importance of maintaining critical pediatric procedural skills was measured using a 5-point Likert-type scale. Secondary outcomes included presence and type of mandatory training, availability of on-site backup, and perceived barriers to maintenance of skills.

RESULTS

Two hundred sixty-two members (25%) responded representing 106 different institutions, 70% of freestanding children's hospitals that received graduate medical education payments in 2014, and 68% of pediatric emergency medicine fellowship programs. More than 90% of respondents felt it was either very or extremely important to maintain competency for 5 of the 6 critical procedures, but no more than 49% of respondents felt that clinical care alone provided opportunity to maintain skills. The proportion of respondents indicating no mandatory training for each critical procedural skill was as follows: oral endotracheal intubation (23%), intraosseous line placement (30%), pharmacologic cardioversion (32%), electrical cardioversion (32%), tube thoracostomy (40%), and defibrillation (32%).

CONCLUSIONS

Critical procedural skills are perceived by emergency providers who care for children as extremely important to maintain. Direct care of pediatric patients likely does not provide sufficient opportunity to maintain these skills. There are widespread deficiencies relating to mandatory maintenance of critical procedural skill training.

Objective Assessment of Resident Teaching Competency Through a Longitudinal, Clinically Integrated, Resident-as-Teacher Curriculum

OBJECTIVE

Competency-based training should be paired with objective assessments. To date, there has been limited objective assessment of resident-as-teacher curricula (RATC). We sought to assess the impact of a longitudinal RATC on postgraduate year-1 (PGY1) resident teaching competency using Observed Structured Teaching Encounters (OSTEs) for the skills of 1) brief didactic teaching [DT], 2) feedback [FB], and 3) precepting [PR].

METHODS

A controlled, prospective, educational study was conducted from May 2015 to June 2016. The RATC consisted of a workshop series with reinforcement of key skills (DT, FB) during clinical rotations. Intervention residents participated in the RATC and completed OSTEs at the beginning and end of the academic year. A control group, PGY1 residents that matriculated the year previously, completed the OSTEs before starting their PGY2 year. OSTEs were reviewed by 2 blinded study personnel. We assessed reliability between raters via intraclass correlation coefficients and differences in OSTE scores via least squared mean differences (LSMD).

RESULTS

In total, 92.5% (n = 37) of eligible control and 100% (n = 41) of eligible intervention residents participated. The OSTEs demonstrated excellent agreement between reviewers (DT: 0.99, FB: 0.89, PR: 0.98). A significant pre-post difference was demonstrated in the intervention group for DT (LSMD [95% confidence interval], 3.14 [2.49-3.79], P < .0001), FB (0.93 [0.49-1.37], P < .0001), and PR (0.64 [0.09-1.18], P < .022). A significant difference between the control and intervention groups was demonstrated for DT (3.00 [2.05-3.96], P < .0001).

CONCLUSIONS

Skill-based OSTEs can be used to detect changes in residents' teaching competency and may represent a potential component of programmatic evaluation of resident-as-teacher curricula.

A Statewide Surgical Coaching Program Provides Opportunity for Continuous Professional Development

OBJECTIVE

We sought to develop and evaluate a video-based coaching program for board-eligible/certified surgeons.

SUMMARY BACKGROUND DATA

Multiple disciplines utilize coaching for continuous professional development; however, coaching is not routinely employed for practicing surgeons.

METHODS

Peer-nominated surgeons were trained as coaches then paired with participant surgeons. After setting goals, each coaching pair reviewed video-recorded operations performed by the participating surgeon. Coaching sessions were audio-recorded, transcribed, and coded to identify topics discussed. The effectiveness with which our coaches were able to utilize the core principles and activities of coaching was evaluated using 3 different approaches: self-evaluation; evaluation by the participants; and assessment by the study team. Surveys of participating surgeons and coach-targeted interviews provided general feedback on the program. All measures utilized a 5-point Likert scale format ranging from 1 (low) to 5 (high).

RESULTS

Coach-participant surgeon pairs targeted technical, cognitive, and interpersonal aspects of performance. Other topics included managing intraoperative stress. Mean objective ratings of coach effectiveness was 3.1 ± 0.7, ranging from 2.0 to 5.0 on specific activities of coaching. Subjective ratings by coaches and participants were consistently higher. Coaches reported that the training provided effectively prepared them to facilitate coaching sessions. Participants were similarly positive about interactions with their coaches. Identified barriers were related to audio-video technology and scheduling of sessions. Overall, participants were satisfied with their experience (mean 4.4 ± 0.7) and found the coaching program valuable (mean 4.7 ± 0.7).

CONCLUSIONS

This is the first report of cross-institutional surgical coaching for the continuous professional development of practicing surgeons, demonstrating perceived value among participants, as well as logistical challenges for implementing this evidence-based program. Future research is necessary to evaluate the impact of coaching on practice change and patient outcomes.

Evaluation of medical student retention of clinical skills following simulation training

BACKGROUND

Adequate clinical skills training is a challenge for present day medical education. Simulation Based Education (SBE) is playing an increasingly important role in healthcare education worldwide to teach invasive procedures. The impact of this teaching on students along with retention of what is taught is not fully understood. The purpose of this study was to evaluate the retention levels of practical skills taught and assessed by SBE and to explore the degree of re-training required to restore decayed performance. In exploring this aim, the study further investigates how skilled performance decays over time and which dimensions of clinical skills were more likely to decay.

METHODS

Study participants were 51 final year medical students. They were provided with online pre-course videos and procedural guides asynchronously with repeatedly access. 7 of the skills taught over 2 years using task trainers were selected. Following demonstration from faculty, students practiced in small groups with faculty facilitated supervision and peer support prior to formal testing. Score sheets with itemised procedure checklists detailing the minimum passing standard (MPS) for each skill were designed. To test retention of skills, 18 months later, there was an unannounced test to demonstrate proficiency in the skills. Students were asked to complete a questionnaire indicating how many times and where they had practiced or performed the skills.

RESULTS

55% of the students were deficient in 3 or more skills and 4% were not competent in 5 or more skills. A significant number of students had never practiced some skills following the initial teaching session. A relationship was noted with the number of times students self-declared that they had practiced and their performance. Decay is evident in both psychomotor and cognitive domains of the skills.

CONCLUSION

A curriculum with deliberate practice significantly increases the competence of students in defined clinical skills. Deliberate practice of clinical skills, under supervision of an engaged instructor, is a key component of the mastery model. Experiences and assessments in the clinical setting need to be augmented with focus on direct observation and focused feedback to reinforce the skills acquired in the simulated setting.