Learn, see, practice, prove, do, maintain: an evidence-based pedagogical framework for procedural skill training in medicine

How does virtual reality simulation compare to simulated practice in the acquisition of clinical psychomotor skills for pre-registration student nurses? A systematic review

INTRODUCTION

Simulated practice, both face-to-face and computer-based, is well established within healthcare education, allowing rehearsal and refinement of clinical skills. Virtual reality is a new and relatively untested method of delivering simulation learning.

AIMS

This project aims to systematically review, critically appraise and synthesise the published evidence in order to answer the question 'How does virtual reality simulation compare to simulated practice in the acquisition of clinical psychomotor skills for pre-registration student nurses?'

METHODS

The databases CINAHL, Medline, Psychinfo, PubMed and the University of Portsmouth 'Discover' database were searched between 4th June 2018 and 7th July 2018 using the terms; pre-registration, pre-licensure, "pre-registration", "pre licensure", trainee, student, students, nurs*, virtual-reality, VR, "virtual reality", "augmented reality", clinical, skil*, competenc* and mastery. Inclusion and exclusion criteria relating to type of paper, population, intervention, comparison and outcome were applied. Selected articles were appraised using the Centre for Reviews and Dissemination guidelines. As clinical psychomotor skill mastery requires the application of theoretical knowledge to a motor skill in a range of contexts, outcomes relating to these elements (namely knowledge, cognitive gain, skill performance, skill success and time to complete) were analysed.

FINDINGS

Nine studies were included in the review. All studies employed a quasi-experimental design but were of mixed methodological quality. There was significant heterogeneity in methods and missing data, limiting synthesis and precluding meta-analysis. Virtual reality groups performed favourably in comparison to simulation groups in posttest knowledge scores, cognitive gain, skill performance scores and skill success rate. There was divergence of results in relation to time taken to complete the skill.

DISCUSSION

Whilst the results are generally favourable for virtual reality, variation in devices, data collection tools and outcome measurements mean that caution must be used in their interpretation. Outcomes relating to psychomotor skill performance support the use of virtual reality as an educational intervention. Time taken to undertake and complete the skill is questioned as a valid outcome measurement due to the potential to forgo skill accuracy in favour of speed.

CONCLUSION

Virtual reality is an emerging technology with a limited body of evidence, which is of variable methodological quality. It appears that virtual reality leads to educational outcomes similar or superior to traditional simulated practice. Consensus in definitions is needed along with further research to advance knowledge of this developing area of practice. Such research is needed to justify the cost of investing in this new technology.

Do Skills Acquired from Training with a Wire Navigation Simulator Transfer to a Mock Operating Room Environment?

BACKGROUND

Skills training and simulation play an increasingly important role in orthopaedic surgical education. The intent of simulation is to improve performance in the operating room (OR), a trait known as transfer validity. No prior studies have explored how simulator-based wire navigation training can transfer to higher-level tasks. Additionally, there is a lack of knowledge on the format in which wire navigation training should be deployed.

QUESTIONS/PURPOSES

(1) Which training methods (didactic content, deliberate practice, or proficiency-based practice) lead to the greatest improvement in performing a wire navigation task? (2) Does a resident's performance using a wire navigation simulator correlate with his or her performance on a higher-level simulation task in a mock OR involving a C-arm, a radiopaque femur model, and a large soft tissue surrogate surrounding the femur?

METHODS

Fifty-five residents from four different medical centers participated in this study over the course of 2 years. The residents were divided into three groups: traditional training (included first-year residents from the University of Iowa, University of Minnesota, and the Mayo Clinic), deliberate practice (included first-year residents from the University of Nebraska and the University of Minnesota), and proficiency training (included first-year residents from the University of Minnesota and the Mayo Clinic). Residents in each group received a didactic introduction covering the task of placing a wire to treat an intertrochanteric fracture, and this was considered traditional training. Deliberate practice involved training on a radiation-free simulator that provided specific feedback throughout the practice sessions. Proficiency training used the same simulator to train on specific components of wire navigation, like finding the correct starting point, to proficiency before moving to assessment. The wire navigation simulator uses a camera system to track the wire and provide computer-generated fluoroscopy. After training, task performance was assessed in a mock OR. Residents from each group were assessed in the mock OR based on their use of fluoroscopy, total time, and tip-apex distance. Correlation analysis was performed to examine the relationship between resident performance on the simulator and in the mock OR.

RESULTS

Residents in the two simulation-based training groups had a lower tip-apex distance than those in the traditional training group (didactic training tip-apex distance: 24 ± 7 mm, 95% CI, 20-27; deliberate practice tip-apex distance: 16 ± 5 mm, 95% CI, 13-19, p = 0.001; proficiency training tip-apex distance: 15 ± 4 mm, 95% CI, 13-18, p < 0.001). Residents in the proficiency training group used more images than those in the other groups (didactic training: 22 ± 12 images, p = 0.041; deliberate practice: 19 ± 8 images; p = 0.012, proficiency training: 31 ± 14 images). In the two simulation-based training groups, resident performance on the simulator, that is, tip-apex distance, image use, and overall time, was correlated with performance in the mock OR (r-square = 0.15 [p = 0.030], 0.61 [p < 0.001], and 0.43 [p < 0.001], respectively).

CONCLUSIONS

As residency programs are designing their curriculum to train wire navigation skills, emphasis should be placed on providing an environment that allows for deliberate practice with immediate feedback about their performance. Simulators such as the one presented in this study offer a safe environment for residents to learn this key skill.

LEVEL OF EVIDENCE

Level II, therapeutic study.

Assessment of Middle Ear Anatomy Teaching Methodologies Using Microscopy versus Endoscopy: A Randomized Comparative Study

Teaching methodologies for the anatomy of the middle ear have not been investigated greatly due to the middle ear's highly complex structure and hidden location inside of the temporal bone. The aim of this randomized study was to quantitatively compare the suitability of using microscope- and endoscope-based methods for teaching the anatomy of the middle ear. We hypothesize that the endoscopic approach will be more efficient compared to the microscopic approach. To answer the study questions, 33 sixth-year medical students, residents and otorhinolaryngology specialists were randomized either into the endoscopy or the microscopy group. Their anatomical knowledge was assessed using a structured anatomical knowledge test before and after each session. Each participant received tutoring on a human cadaveric specimen using one of the two methods. They then performed a hands-on dissection. After 2-4 weeks, the same educational curriculum was repeated using the other technique. The mean gains in anatomical knowledge for the specialists, residents, and medical students were +19.0%, +34.6%, and +23.4%, respectively. Multivariate analyses identified a statistically significant increase in performance for the endoscopic method compared to the microscopic technique (P < 0.001). For the recall of anatomical structures during dissection, the endoscopic method outperformed the microscopic technique independently of the randomization or the prior training level of the attendees (P < 0.001). In conclusion, the endoscopic approach to middle ear anatomy education is associated to an improved gain in knowledge as compared to the microscopic approach. The participants subjectively preferred the endoscope for educational purposes.

Perfecting practice: a protocol for assessing simulation-based mastery learning and deliberate practice versus self-guided practice for bougie-assisted cricothyroidotomy performance

BACKGROUND

Simulation-based medical education (SBME) is a cornerstone for procedural skill training in residency education. Multiple studies have concluded that SBME is highly effective, superior to traditional clinical education, and translates to improved patient outcomes. Additionally it is widely accepted that mastery learning, which comprises deliberate practice, is essential for expert level performance for routine skills; however, given that highly structured practice is more time and resource-intensive, it is important to assess its value for the acquisition of rarely performed technical skills. The bougie-assisted cricothyroidotomy (BAC), a rarely performed, lifesaving procedure, is an ideal skill for evaluating the utility of highly structured practice as it is relevant across many acute care specialties and rare - making it unlikely for learners to have had significant previous training or clinical experience. The purpose of this study is to compare a modified mastery learning approach with deliberate practice versus self-guided practice on technical skill performance using a bougie-assisted cricothyroidotomy model.

METHODS

A multi-centre, randomized study will be conducted at four Canadian and one American residency programs with 160 residents assigned to either mastery learning and deliberate practice (ML + DP), or self-guided practice for BAC. Skill performance, using a global rating scale, will be assessed before, immediately after practice, and 6 months later. The two groups will be compared to assess whether the type of practice impacts performance and skill retention.

DISCUSSION

Mastery learning coupled with deliberate practice provides systematic and focused feedback during skill acquisition. However, it is resource-intensive and its efficacy is not fully defined. This multi-centre study will provide generalizable data about the utility of highly structured practice for technical skill acquisition of a rare, lifesaving procedure within postgraduate medical education. Study findings will guide educators in the selection of an optimal training strategy, addressing both short and long term performance.

What did they do next? A survey of dentists who have passed the Overseas Registration Examination of the General Dental Council

Aims To establish what work a sample of Overseas Registration Examination (ORE) registrants were undertaking and understand what had facilitated or impeded them from finding suitable employment as dentists. Method An online questionnaire, consisting of both closed and open questions, was used to capture data from a population of 1,106 former ORE candidates who passed the examination between 2009 and 2014 and were registered by the General Dental Council (GDC). The data were analysed and presented in the form of tables, figures and a presentation of the major themes that emerged from the responses. Results There was a 42% response rate. Seventy-one percent of respondents were employed as dentists in the UK, with the majority providing a mixture of private and NHS patient treatment. Most who were not working as dentists were actively seeking training places. Additional themes that were identified included: the availability of Dental Foundation/Vocational Training places; poor employment practices; perceptions of the strengths and weaknesses of the ORE; and some ideas about the future responsibilities of the GDC. Conclusions This survey has highlighted some difficulties that many ORE registrants face finding suitable work as dentists. Stakeholders should be aware of these challenges.

Neonatal Intubation Competency Assessment Tool: Development and Validation

BACKGROUND

Neonatal tracheal intubation (NTI) is an important clinical skill. Suboptimal performance is associated with patient harm. Simulation training can improve NTI performance. Improving performance requires an objective assessment of competency. Competency assessment tools need strong evidence of validity. We hypothesized that an NTI competency assessment tool with multisource validity evidence could be developed and be used for formative and summative assessment during simulation-based training.

METHODS

An NTI assessment tool was developed based on a literature review. The tool was refined through 2 rounds of a modified Delphi process involving 12 subject-matter experts. The final tool included a 22-item checklist, a global skills assessment, and an entrustable professional activity (EPA) level. The validity of the checklist was assessed by having 4 blinded reviewers score 23 videos of health care providers intubating a neonatal simulator.

RESULTS

The checklist items had good internal consistency (overall α = 0.79). Checklist scores were greater for providers at greater training levels and with more NTI experience. Checklist scores correlated with global skills assessment (ρ = 0.85; P < .05), EPA levels (ρ = 0.87; P < .05), percent glottic exposure (r = 0.59; P < .05), and Cormack-Lehane scores (ρ = 0.95; P < .05). Checklist scores reliably predicted EPA levels.

CONCLUSIONS

We developed an NTI competency assessment tool with multisource validity evidence. The tool was able to discriminate NTI performance based on experience. The tool can be used during simulation-based NTI training to provide formative and summative assessment and can aid with entrustment decisions.

Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53

AIM

Cognitive load theory (CLT) is of increasing interest to health professions education researchers. CLT has intuitive applicability to workplace settings, yet how CLT should inform teaching, learning, and research in health professions workplaces is unclear.

METHOD

To map the existing literature, we performed a scoping review of studies involving cognitive load, mental effort and/or mental workload in professional workplace settings within and outside of the health professions. We included actual and simulated workplaces and workplace tasks.

RESULT

Searching eight databases, we identified 4571 citations, of which 116 met inclusion criteria. Studies were most often quantitative. Methods to measure cognitive load included psychometric, physiologic, and secondary task approaches. Few covariates of cognitive load or performance were studied. Overall cognitive load and intrinsic load were consistently negatively associated with the level of experience and performance. Studies consistently found distractions and other aspects of workplace environments as contributing to extraneous load. Studies outside the health professions documented similar findings to those within the health professions, supporting relevance of CLT to workplace learning.

CONCLUSION

The authors discuss implications for workplace teaching, curricular design, learning environment, and metacognition. To advance workplace learning, the authors suggest future CLT research should address higher-level questions and integrate other learning frameworks.

Impact of a dedicated nursing team on central line-related complications in neonatal intensive care unit

Aim: To evaluate the impact of a dedicated nursing team on central line insertion success and catheter-related complications.Methods: Five nurses were trained in central line insertion and maintenance practices and replaced a team primarily comprised of neonatal-perinatal medicine fellows. A prospective observational cohort study with pre/post-intervention analysis was designed to compare certain aspects of central line insertion and related complications between the two models.Results: Six hundred and twenty peripherally inserted central catheters were attempted preintervention (period 1) in 325 infants, and 630 were attempted in 406 infants postintervention (period 2). Successful central line placement on the first attempt increased significantly from 56.6% in period 1-71.4% in period 2 (p < .001), and needle sticks per attempt decreased (3.5 versus 3; p = .03). All central line-related complications decreased from 12.8 per 1000 line days in period 1 to 5.5 in period 2 (rate ratio = 0.40; 95% confidence interval: 0.29, 0.65). The most significant reduction was noted in phlebitis (4.9-0.5 per 1000 line days; rate ratio = 0.10; 95% confidence interval: 0.03, 0.30)Conclusion: Implementation of a dedicated nursing-based central line team, skilled in insertion and certain aspects of catheter maintenance, significantly improved insertion rates and reduced line-related complications.

Trainee colonoscopy quality is influenced by the independent and unobserved performance characteristics of supervising physicians

Background  Endoscopy training remains an apprenticeship, and the characteristics that facilitate transfer of high quality procedural skills from role models to trainees are unknown. We sought to determine whether unobserved supervisor performance influences the quality of colonoscopy performed by trainees, by studying how supervisors perform alone and how trainees perform while under those same supervisors. Methods  This was a retrospective cross-sectional study conducted among ambulatory adults ≥ 50 years old who underwent colonoscopy for cancer screening or polyp surveillance from 2006 to 2015 at one academic medical center. The primary exposures were the colonoscopy withdrawal time (WT) and adenoma detection rate (ADR) of supervisors while performing colonoscopies alone. The primary outcomes were the WT and ADR of trainees performing colonoscopies under supervision. Results  Data were included from 22 attending gastroenterologist supervisors, 56 gastroenterology fellow trainees, and 2777 adults undergoing 3094 colonoscopy procedures. Among all supervised colonoscopies, mean trainee WT was 12.7 minutes (SD 4.9) and trainee ADR was 33.5 %. The trainee WT was 0.42 minutes longer (standard error = 0.16, P  = 0.01) per minute increase in supervisor WT. Similarly, trainee ADR was higher under a high ADR supervisor, and the odds ratio of high compared to low supervisor ADR category was 1.28 (95 %CI 1.01 - 1.62, P  = 0.04) after adjusting for other factors. Conclusions  The unobserved performance characteristics of supervising endoscopists may influence the quality of colonoscopy performed by trainees.

A Real Human Umbilical Cord Simulator Model for Emergency Umbilical Venous Catheter Placement Training

Emergency umbilical venous catheter placement is a critical procedure during newborn resuscitation. Providing training in this high-acuity and low-frequency procedure is important to optimize the skills of newborn resuscitation teams. Available simulators use simulated umbilical cords which are lower in fidelity than real human umbilical cords and may not provide optimal training. This technical report describes the creation and use of a real human umbilical cord simulator model for emergency umbilical venous catheter placement training. This low-cost model provides learners the opportunity to experience placing an emergency umbilical venous catheter in a real umbilical cord, providing a more realistic training model than currently available commercial simulators.

Pretest Scores Uniquely Predict 1-Year-Delayed Performance in a Simulation-Based Mastery Course for Central Line Insertion

INTRODUCTION

Within simulation-based mastery learning (SBML) courses, there is inconsistent inclusion of learner pretesting, which requires considerable resources and is contrary to popular instructional frameworks. However, it may have several benefits, including its direct benefit as a form of deliberate practice and its facilitation of more learner-specific subsequent deliberate practice. We consider an unexplored potential benefit of pretesting: its ability to predict variable long-term learner performance.

METHODS

Twenty-seven residents completed an SBML course in central line insertion. Residents were tested on simulated central line insertion precourse, immediately postcourse, and after between 64 and 82 weeks. We analyzed pretest scores' prediction of delayed test scores, above and beyond prediction by program year, line insertion experiences in the interim, and immediate posttest scores.

RESULTS

Pretest scores related strongly to delayed test scores (r = 0.59, P = 0.01; disattenuated ρ = 0.75). The number of independent central lines inserted also related to year-delayed test scores (r = 0.44, P = 0.02); other predictors did not discernibly relate. In a regression model jointly predicting delayed test scores, pretest was a significant predictor (β = 0.487, P = 0.011); number of independent insertions was not (β = 0.234, P = 0.198).

CONCLUSIONS

This study suggests that pretests can play a major role in predicting learner variance in learning gains from SBML courses, thus facilitating more targeted refresher training. It also exposes a risk in SBML courses that learners who meet immediate mastery standards may be incorrectly assumed to have equal long-term learning gains.

Building a Community of Practice for Researchers: The International Network for Simulation-Based Pediatric Innovation, Research and Education

STATEMENT

The scope and breadth of simulation-based research is growing rapidly; however, few mechanisms exist for conducting multicenter, collaborative research. Failure to foster collaborative research efforts is a critical gap that lies in the path of advancing healthcare simulation. The 2017 Research Summit hosted by the Society for Simulation in Healthcare highlighted how simulation-based research networks can produce studies that positively impact the delivery of healthcare. In 2011, the International Network for Simulation-based Pediatric Innovation, Research and Education (INSPIRE) was formed to facilitate multicenter, collaborative simulation-based research with the aim of developing a community of practice for simulation researchers. Since its formation, the network has successfully completed and published numerous collaborative research projects. In this article, we describe INSPIRE's history, structure, and internal processes with the goal of highlighting the community of practice model for other groups seeking to form a simulation-based research network.

A Three-dimensional Printed Low-cost Anterior Shoulder Dislocation Model for Ultrasound-guided Injection Training

Anterior shoulder dislocations are the most common, large joint dislocations that present to the emergency department (ED). Numerous studies support the use of intraarticular local anesthetic injections for the safe, effective, and time-saving reduction of these dislocations. Simulation training is an alternative and effective method for training compared to bedside learning. There are no commercially available ultrasound-compatible shoulder dislocation models. We utilized a three-dimensional (3D) printer to print a model that allows the visualization of the ultrasound anatomy (sonoanatomy) of an anterior shoulder dislocation. We utilized an open-source file of a shoulder, available from embodi3D® (Bellevue, WA, US). After approximating the relative orientation of the humerus to the glenoid fossa in an anterior dislocation, the humerus and scapula model was printed with an Ultimaker-2 Extended+ 3D® (Ultimaker, Cambridge, MA, US) printer using polylactic acid filaments. A 3D model of the external shoulder anatomy of a live human model was then created using Structure Sensor®(Occipital, San Francisco, CA, US), a 3D scanner. We aligned the printed dislocation model of the humerus and scapula within the resultant external shoulder mold. A pourable ballistics gel solution was used to create the final shoulder phantom. The use of simulation in medicine is widespread and growing, given the restrictions on work hours and a renewed focus on patient safety. The adage of "see one, do one, teach one" is being replaced by deliberate practice. Simulation allows such training to occur in a safe teaching environment. The ballistic gel and polylactic acid structure effectively reproduced the sonoanatomy of an anterior shoulder dislocation. The 3D printed model was effective for practicing an in-plane ultrasound-guided intraarticular joint injection. 3D printing is effective in producing a low-cost, ultrasound-capable model simulating an anterior shoulder dislocation. Future research will determine whether provider confidence and the use of intraarticular anesthesia for the management of shoulder dislocations will improve after utilizing this model.

Preferred learning modalities and practice for critical skills: a global survey of paediatric emergency medicine clinicians

Objective

To describe senior paediatric emergency clinician perspectives on the optimal frequency of and preferred modalities for practising critical paediatric procedures.

Methods

Multicentre multicountry cross-sectional survey of senior paediatric emergency clinicians working in 96 EDs affiliated with the Pediatric Emergency Research Network.

Results

1332/2446 (54%) clinicians provided information on suggested frequency of practice and preferred learning modalities for 18 critical procedures. Yearly practice was recommended for six procedures (bag valve mask ventilation, cardiopulmonary resuscitation (CPR), endotracheal intubation, laryngeal mask airway insertion, defibrillation/direct current (DC) cardioversion and intraosseous needle insertion) by at least 80% of respondents. 16 procedures were recommended for yearly practice by at least 50% of respondents. Two procedures (venous cutdown and ED thoracotomy) had yearly practice recommended by <40% of respondents. Simulation was the preferred learning modality for CPR, bag valve mask ventilation, DC cardioversion and transcutaneous pacing. Practice in alternative clinical settings (eg, the operating room) was the preferred learning modality for endotracheal intubation and laryngeal mask insertion. Use of models/mannequins for isolated procedural training was the preferred learning modality for all other invasive procedures. Free-text responses suggested the utility of cadaver labs and animal labs for more invasive procedures (thoracotomy, intercostal catheter insertion, open surgical airways, venous cutdown and pericardiocentesis).

Conclusions

Paediatric ED clinicians suggest that most paediatric critical procedures should be practised at least annually. The preferred learning modality depends on the skill practised; alternative clinical settings are thought to be most useful for standard airway manoeuvres, while simulation-based experiential learning is applicable for most other procedures.

Optimal human and system performance during neonatal resuscitation

Performance in the delivery of care to sick neonates in need of resuscitation has long been defined primarily in terms of the extent of the knowledge possessed and hands-on skill demonstrated by physicians and other healthcare professionals. This definition of performance in neonatal resuscitation is limited by its focus solely on the human beings delivering care and a perceived set of the requisite skills to do so. This manuscript will expand the definition of performance to include all of the skill sets that humans must use to resuscitate newborns as well as the often complex systems in which those humans operate while delivering that care. It will also highlight how the principles of human factors and ergonomics can be used to enhance human and system performance during patient care. Finally, it will describe the role of simulation and debriefing in the assessment of human and system performance.

Mock ECHO: A Simulation-Based Medical Education Method

Problem: This study was designed to develop a deeper understanding of the learning and social processes that take place during the simulation-based medical education for practicing providers as part of the Project ECHO® model, known as Mock ECHO training. The ECHO model is utilized to expand access to care of common and complex diseases by supporting the education of primary care providers with an interprofessional team of specialists via videoconferencing networks. Intervention: Mock ECHO trainings are conducted through a train the trainer model targeted at leaders replicating the ECHO model at their organizations. Trainers conduct simulated teleECHO clinics while participants gain skills to improve communication and self-efficacy. Context: Three focus groups, conducted between May 2015 and January 2016 with a total of 26 participants, were deductively analyzed to identify common themes related to simulation-based medical education and interdisciplinary education. Principal themes generated from the analysis included (a) the role of empathy in community development, (b) the value of training tools as guides for learning, (c) Mock ECHO design components to optimize learning, (d) the role of interdisciplinary education to build community and improve care delivery, (e) improving care integration through collaboration, and (f) development of soft skills to facilitate learning. Outcome: Mock ECHO trainings offer clinicians the freedom to learn in a noncritical environment while emphasizing real-time multidirectional feedback and encouraging knowledge and skill transfer. Lessons Learned: The success of the ECHO model depends on training interprofessional healthcare providers in behaviors needed to lead a teleECHO clinic and to collaborate in the educational process. While building a community of practice, Mock ECHO provides a safe opportunity for a diverse group of clinician experts to practice learned skills and receive feedback from coparticipants and facilitators.

Paracentesis Simulation: A Comprehensive Approach to Procedural Education

INTRODUCTION

Structured procedural education and assessment of competency are growing needs for residency and fellowship programs. Simulation is a useful way to learn, experience, and practice procedural skills with competence. Paracentesis is a common procedure encountered in internal medicine. This educational resource for paracentesis education includes didactics, cases, and assessments to address cognitive skills, a simulation experience to address psychomotor procedural skills, and an entrustment-based assessment tool.

METHODS

Prior to the simulation, learners completed preprocedural didactics and self-assessments. Utilizing a paracentesis trainer, ultrasound, and paracentesis kit, the case of a 46-year-old male with ascites in need of a paracentesis was presented. During the simulation, learners initially performed a paracentesis step by step, with assistance and feedback from the case instructor. This was immediately followed by paracentesis without assistance, where the instructor evaluated the learners with an assessment tool encompassing a procedural checklist, global skill assessment scale, and entrustment scale. Afterwards, learners completed case-based reviews and returned to the simulation lab several months later to repeat an unassisted paracentesis.

RESULTS

The curriculum was used with internal medicine and medicine-pediatric residents of all training levels. To date, over 120 residents have completed the curriculum. Residents reported an increase in self-confidence and competence using ultrasound to identify ascites and performing a paracentesis. Learners provided positive feedback.

DISCUSSION

This curriculum offers the opportunity for both cognitive and psychomotor paracentesis education in a low-risk simulation environment. The comprehensive strategy with didactics, cases, and multiple simulations is designed to promote knowledge and skill retention.

The amount of supervision trainees receive during neonatal resuscitation is variable and often dependent on subjective criteria

OBJECTIVE

Measure variation in delivery room supervision provided by neonatologists using hypothetical scenarios and determine the factors used to guide entrustment decisions.

STUDY DESIGN

A survey was distributed to members of the American Academy of Pediatrics Section on Perinatal Pediatrics. Neonatologists were presented with various newborn resuscitation scenarios and asked to choose the level of supervision they thought appropriate and grade factors on their importance in making entrustment decisions.

RESULTS

There was significant variation in supervision neonatologists deemed necessary for most scenarios (deviation from the mode 0.36-0.69). Post-graduate year of training and environmental circumstances influence the amount of autonomy neonatologists grant trainees. Few neonatologists have objective assessment of a trainees' competence in neonatal resuscitation available to them and most never document how the trainee performed.

CONCLUSION

Delivery room supervision is often determined by subjective evaluation of trainees' competence and may not provide a level of supervision congruent with their capability.

Analysis of midwifery teachers' approach to identifying student midwives with poor clinical reasoning skills

OBJECTIVE

To analyse midwifery teachers' pedagogic approaches to remediation for student midwives with poor clinical reasoning skills.

METHODS

A mixed-methods approach using a questionnaire and in-depth interviews.

SETTING

Midwifery schools in France.

PARTICIPANTS

Teachers in French midwifery schools.

MEASUREMENTS AND FINDINGS

A quarter of the teachers had no training in clinical reasoning. Midwifery teachers mainly identified students' clinical reasoning difficulties during clinical supervision with a non-validated tool. All teachers detected the warning signs and the main obstacles identifying student midwives with poor clinical reasoning skills along with some identifying factors favouring those difficulties. However, the remedial strategies proposed were mainly reassessment without personalised corrective learning activities.

KEY CONCLUSIONS

The approach to identifying student midwives with poor clinical reasoning skills was incomplete and remedial strategies were stereotypical.

IMPLICATIONS FOR PRACTICE

Midwifery teachers should be trained to recognise their students' clinical reasoning issues to identify them early, using all types of learning activities. Remedial strategies should be implemented promptly, adapted to each student and foster the transfer of learning.

Clinical Cadavers as a Simulation Resource for Procedural Learning

"See one, do one, teach one" remains an unofficial, unsanctioned framework for procedural skill learning in medicine. Appropriately, medical educators have sought alternative simulation venues for students to safely learn their craft. With the end goal of ensuring competence, educational programming will require the use of valid simulation with appropriate fidelity. While cadavers have been used for teaching anatomy for hundreds of years, more recently they are being repurposed as a "high-fidelity" procedural skill learning simulation resource. Newly deceased, previously frozen, and soft-preserved cadavers, such as those used in Baltimore and Halifax, produce clinical cadavers with high physical and functional fidelity that can serve as simulators for performing many high-acuity procedures for which there is otherwise limited clinical or simulation opportunities to practice. While access and cost may limit the use of cadavers for simulation, there are opportunities for sharing resources to provide an innovative procedural learning experience using the oldest of medical simulation assets, the human body.

Twelve tips for efficient procedural simulation

Procedural simulation (PS) is increasingly being used worldwide in healthcare for training caregivers in psychomotor competencies. It has been demonstrated to improve learners' confidence and competence in technical procedures, with consequent positive impacts on patient outcomes and safety. Several frameworks can guide healthcare educators in using PS as an educational tool. However, no theory-informed practical framework exists to guide them in including PS in their training programs. We present 12 practical tips for efficient PS training that translates educational concepts from theory to practice, based on the existing literature. In doing this, we aim to help healthcare educators to adequately incorporate and use PS both for optimal learning and for transfer into professional practice.

Improving neonatal intubation safety: A journey of a thousand miles

Neonatal intubation is one of the most common procedures performed by neonatologists, however, the procedure is difficult and high risk. Neonates who endure the procedure often experience adverse events, including bradycardia and severe oxygen desaturations. Because of low first attempt success rates, neonates are often subjected to multiple intubation attempts before the endotracheal tube is successfully placed. These factors conspire to make intubation one of the most dangerous procedures in neonatal medicine. In this commentary we review key elements in the journey to improve neonatal intubation safety. We begin with a review of intubation success rates and complications. Then, we discuss the importance of intubation training. Next, we examine quality improvement efforts and patient safety research to improve neonatal intubation safety. Finally, we evaluate new tools which may improve success rates, and decrease complications during neonatal intubation.

Downward Trend in Pediatric Resident Laryngoscopy Participation in PICUs

OBJECTIVES

As of July 2013, pediatric resident trainee guidelines in the United States no longer require proficiency in nonneonatal tracheal intubation. We hypothesized that laryngoscopy by pediatric residents has decreased over time, with a more pronounced decrease after this guideline change.

DESIGN

Prospective cohort study.

SETTING

Twenty-five PICUs at various children's hospitals across the United States.

PATIENTS

Tracheal intubations performed in PICUs from July 2010 to June 2016 in the multicenter tracheal intubation database (National Emergency Airway Registry for Children).

INTERVENTION

None.

MEASUREMENTS AND MAIN RESULTS

Prospective cohort study in which all primary tracheal intubations occurring in the United States from July 2010 to June 2016 in the multicenter tracheal intubation database (National Emergency Airway Registry for Children) were analyzed. Participating PICU leaders were also asked to describe their local airway management training for residents. Resident participation trends over time, stratified by presence of a Pediatric Critical Care Medicine fellowship and airway training curriculum for residents, were described. A total of 9,203 tracheal intubations from 25 PICUs were reported. Pediatric residents participated in 16% of tracheal intubations as first laryngoscopists: 14% in PICUs with a Pediatric Critical Care Medicine fellowship and 34% in PICUs without one (p < 0.001). Resident participation decreased significantly over time (3.4% per year; p < 0.001). The decrease was significant in ICUs with a Pediatric Critical Care Medicine fellowship (p < 0.001) but not in ICUs without one (p = 0.73). After adjusting for site-level clustering, patient characteristics, and Pediatric Critical Care Medicine fellowship presence, the Accreditation Council for Graduate Medical Education guideline change was not associated with lower participation by residents (odds ratio, 0.86; 95% CI, 0.59-1.24; p = 0.43). The downward trend of resident participation was similar regardless of the presence of an airway curriculum for residents.

CONCLUSION

Laryngoscopy by pediatric residents has substantially decreased over time. This downward trend was not associated with the 2013 Accreditation Council for Graduate Medical Education change in residency requirements.

More Learners, Finite Resources, and the Changing Landscape of Procedural Training at the Bedside

There is growing competition for nonoperative, procedural training in teaching hospitals, due to an increased number of individuals seeking to learn procedures from a finite number of appropriate teaching cases. Procedural training is required by students, postgraduate learners, and practicing providers who must maintain their skills. These learner groups are growing in size as the number of medical schools increases and advance practice providers expand their skills to include complex procedures. These various learner needs occur against a background of advancing therapeutic techniques that improve patient care but also act to reduce the overall numbers of procedures available to learners. This article is a brief review of these and other challenges that are arising for program directors, medical school leaders, and hospital administrators who must act to ensure that all of their providers acquire and maintain competency in a wide array of procedural skills. The authors conclude their review with several recommendations to better address procedural training in this new era of learner competition. These include a call for innovative clinical rotations deliberately designed to improve procedural training, access to training opportunities at new clinical sites acquired in health system expansions, targeted faculty development for those who teach procedures, reporting of competition for bedside procedures by trainees, more frequent review of resident procedure and case logs, and the creation of an institutional oversight committee for procedural training.

Does medical students’ gender affect their clinical learning of gynaecological examination? A retrospective cohort study

Introduction

Medical graduates should be competent in gynaecological examination as well as associated skills including speculum use and swabbing. Male and female medical students may have different opportunities to practise these skills in clinical environments, potentially impacting on confidence and competence. This study explores this further via reviewing students’ learning experience in genitourinary medicine (GUM) and obstetrics and gynaecology (O&G) clinics.

Methods

Cross-sectional study of 759 year 4 and year 5 University of Birmingham medical students via an online questionnaire. This explored degree of participation, impact of gender and self-reported confidence.

Results

Overall response rate was 31% (233/759). Students of either gender who observed an examination being performed by a clinician were more likely to perform the same examination. Female students reported more opportunities to practise gynaecological examination and associated skills. Female students were more likely to be granted consent to perform speculum examinations, vaginal swabbing and vaginal bimanual examinations. Sixty-five per cent of male students felt that their gender affected their learning experience with female patients. Despite this, there was no significant difference in self-reported confidence level in performing gynaecological examinations between genders at the end of placement.

Conclusion

The majority of male students perceived that their gender impacted their clinical experience in O&G and GUM. Self-reported confidence levels were unaffected, which could reflect varying approaches to competence between genders. The link between observing examinations and subsequent opportunities to practise is key. This could demonstrate students developing rapport and trust with patients, and clinicians’ roles as gatekeepers.

Twelve tips for teaching brief motivational interviewing to medical students

BACKGROUND

Shifting from paternalistic to patient-centred doctor-patient relationships has seen a growing number of medical programs incorporate brief motivational interviewing training in their curriculum. Some medical educators, however, are unsure of precisely what, when, and how to incorporate such training.

AIMS

This article provides educators with 12 tips for teaching brief motivational interviewing to medical students, premised on evidence-based pedagogy.

METHODS

Tips were drawn from the literature and authors' own experiences.

RESULTS

The 12 tips are: (1) Set clear learning objectives, (2) Select experienced educators, (3) Provide theoretical perspectives, (4) Share the evidence base, (5) Outline the "spirit", principles, and sequence, (6) Show students what it looks like, (7) Give students a scaffold to follow, (8) Provide opportunities for skill practice, (9) Involve clinical students in teaching, (10) Use varied formative and summative assessments, (11) Integrate and maintain, and (12) Reflect and evaluate.

CONCLUSIONS

We describe what to include and why, and outline when and how to teach the essential components of brief motivational interviewing knowledge and skills in a medical curriculum.

Randomized trial of high- and low-fidelity simulation to teach intrauterine contraception placement

BACKGROUND

High-fidelity simulation creates conditions that resemble real circumstances, and can help teach procedures such as intrauterine contraception placement. Its impact on skill retention has not been studied.

OBJECTIVE

We sought to evaluate novice learners' skills, attitudes, and knowledge on placement of intrauterine contraception when trained using a high-fidelity commercially available simulator compared with a low-fidelity simulator.

STUDY DESIGN

We recruited senior nurse practitioner students and interns in obstetrics and gynecology and family medicine inexperienced with intrauterine contraception placement. In this unblinded, randomized controlled trial, participants were assigned to practice within a high-fidelity simulator group or a coasterlike model group. We evaluated intrauterine contraception placement skills, self-perceived comfort and competence, and knowledge before and after simulation, as well as at 3 months. Our primary outcome was the change in scores for intrauterine contraception placement skills before and after practice. Assuming a standard deviation of 15 points, we needed 10 participants per group to detect a 20-point difference in scores with 80% power.

RESULTS

From June through July 2014, 60 participants enrolled; 59 completed the initial study visit and 1 withdrew. In all, 48 (80%) completed the second study visit at 3 months. Demographic characteristics were similar for the randomization groups. We observed an improvement in intrauterine contraception placement skills for both groups following practice on simulators (P < .01); the proportion that improved was similar (20% for the high-fidelity simulator group and 15% for the coaster group, P = .55). Increases in self-perceived comfort and competence with placing copper, levonorgestrel 52-mg, and levonorgestrel 13.5-mg devices were similar (all P ≥ .11). Knowledge assessment scores were comparable between the 2 groups postsimulation (73% for the high-fidelity simulator group and 80% for the coaster group, P = .29) and at 3 months (87% for both groups, P = 1.0).

CONCLUSION

Trainees' knowledge, intrauterine contraception placement skills, and self-perceived comfort and competence were comparable whether they used high- or low-fidelity simulators.

Credentialing of Hospitalists in Ultrasound-Guided Bedside Procedures: A Position Statement of the Society of Hospital Medicine

Ultrasound guidance is used increasingly to perform the following 6 bedside procedures that are core competencies of hospitalists: abdominal paracentesis, arterial catheter placement, arthrocentesis, central venous catheter placement, lumbar puncture, and thoracentesis. Yet most hospitalists have not been certified to perform these procedures, whether using ultrasound guidance or not, by specialty boards or other institutions extramural to their own hospitals. Instead, hospital privileging committees often ask hospitalist group leaders to make ad hoc intramural certification assessments as part of credentialing. Given variation in training and experience, such assessments are not straightforward "sign offs." We thus convened a panel of experts to conduct a systematic review to provide recommendations for credentialing hospitalist physicians in ultrasound guidance of these 6 bedside procedures. Pathways for initial and ongoing credentialing are proposed. A guiding principle of both is that certification assessments for basic competence are best made through direct observation of performance on actual patients.

When Old Habits Train a New Generation: Findings From a National Survey of Internal Medicine Program Directors on Procedural Training

Resident physicians routinely perform bedside procedures that pose substantial risk to patients. However, no standard programmatic approach to supervision and procedural competency assessment among residents currently exists. The authors performed a national survey of internal medicine (IM) program directors to examine procedural assessment and supervision practices of IM residency programs. Procedures chosen were those commonly performed by medicine residents at the bedside. Of the 368 IM programs, 226 (61%) completed the survey. Programs reported the predominant method of training as 171 (74%) apprenticeship and 106 (46%) as module based. The majority of programs used direct observation to attest to competence, with 55% to 62% relying on credentialed residents. Most programs also relied on a minimum number of procedures to determine competence (64%-88%), 72% of which reported 5 procedures (a lapsed historical standard). This national survey demonstrates that procedural assessment practices for IM residents are insufficiently robust and may put patients at undue risk.

Learner, Patient, and Supervisor Features Are Associated With Different Types of Cognitive Load During Procedural Skills Training: Implications for Teaching and Instructional Design

PURPOSE

Cognitive load theory, focusing on limits of the working memory, is relevant to medical education; however, factors associated with cognitive load during procedural skills training are not well characterized. The authors sought to determine how features of learners, patients/tasks, settings, and supervisors were associated with three types of cognitive load among learners performing a specific procedure, colonoscopy, to identify implications for procedural teaching.

METHOD

Data were collected through an electronically administered survey sent to 1,061 U.S. gastroenterology fellows during the 2014-2015 academic year; 477 (45.0%) participated. Participants completed the survey immediately following a colonoscopy. Using multivariable linear regression analyses, the authors identified sets of features associated with intrinsic, extraneous, and germane loads.

RESULTS

Features associated with intrinsic load included learners (prior experience and year in training negatively associated, fatigue positively associated) and patient/tasks (procedural complexity positively associated, better patient tolerance negatively associated). Features associated with extraneous load included learners (fatigue positively associated), setting (queue order positively associated), and supervisors (supervisor engagement and confidence negatively associated). Only one feature, supervisor engagement, was (positively) associated with germane load.

CONCLUSIONS

These data support practical recommendations for teaching procedural skills through the lens of cognitive load theory. To optimize intrinsic load, level of experience and competence of learners should be balanced with procedural complexity; part-task approaches and scaffolding may be beneficial. To reduce extraneous load, teachers should remain engaged, and factors within the procedural setting that may interfere with learning should be minimized. To optimize germane load, teachers should remain engaged.

Core Competencies or a Competent Core? A Scoping Review and Realist Synthesis of Invasive Bedside Procedural Skills Training in Internal Medicine

PURPOSE

Invasive bedside procedures are core competencies for internal medicine, yet no formal training guidelines exist. The authors conducted a scoping review and realist synthesis to characterize current training for lumbar puncture, arthrocentesis, paracentesis, thoracentesis, and central venous catheterization. They aimed to collate how educators justify using specific interventions, establish which interventions have the best evidence, and offer directions for future research and training.

METHOD

The authors systematically searched Medline, Embase, the Cochrane Library, and ERIC through April 2015. Studies were screened in three phases; all reviews were performed independently and in duplicate. The authors extracted information on learner and patient demographics, study design and methodological quality, and details of training interventions and measured outcomes. A three-step realist synthesis was performed to synthesize findings on each study's context, mechanism, and outcome, and to identify a foundational training model.

RESULTS

From an initial 6,671 studies, 149 studies were further reduced to 67 (45%) reporting sufficient information for realist synthesis. Analysis yielded four types of procedural skills training interventions. There was relative consistency across contexts and significant differences in mechanisms and outcomes across the four intervention types. The medical procedural service was identified as an adaptable foundational training model.

CONCLUSIONS

The observed heterogeneity in procedural skills training implies that programs are not consistently developing residents who are competent in core procedures. The findings suggest that researchers in education and quality improvement will need to collaborate to design training that develops a "competent core" of proceduralists using simulation and clinical rotations.

Framework for Teaching Psychomotor and Procedural Skills in Nursing

The development of psychomotor and procedural skills requires opportunities for repetitive practice combined with specific, informational feedback from the teacher, another expert, or simulator to correct performance errors. Practice enables learners to refine skills and progress through the phases of motor learning: cognitive, associative, and autonomous. Practice should be spaced over time, can occur in dyads, and can rapidly cycle between practicing and receiving feedback and coaching until skills are mastered. The purpose of this article is to examine psychomotor skill learning in nursing and to suggest strategies for nurse educators in teaching motor and procedural skills in nursing programs, as well as in clinical settings. J Contin Educ Nurs. 2016;47(6):278-282.

A Simulation-based Quality Improvement Approach to Improve Pediatric Resident Competency with Required Procedures

INTRODUCTION

Pediatric residents report a lack of confidence and competence with procedural skills at graduation. Training programs could benefit from improved approaches to target these needs. Using the Institute for Healthcare Improvement (IHI) Model for Improvement and three Plan-Do-Study-Act (PDSA) cycles, we examined the impact of a procedure simulation boot camp on self-reported procedural confidence and competence as well as the longitudinal impacts of these sequential interventions on Accreditation Council for Graduate Medical Education (ACGME) Graduating Resident Survey (GRS) results.

METHODS

Three rapid cycle interventions were performed in successive academic years. The interventions included 1) increased awareness of available procedural experiences, 2) institution of procedural educational conferences, and 3) implementation of a senior resident procedure boot camp. Senior resident self-reported procedural confidence was measured before and after the boot camp. Procedural competence was measured using the ACGME GRS.  Results: Thirty-two of 34 senior residents (94%) completed the 2016 ACGME GRS, similar to the response rates of 2014 (92%) and 2015 (94%), and 30 of 34 third-year residents participated in the procedure boot camp (88%). Resident confidence and competence with procedural skills improved after the institution of the quality improvement intervention. ACGME GRS-reported competency increased in bag and mask ventilation (77% to 94%), neonatal endotracheal intubation (39% to 69%), peripheral IV placement (10% to 50%), and umbilical catheter placement (35% to 53%).

CONCLUSION

A quality improvement intervention with three rapid PDSA cycles was successful in improving senior pediatric resident confidence and competence with ACGME required procedural skills.

Real vs simulated umbilical cords for emergency umbilical catheterization training: a randomized crossover study

OBJECTIVE

To measure performance, fidelity and preference of two emergency umbilical vessel catheter (eUVC) simulation models.

STUDY DESIGN

A randomized crossover trial of senior pediatric residents randomized to place an eUVC first using a real cord (RC) or simulated cord (SC), and then place an eUVC using the other model. The eUVC placement times were recorded and analyzed. Subjects rated physical and functional fidelity and preference for each model.

RESULTS

The eUVC placement time (mean±s.d. s) was slower in RC vs SC (153 s ±71 vs 88 s ±35, P<0.001), however, there was no difference in eUVC placement time in the group that worked with SC first (115 s ±36 vs 97 s ±35, P=0.161). Physical and functional fidelity of RC were rated higher than SC (P<0.001), and RC were preferred.

CONCLUSION

RC has higher physical and functional fidelity, and are preferred for training by pediatric residents, despite longer placement times.

An evidence-based approach to surgical teaching in ophthalmology

An apprenticeship model has traditionally been used in procedural and surgical teaching. As the pressures of work hours and patient outcome monitoring increase, surgical teachers need a more flexible plan for teaching procedural skills. We attempt to delineate a program of preprocedural, intraprocedural, and postprocedural teaching that can be used in the field of ophthalmology to maximize a resident's skill acquisition in a constructive learning environment. We review the literature on surgical teaching from within ophthalmology as well as other surgical fields and combine this with teaching experience in an ophthalmic surgical training program to produce a collection of procedural teaching guidelines. These guidelines are structured to serve in both individual teaching settings and in curriculum design.

Simulation and Web-based learning increases utilization of Bier block for forearm fracture reduction in the pediatric emergency department

OBJECTIVES

Bier block (BB) is a safe and effective alternative to procedural sedation for analgesia during forearm fracture reductions, yet remains infrequently used in the pediatric emergency department (PED). No standardized methods of BB training have previously been described. The objective of this study was to determine whether a multimodal instructional course increases comfort with BB and translates to increased use of this technique.

METHODS

A novel interdisciplinary simulation and Web-based training course was developed to teach the use of BB for forearm fracture reduction at a tertiary PED. Participants were surveyed pre-/post-training, and at 2 and 6 months regarding their comfort with BB. In parallel, we prospectively assessed the clinical use of BB for children ages 6 to 18 years requiring closed reduction of forearm fractures during the 24-month post-course period.

RESULTS

Course participation included 26 physicians and 12 nurses. Survey response rate was 100%. Course participation increased both comfort (10% pre-training v. 89% post-training, p<0.001) and the willingness to use BB (51% pre-training v. 95% post-training, p<0.001), an effect sustained at 6 months post-course (66% and 92%, respectively, p<0.001 for both). In clinical practice, there were no BBs performed prior to course administration. We observed a consistent and sustained increase in clinical use among the BB-trained physicians, with 37% of all forearm reductions performed using BB at 24 months post-course completion.

CONCLUSIONS

A novel combined simulation and Web-based training course increased comfort and willingness to use BB and was associated with increased use of this technique for forearm fracture reduction in the PED.

Procedural training and assessment of competency utilizing simulation

This review examines the current environment of neonatal procedural learning, describes an updated model of skills training, defines the role of simulation in assessing competency, and discusses potential future directions for simulation-based competency assessment. In order to maximize impact, simulation-based procedural training programs should follow a standardized and evidence-based approach to designing and evaluating educational activities. Simulation can be used to facilitate the evaluation of competency, but must incorporate validated assessment tools to ensure quality and consistency. True competency evaluation cannot be accomplished with simulation alone: competency assessment must also include evaluations of procedural skill during actual clinical care. Future work in this area is needed to measure and track clinically meaningful patient outcomes resulting from simulation-based training, examine the use of simulation to assist physicians undergoing re-entry to practice, and to examine the use of procedural skills simulation as part of a maintenance of competency and life-long learning.

Update on simulation for the Neonatal Resuscitation Program

The goal of the Neonatal Resuscitation Program is to have a trained provider in neonatal resuscitation at every delivery. The Neonatal Resuscitation Program develops its course content on review of the scientific evidence available for the resuscitation of newborns. Just as importantly, the educational structure and delivery of the course are based on evidence and educational theory. Thus, as simulation became a more accepted model in medical education and evidence was developing suggesting benefit of simulation, the Neonatal Resuscitation Program officially added simulation into its courses in 2010. Simulation-based medical education is now an integral part of the Neonatal Resuscitation Program courses both in teaching the psychomotor skills as well as the teamwork skills needed for effective newborn resuscitations. While there is evidence, as in other fields, suggesting that simulation for teaching newborn resuscitation is beneficial whether using high- or low-technology manikins or video-assisted debriefing or not, there are still many unanswered questions as to best practice and patient outcome effects.

Neonatal-perinatal medicine fellow procedural experience and competency determination: results of a national survey

OBJECTIVE

Ensuring that neonatal-perinatal medicine (NPM) fellows attain competency in performing neonatal procedures is a requirement of training-competent neonatologists.

STUDY DESIGN

A survey of NPM fellows was performed to determine the procedural experience of current fellows, investigate techniques used to track procedural experience and examine the methods programs use to verify procedural competency.

RESULTS

One hundred and sixty-three fellows in 57 accredited training programs responded to the survey. Reported number of procedures provide contemporary normative data on procedural experience during training. The majority of fellows reported using an online reporting system to track experience. The most common technique to verify procedural competency was supervised practice until an arbitrary number of procedures had been performed.

CONCLUSIONS

NPM fellow procedural experience increases significantly for most, but not all, procedures duration training. We speculate that supplemental simulation training for rare neonatal procedures would help ensure the competency of graduating NPM fellows. Experience alone is insufficient to verify competency. Further work on the accurate tracking of experience and verification of procedural competency is needed.