Use of the eLogbook in surgical training in the United Kingdom: A nationwide survey and consensus recommendations from the Association of Surgeons in Training

Characterizing individual trainee learning curves in surgical training: Challenges and opportunities

The surgical learning curve is an observable and measurable phenomenon. In the era of competency-based approaches to surgical training, monitoring the trajectory of individual trainee competence attainment could represent a meaningful method of formative and summative assessment. While technology can assist this approach, a number of significant barriers to the implementation of such assessment methods remain, including: accurate data collection, standard setting, and reliable assessment. Translating individual learning curve data into quantifiable case minimum targets in training poses further difficulties, and may not be possible for all procedures, particularly those that are less frequently performed and assessed. In spite of these challenges, significant benefits could be realized through an individualized approach to competency assessment using trainee learning curve data. Tracking competence acquisition against criterion-referenced standards could allow for targeted training and remediation, conforming with modern theories of adult education and empowering trainees to take control of their own learning. Learning curve data could also be used to assess the effects of educational interventions such as simulation-based training on subsequent competence acquisition rates. Ultimately, the individual learning curves of trainees could be used to inform personalised decisions regarding entrustment, credentialing, and certification, allowing training programmes to move beyond minimum operative experience targets as a crude proxy measure of competence.

Electronic logbooks (e-logbooks) for the continuous assessment of medical licentiates and their medical skill development in the low-resource context of Zambia: A mixed-methods study

Background

Countries in sub-Saharan Africa continue to face insufficient health education resources and facilities, as well as a severe shortage of health care professionals. In 2019, the Levy Mwanawasa Medical University (LMMU) in Lusaka was launched to address the shortage of healthcare professionals implementing a decentralized training model utilizing selected regional and district hospitals in Zambia as training sites for various cadres. Decentralization makes it more challenging to monitor the learning process as part of continuous assessment; consequently, adequate approaches are necessary to ensure the quality and quantity of medical skills training. Electronic logbooks (e-logbooks) provide a promising tool for monitoring and evaluation of the medical training process.

Objective

We designed and implemented an e-logbook for Medical Licentiate students based on an existing software system. We evaluated the feasibility of this e-logbook, its acceptability among a cohort of Medical Licentiate students and their mentors, as well as its facilitators and barriers.

Materials and methods

During the course of a five-week-long clinical rotation in a training site in Kabwe, Zambia, two mentors and ten students participated in the pilot study and its evaluation. A mixed-methods approach utilized log-based usage data from the e-logbook web platform and conducted semi-structured in-depth interviews.

Results

Overall, both students and mentors accepted e-logbooks as a means to monitor skills development in this context, indicating that e-logbooks are a feasible tool in this decentralized setting. Feedback pointed out that the design and software-induced terminology of the e-logbook posed usability issues. The complexity and greater time commitment (mentors used a web-based platform instead of an app) limited the e-logbook's potential.

Conclusion

We conclude that there is acceptability of monitoring medical skill development through a tablet-based e-logbook. However, the e-logbook in its current form (based on an existing software system, with limited adaptation possibilities to the local context) was insufficient for the LMMU environment. Given that this was attributable to design flaws rather than technology issues or rejection of the e-logbook as a quality assessment tool in and of itself, we propose that the e-logbook be implemented in a co-design approach to better reflect the needs of students and mentors.

Defining operative experience targets in surgical training: A systematic review

BACKGROUND

The surgical learning curve is an observable and measurable phenomenon. Operative experience targets are well established as a proxy measure for operative competence in surgical training across jurisdictions. The aim of this study was to critique the available evidence regarding the relationship between operative experience in surgical training and trainee competence.

METHODS

A systematic review of the PubMed, Embase, Web of Science, and Cochrane library databases was conducted in accordance with the Preferred Items for Systematic Reviews and Meta-Analyses guidelines. Articles were sought that defined the relationship between procedural volume in surgical training and trainee competence, proficiency, or mastery. The educational impact of included studies was evaluated using a modified Kirkpatrick model.

RESULTS

Of 3,672 records identified on database searching, 30 papers were ultimately included. Fourteen studies defined operative experience thresholds using operative time as a surrogate measure of competence, whereas another 8 used trainer assessments of operative performance (Kirkpatrick level 3). A further 5 studies were able to determine the relationship between trainee case volumes and subsequent patient outcomes (Kirkpatrick level 4b).

CONCLUSION

Many studies have recorded competent trainee performance in key index procedures after reaching experience threshold numbers in excess of currently mandated targets across jurisdictions. The evidence relating current operative experience targets to patient outcomes across a range of surgical subspecialties of surgical subspecialties is lacking. This review supports a move toward criterion-based referencing of operative performance targets in surgical training.

Assessing Operative Skill in the Competency-based Education Era: Lessons From the UK and Ireland

INTRODUCTION

Decisions regarding the operative competence of surgical residents in the United Kingdom and Ireland are informed by operative workplace-based assessments (WBAs) and operative number targets for index procedures. This review seeks to outline the validity evidence of these assessment methods.

METHODS

A review of the MEDLINE (Pubmed), EMBASE and Cochrane Library databases was undertaken in accordance with the Joanna Briggs Institute Protocol for Scoping Reviews (2020). Articles were included if they provided evidence of the validity of procedure-based assessments, direct observation of procedural skills, or indicative operative number targets. The educational impact of each article was evaluated using a modified Kirkpatrick model.

RESULTS

Twenty-eight articles outlining validity evidence of WBAs and operative number targets were synthesised by narrative review. Five studies documented users' views on current assessment methods (Kirkpatrick level 1). Two articles recorded changes in attitudes towards current operative assessments (level 2a). Ten studies documented the ability of current assessments to record improvements in operative competence (level 2b). Ten studies measured a change in behaviour as a result of the introduction of these assessments (level 3). One article studied the ability of operative assessments to predict clinical outcomes (level 4b).

CONCLUSIONS

Operative WBAs are reliable. Scores achieved correlate with both time spent in training and recorded operative experience. Trainers and residents have concerns regarding the subjectivity of these assessments and the opportunistic nature in which they are used. Operative number targets are not criterion-referenced, lack validity evidence, and may be set too low to ensure operative competence.

A thematic review of the use of electronic logbooks for surgical assessment in sub-Saharan Africa

INTRODUCTION

Ensuring that surgical training programmes in low- and middle-income countries (LMICs) provide high quality training, including adequate operative experience, is of crucial importance in meeting the goals set out in the Lancet Global Surgery 2030. Electronic logbooks (eLogbooks) have been adopted to monitor both individual trainee progression and the performance of surgical training programmes.

METHODS

We performed a thematic review of the current evidence base surrounding the use of eLogbooks for the assessment of surgeons in training in sub-Saharan Africa, with a view to identifying the learning to date and areas for future research.

RESULTS

Whilst there are multiple papers highlighting the use of surgical eLogbooks in high-income countries, we identified only three papers which discussed their use in sub-Saharan Africa. Four common themes emerged which related to the use of surgical eLogbooks throughout sub-Saharan Africa: ease of analysis, centralised databases, discrepancies in reporting and technology limitations.

CONCLUSIONS

Robust data to demonstrate trainee progression and the quality of surgical training programmes are of crucial importance in ensuring that surgical training programmes can rapidly scale up to deliver large numbers of well-trained surgical providers to address the unmet patient need in LMICs in the next decade. The limited data on the use of well designed, centralised electronic surgical logbooks indicate that this tool may play an important role in providing key data to underpin these training programmes.

Postgraduate Surgical Training in the UK: the Trainees' Perspective

Over the last 20 years, surgical training in the United Kingdom (UK) has changed dramatically. There have been considerable efforts towards creating a programme that delivers the highest standard of training while maintaining patient safety. However, the journey to improve the quality of training has faced several hurdles and challenges. Recruitment processes, junior doctor contracts, flexible working hours and equality and diversity have all been under the spotlight in recent times. These issues, alongside the extended surgical team and the increasingly recognised importance of trainee wellbeing, mean that postgraduate surgical training is extremely topical. Alongside this, as technology has evolved, this has been incorporated into all aspects of training, from recruitment to simulated training opportunities and postgraduate examinations. The coronavirus (COVID-19) pandemic has brought technology and simulation to the forefront in an attempt to compensate for reduced operative exposure and experience, and has transformed the way that we learn and work. In this article, we reflect on the UK surgical trainee experience and discuss areas of success as well as highlighting potential areas for improvement going forward.