Competency-Based Medical Training in Anesthesiology: Has It Delivered on the Promise of Better Education?

Development of competency framework for postgraduate anesthesia training in China: a Delphi study

BACKGROUND

Standardized postgraduate medical education (PGME) in anesthesiology is not well-defined in China. Establishing a competency framework for postgraduate anesthesia training (PGAT) is essential for standardizing and enhancing the quality of training and patient care.

OBJECTIVES

This study aimed to develop a competency framework for PGAT in China.

METHODS

This study employed a multi-step approach, including a literature review, semi-structured interviews, to formulate a list of preliminary competencies. This initial list included seven roles, 26 enabling competencies, and 162 competency items. A modified Delphi method was utilized to achieve consensus involving three rounds. Experts were recruited from the Chinese Society of Anesthesiology (CSA) across various regions of the country. Consensus was determined using a 5-point Likert scale, with a mean score of ≥ 4 and a consensus rate of ≥ 80% serving as criteria for agreement.

RESULTS

Forty-seven experts accepted the invitation to participate, with 46 returning scores for round 1 (90.2% response rate) and 45 returning scores for rounds 2 and 3 (88.2% response rate). The final competency framework includes 140 competency items within 23 enabling competencies, categorized into seven roles: medical expert, communicator, collaborator, professional leader, health advocate, academic scholar, and specialized professional.

CONCLUSIONS

This study represents an initial step towards establishing a contemporary competency-based medical education and training (CBMET) program for PGAT in China.

Simulation-based medical education in Canadian anesthesiology academic institutions: a national survey

PURPOSE

Simulation-based medical education (SBME) is provided by all anesthesiology residency programs in Canada. The purpose of this study was to characterize SBME in Canadian anesthesiology residency training programs.

METHODS

We administered a 21-question survey to the simulation director/coordinator for all 17 Canadian academic departments of anesthesiology from October 2019 to January 2020. The survey consisted of questions pertaining to the characteristics of the simulation centres, their faculty, learners, curriculum, and assessment processes.

RESULTS

All 17 residency training programs participated in the survey and reported large variability in the number and formal training of simulation faculty and in content delivery. Five programs (29%) did not provide faculty recognition for curriculum design and running simulation sessions. Most programs offered one to four simulation sessions per academic year for each year of residency. All programs offered mannequin-based and part-task trainers for teaching technical and nontechnical skills. Fourteen programs (82%) offered interprofessional and interdisciplinary simulation sessions, and ten programs (59%) did not include in situ simulation training. Commonly reported barriers to faculty involvement were lack of protected time (12 programs, 71%), lack of financial compensation (ten programs, 59%), and lack of appreciation for SBME (seven programs, 41%).

CONCLUSION

Large variability exists in the delivery of SBME in Canadian anesthesiology residency simulation programs, in part because of differences in financial/human resources and educational content. Future studies should explore whether training and patient outcomes differ between SBME programs and, if so, whether additional standardization is warranted.

Test-enhanced learning in Neuroanesthesia for the First Year anesthetic residents: a randomized controlled trial

BACKGROUND

Test enhancing learning (TEL) had shown a significant effect in promoting the learning of many learning contents. However, its effect on the postgraduate medical level was unclear. This study aimed to investigate the effect of TEL in 1st year anesthesiology residents learning neuroanesthesia.

METHOD

The residents were randomized to either group A, which was assigned to do the intervention exam (exam A) for two times during learning in neuroanesthesia, or group B, which studied in the same environment without doing the exam. All participants were assigned to do the assessment exam (exam B) at one month after the end of the rotation. All of the exams were ten multiple choice questions (MCQ). Since the anesthesia residents rotated to neuroanesthesia for two weeks twice during the first year, we conducted the experiments twice, using exams that covered both basic science (BS) and clinical science (CS) topics.

RESULTS

There was no significant difference in mean ± SD of the scores for assessment exams asking about the basic science topic (BS_B) [group A (5.25 ± 2.05) VS group B (4.90 ± 1.80); p = 0.570] and the clinical science topic (CS_B) [group A (6.30 ± 1.26) VS group B (5.95 ± 1.61); p = 0.448].

CONCLUSION

This study showed null findings on the effect of TEL on learning in residents of the first year of anesthesiology. More studies on TEL were required to confirm the effect of TEL and find the appropriate test format that could enhance learning for post-graduate medical trainees.

The Accuracy of the Learning-Curve Cumulative Sum Method in Assessing Brachial Plexus Block Competency

BACKGROUND

The learning-curve cumulative sum method (LC-CUSUM) and its risk-adjusted form (RA-LC-CUSUM) have been proposed as performance-monitoring methods to assess competency during the learning phase of procedural skills. However, scarce data exist about the method's accuracy. This study aimed to compare the accuracy of LC-CUSUM forms using historical data consisting of sequences of successes and failures in brachial plexus blocks (BPBs) performed by anesthesia residents.

METHODS

Using historical data from 1713 BPB performed by 32 anesthesia residents, individual learning curves were constructed using the LC-CUSUM and RA-LC-CUSUM methods. A multilevel logistic regression model predicted the procedure-specific risk of failure incorporated in the RA-LC-CUSUM calculations. Competency was defined as a maximum 15% cumulative failure rate and was used as the reference for determining the accuracy of both methods.

RESULTS

According to the LC-CUSUM method, 22 residents (84.61%) attained competency after a median of 18.5 blocks (interquartile range [IQR], 14-23), while the RA-LC-CUSUM assigned competency to 20 residents (76.92%) after a median of 17.5 blocks (IQR, 14-25, P = .001). The median failure rate at reaching competency was 6.5% (4%-9.75%) under the LC-CUSUM and 6.5% (4%-9%) for the RA-LC-CUSUM method ( P = .37). The sensitivity of the LC-CUSUM (85%; 95% confidence interval [CI], 71%-98%) was similar to the RA-LC-CUSUM method (77%; 95% CI, 61%-93%; P = .15). Identical specificity values were found for both methods (67%; 95% CI, 29%-100%, P = 1).

CONCLUSIONS

The LC-CUSUM and RA-LC-CUSUM methods were associated with substantial false-positive and false-negative rates. Also, small lower limits for the 95% CIs around the accuracy measures were observed, indicating that the methods may be inaccurate for high-stakes decisions about resident competency at BPBs.

Anaesthesia training designs across Europe: A survey-based study from the trainees committee of the European Society of Anaesthesiology and Intensive Care

BACKGROUND

Anaesthesiology training programs in Europe vary in duration, content, and requirements for completion. This survey-based study conducted by the Trainees Committee of the European Society of Anaesthesiology and Intensive Care explores current anaesthesia training designs across Europe.

METHODS

Between May and July 2018, we sent a 41-item online questionnaire to all National Trainee Representatives, members of the National Anaesthesiologists Societies Committee, and Council Representatives of the European Society of Anaesthesiology and Intensive Care (ESAIC) of all member countries. We cross-validated inconsistent data with different country representatives.

RESULTS

Forty-three anaesthesiologists from all 39 associated ESAIC countries completed the questionnaire. Results showed considerable variability in teaching formats, frequency of teaching sessions during training, and differences in assessments made during and at the end of training. The reported duration of training was 60 months in 59% (n = 23) of participating countries, ranging from 24 months in Russia and Ukraine to 84 months in the UK.

CONCLUSION

This study shows the significant differences in anaesthesiology training formats across Europe, and highlights the importance of developing standardised training programs to ensure a consistent level of training and to improve patient safety. This study provides valuable insights into European anaesthesia training, and underlines the need for further research and collaboration to improve requirements.

Competency-Based Time-Variable Anesthesiology Residency Training: Identification of Problems and Solutions

BACKGROUND

Global medical education is gradually moving toward more comprehensive implementations of a competency-based education (CBE) model. Elimination of standard time-based training and adoption of time-variable training (competency-based time-variable training [CB-TVT]) is one of the final stages of implementation of CBE. While CB-TVT has been implemented in some programs outside the United States, residency programs in the United States are still exploring this approach to training. The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) are encouraging member boards and residency review committees to consider innovative ways programs could implement CB-TVT. The goals of this study were to (1) identify potential problems with the implementation of CB-TVT in anesthesiology residency training, (2) rank the importance of the problems and the perceived difficulty of solving them, and (3) develop proposed solutions to the identified problems.

METHODS

Study participants were recruited from key stakeholder groups in anesthesiology education, including current or former program directors, department chairs, residents, fellows, American Board of Anesthesiology (ABA) board members, ACGME residency review committee members or ACGME leaders, designated institutional officials, residency program coordinators, clinical operations directors, and leaders of large anesthesiology community practice groups. This study was conducted in 2 phases. In phase 1, survey questionnaires were iteratively distributed to participants to identify problems with the implementation of CB-TVT. Participants were also asked to rank the perceived importance and difficulty of each problem and to identify relevant stakeholder groups that would be responsible for solving each problem. In phase 2, surveys focused on identifying potential solutions for problems identified in phase 1.

RESULTS

A total of 36 stakeholders identified 39 potential problems, grouped into 7 major categories, with the implementation of CB-TVT in anesthesiology residency training. Of the 39 problems, 19 (48.7%) were marked as important or very important on a 5-point scale and 12 of 19 (63.2%) of the important problems were marked as difficult or very difficult to solve on a 5-point scale. Stakeholders proposed 165 total solutions to the identified problems.

CONCLUSIONS

CB-TVT is a promising educational model for anesthesiology residency, which potentially results in learner flexibility, individualization of curricula, and utilization of competencies to determine learner advancement. Because of the potential problems with the implementation of CB-TVT, it is important for future pilot implementations of CB-TVT to document realized problems, efficacy of solutions, and effects on educational outcomes to justify the burden of implementing CB-TVT.

Current use of simulation for EPA assessment in emergency medicine

OBJECTIVE

Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment.

METHODS

A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views.

RESULTS

A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment.

CONCLUSIONS

Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

Education in obstetric anesthesiology: an international approach

Competency-based training and active teaching methods are increasingly becoming accepted and utilized in medical schools and hospitals, and obstetric anesthesiology training is expected to follow this process. This article summarizes current modalities of obstetric anesthesiology training in five countries from various parts of the world. Analysis of these curricula shows that implementation of new educational methods is variable, incomplete, and lacking in data related to patient outcomes. Research in assessments and practical applications are required to avoid wide ranges of educational strategies.

Postgraduate training in anaesthesiology - A modular curriculum

The primary objective of postgraduate medical education is to produce specialists who provide highest quality of health care to suffering patients and return them to the community in the most functional capability. The secondary objective is to advance the cause of science through research and training. A postgraduate, after undergoing the required training in anaesthesiology, should be able to recognise the health needs of the community and apply cognitive and psychomotor skills to provide optimal anaesthetic care. Additionally, the anaesthesiologist should function as a perioperative physician being adept in perioperative care, pain medicine and critical care medicine. The 3-year postgraduate curriculum comprises experience in basic, subspeciality and advanced anaesthesia training. This structured training programme with a curriculum of increasing difficulty and learning incorporates ascending grades of difficulty, posing a challenge to the trainee's intellect and technical skills. Experience in basic anaesthesia training is aimed to lay stress on basic and fundamental aspects of anaesthetic management. Subspeciality anaesthesia training is needed to lay stress on the theory, special considerations and practice of subdisciplines of anaesthesiology. This document proposes a modular-structured, continuous, objectively evaluated, systematic training process that is monitored frequently and periodically, such that the trainee, at the end of training, is capable of appropriate anaesthetic management of disease conditions in a wide variety of situations.

The Role of Entrustable Professional Activities in Competency-based Medical Education for Anesthesiology Residents: A Pilot Phase

Background

After graduation, physicians should be able to provide professional and safe services without the need for supervision by their clinical professors, mandating a competency-based medical education (CBME) approach.

Objectives

This study aimed to develop a national model of entrustable professional activities (EPAs) based on our experiences in the Department of Anesthesiology and Critical Care (DACC), Shahid Beheshti University of Medical Sciences (SBMU).

Methods

The primary EPA design plan was designed in a 10-step model as a career roadmap for the project. The texts were prepared according to a consensus-based approach. On the other hand, the texts were reviewed and revised by a broad team of faculty in a daily workshop.

Results

The final product included 14 topics for EPA as the first round of targeting topics for anesthesiology residents. The texts were developed using previous studies and were standardized considering national standards.

Conclusions

We described a clear path toward designing and implementing EPAs in anesthesiology residency programs to improve the quality of the graduated residents. Though the basic theory is the same, each country needs its formula for implementing the process.

Looking Ahead to the Frontier of Anesthesiology Education

In this issue of Anesthesia & Analgesia, a series of articles focus on the elemental changes to anesthesia training and education. Kealey and Naik review the status of competency-based medical training, a method through which learners are deliberately observed for progression to mastery in clinical management. This is contrast to the assumption that trainees will presumably reach the same intended endpoint merely by spending a pre-specified amount of time in post-graduate residency training. The advantages and disadvantages of the competency-based approach are reviewed. Alam and Matava describe how education has also changed to incorporate digital technology by way of immersive simulation. They detail the use of virtual and augmented reality to offer trainees the opportunity to engage in clinical exercises that are infrequently encountered in real practice, increase the exposure to challenging scenarios and foster real-time collaborations on a global scale. An accompanying editorial offers further perspective on the future of training in our specialty. The reader is strongly encouraged to review the cited articles for an in-depth appreciation of the concepts discussed.