Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

OBJECTIVES

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency.

DESIGN

A participatory action research model with a mixed methods approach.

SETTING

This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department.

PARTICIPANTS

Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention.

RESULTS

The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4.

CONCLUSION

Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Twelve tips for creating a multicultural mindfulness-based intervention in diverse healthcare settings

High levels of stress and burnout, low help-seeking behaviour and unhealthy coping in healthcare professionals (HCPs), are a critical concern globally. Mindfulness-based interventions (MBIs) reduce stress, are a healthy coping mechanism and have become increasingly popular among HCPs, especially during the Covid-19 pandemic. Nevertheless, HCPs' busy schedules require the intervention to be accessible, pragmatic, and context specific. Key to this goal is approaching mindfulness from a multicultural perspective, especially in diverse settings such as Africa. This paper aims to provide practical tips to ensure that the MBI offered is effective with multicultural HCPs during intense stress. Four tips each discuss the curriculum, implementation, and sustainability, respectively. The tips are elucidated by practical examples of regulating stress in healthcare settings and to offer a guideline to help structure future MBIs to be culturally and context appropriate.

Specialist training: workplace-based assessments impact on teaching, learning and feedback to support competency-based postgraduate programs

BACKGROUND

Workplace-based assessments (WBAs) are part of a competency-based curriculum where training progression is dependent on the achievement of defined competencies in a real-world clinical environment. There is a significant literature gap on the impact of WBAs implemented in resource constrained countries and their contextual challenges. This study aimed to examine the use, impact, and educational context of WBAs in South African medical specialist training programs drawing on perspectives from both trainees and trainers to identify educational challenges and propose effective solutions.

METHODS

A mixed methods national electronic survey was conducted with specialist medical trainees and supervising trainers from all eight specialist training institutions in South Africa involving 16 specialities. The survey responses were voluntary and anonymous. The survey was closed after seven months when data saturation was achieved. Descriptive statistical analysis was performed using SPSS Version 27 (SPSS Inc, 2012, Chicago, IL) for the quantitative analysis. The thematic coding framework for the qualitative analysis was facilitated by NVivo Version 12 software.

RESULTS

There were 108 ethnically diverse supervising trainers and 248 specialist trainees' survey respondents. Across the 16 medical specialities, 45% of the respondents were using WBAs. Despite contextual resource and staff challenges, this study found that WBAs had a positive impact to Kirkpatrick level 2 in providing actionable feedback to improve competency. WBA users had a significantly higher rating for trainee supervision (p < 0.01), general quality of feedback on trainee competence (< 0.01) and the specialist training program (p = 0.03) compared to WBA non-users. They also had a higher rating for the assessment of the trainee as a professional (p < 0.01); scholar (p < 0.01); communicator (p < 0.01); collaborator (p = 0.001) and leader/manager (p < 0.001) based on the AfriMEDS competency framework. Racism, sexism and favouritism were challenges that negatively affected the training programs.

CONCLUSION

Overall, this study reports that the use of WBAs had a substantially favourable impact on teaching, learning, feedback and supports a competency-based approach to specialist training programs. Addressing the contextual concerns that negatively impact training; training the trainees and trainers about their relationship, roles and responsibilities; and focusing on a trainee-centred, inclusive and empowering teaching approach will help further enhance its effectiveness.

Factors That Enhance and Hinder the Retention and Transfer of Online Pre-Clinical Skills Training to Facilitate Blended Learning

Purpose

During the SARS-CoV-2 pandemic, various online instructional strategies in clinical skills training were piloted. The sudden transition to the remote platform circumvented the rigorous planning associated with curriculum reform. This study aimed to explore students' and tutors' perceptions of factors that promoted or hindered successful learning transfer and to propose a blended conceptual model to guide affective, cognitive, and psychomotor clinical skills training in the pre-clinical phase of medical education.

Methods

A mixed-method quasi-experimental study assessed third-year students' transfer of clinical skills and knowledge following online learning in 2021. Students and their tutors completed online surveys that included open and closed-ended questions regarding factors influencing their experience of the adapted teaching methods. Descriptive statistical analysis was used for the quantitative data. Qualitative responses were thematically analyzed.

Results

One hundred fourteen students (48%) and seven tutors (100%) responded to the surveys. The questionnaires' internal consistency and construct validity were determined using Cronbach's α-Coefficient. There was an overall positive response (86%) to the acceptability of the online platform in clinical skills training. Using online simulations with targeted onsite practice was reported as effective in clinical skills training. Tutors perceived students as well-prepared for the skills laboratory. Five emergent themes, qualified by a linear model of asynchronous and synchronous online and onsite teaching with the evaluation of the instructional design and institutional support, informed the proposed blended learning guide for clinical skills training in the pre-clinical phase.

Conclusion

Blended clinical skills learning that included the flipped classroom concept was well-accepted. Virtual patients proved a convenient cognitive preparation tool for skills training and potentially optimized teaching delivery. The study found that the adapted teaching frameworks incorporating an online clinical skills component into a modified onsite curriculum augmented learners' ability to transfer knowledge to the clinical skills laboratory. An integrated five-step blended model is proposed for future interventions.

The Use, Effectiveness, and Impact of Workplace-Based Assessments on Teaching, Supervision and Feedback Across Surgical Specialties

OBJECTIVE

To investigate the use and effectiveness of Workplace-based assessments (WBAs) and their impact on training, feedback, and perioperative teaching in surgical training programs.

DESIGN

A mixed methods cross-sectional, national electronic survey was conducted with surgical trainees and consultant trainers.

SETTINGS

The trainees and supervising faculty were from all 8 major surgical training universities across 11 surgical disciplines in South Africa.

PARTICIPANTS

A total of 108 surgical trainees and 41 supervising consultant trainers from 11 surgical disciplines across 8 surgical training universities responded to the survey.

RESULTS

The most significant educational gap identified by both the surgical trainees and trainers across all surgical disciplines was inadequate perioperative feedback. A third of the respondents were currently using workplace-based assessments. The WBA users (both trainees and trainers) had a higher rating for the general quality of surgical feedback than WBA nonusers (p = 0.02). WBA users also had a higher rating for the general quality of feedback given to trainees on their skills and competence (p = 0.04) and a higher rating for trainee supervision (p = 0.01) and the specialist training program overall (p = 0.01). The WBA users also had a higher rating for the assessment of competencies such as the trainee as an effective communicator (p < 0.01) and collaborator (p = 0.04).

CONCLUSION

This study found that the use of WBAs enhances the quality and effectiveness of feedback in surgical training programs. We also found that the use of WBAs enhance perioperative teaching and learning and improves the assessment of relational competencies. This was also associated with high ratings for the quality of trainee supervision. Faculty and trainee development, strengthening the trainee-trainer relationship, and integrating iterative stakeholder feedback could help realize the full potential of WBAs to augment surgical training across disciplines.

Effects of personality traits on mindful self-care practices of healthcare workers

Background: Coronavirus disease 2019 (COVID-19) placed healthcare professionals (HCPs) at a higher risk for stress-related conditions. Implementing a brief online mindfulness-based intervention (MBI) was hypothesised to transform the HCPs’ ability to cope with stress by enhancing their self-care.Aim: This study aimed to explore the impact of an online MBI on HCPs’ self-care practices and determine if personality traits were a moderating variable.Setting: An online MBI was implemented for HCPs working in South Africa during the COVID-19 pandemic lockdowns.Methods: A quantitative study design included a pre-assessment and post-assessment component, which allowed paired comparison and regression analysis to confer correlations. Data were collected via two validated instruments: the Mindful Self-Care scale-2018 and the Big Five Personality test.Results: Forty-nine HCPs participated in the study. Significant improvements were found in all the major self-care subscales post-intervention (p 0.05). No significant associations were found between the personality traits and self-care except for neuroticism, which appeared to be an essential moderating variable.Conclusion: An online MBI significantly impacted health professionals’ ability to care for themselves, despite their personality styles.Contribution: The impact of an online MBI on HCPs’ self-care during the most intense time of stress and with a cohort of people known to be the most vulnerable to stress, namely those with neuroticism to date, has not been commented on.

Use of digital technology to give and receive feedback in clinical training: a scoping review protocol

BACKGROUND

Feedback is vital to improving trainee competencies in medical education. The challenges of the global COVID-19 pandemic related to social distancing to curb the spread of the virus ignited a rapid transition to online medical education. These changes highlight the need for digital feedback tools that enhance the efficacy and efficiency of feedback practices. This protocol is for a scoping review that aims to identify the different digital tools and applications in medical education as reported in the literature, as well as highlight gaps in the current literature and provide suggestions for future technological developments and research.

METHODS AND ANALYSIS

A review of the relevant literature will be guided using the Joanna Briggs Institute methodological framework for scoping studies. Using the search strategy developed by the authors, an electronic search of the following databases will be conducted: PubMed/MEDLINE, EBSCOhost (academic search complete, CINAHL with full text) Scopus, Google Scholar, Union Catalogue of Theses and Dissertations (UCTD) via SABINET Online and World Cat Dissertations and Theses via OCLC. Studies will be identified by searching literature from January 2010 to date of review. Using a validated data extraction form developed for the scoping review, the review team will screen eligible studies and import them onto an electronic library created specifically for this purpose. Data collection for the review will be documented through a PRISMA-P flowchart, and the scoping review will use a basic descriptive content analysis to analyse and categorise the extracted data. All review steps will involve two or more reviewers.

DISSEMINATION

The review will provide a comprehensive list of digital tools and applications used to enhance feedback in clinical training and inform future technological developments. The findings will be disseminated through medical education conferences and publications.

Medical internship training during the COVID-19 pandemic – A case of ‘sacrificial pawns’ or not?

Background

Newly qualified medical practitioners in South Africa (SA) are part of the frontline health care workers who face Africa’s most severe coronavirus disease 2019 (COVID-19) pandemic. The experiences of interns during the pandemic reflect SA’s preparedness to respond in a crisis and inform strategies that could be adopted to balance training and service in resource-challenged contexts.

Aim

To explore the strengths, weaknesses, opportunities and threats posed during the first wave of the COVID-19 pandemic as reflected on by interns within the clinical training platforms in SA.

Setting

Public hospitals in KwaZulu-Natal.

Methods

An online questionnaire consisting of eight open-ended questions based on the SWOT framework related to personal and professional perspectives to clinical training during the COVID-19 pandemic was developed using SurveyMonkey. All data were collected remotely via social media platforms. Data were thematically analysed.

Results

Forty-six interns reflected on personal and systemic challenges as the major threats and weaknesses in intern training during the COVID-19 pandemic. Extrapolating on strengths and opportunities, there were three overarching learnings interns reflected on. These related to being a medical professional, communities of practice and the development and enhancement of clinical and non-clinical competencies. Existing challenges in the environment exacerbated the threats posed by COVID-19 and innovative strategies related to improving support, feedback, broadening the intern curriculum and online training.

Conclusion

Although the clinical environment where interns learn and work is often stressful and overpowered by high service burdens, there are unique opportunities to enhance self-directed learning and graduate competencies, even in the midst of the COVID-19 pandemic.

Self-Directed Learning During the COVID-19 Pandemic: Perspectives of South African Final-Year Health Professions Students

PURPOSE

Self-directed learning (SDL) has been advocated for effective training of final-year health professions students. COVID-19 challenges conventional teaching, learning, and assessment in the clinical environment. This study aimed to identify and explore enablers and barriers to SDL among final-year health professions students training during the COVID-19 pandemic.

METHODS

Adopting the SWOT (strengths, weaknesses, opportunities and threats) framework, this study explored the clinical learning and training experiences of final-year health professions students during the pandemic. A survey was conducted via online platforms. Data from 155 respondents were thematically analyzed.

RESULTS

Personal attributes such as reflection, self-determination, motivation, resilience, and positive learning behaviors and skills were identified as SDL enablers. Collaborative learning networks and online learning platforms facilitate learning needs and goals. Fear and anxiety, untrusted learning sites, uncertainty about graduation, financial issues and challenges in the learning environments were the major themes related to barriers to SDL.

CONCLUSION

The importance of SDL as a skill for uncertain times warrants further investigation in the training of future healthcare professionals. Inclusive planning and engagement with final-year health professions students to address identified stressors, as well as the creation of shared platforms where students are part of the decision-making processes for clinical learning and training are recommended. Responsive curricula that optimize unpredictable disruptions in clinical training are needed to equip students to diagnose their own learning needs and implement appropriate learning strategies.

Efficacy of a brief online mindfulness-based intervention on the psychological well-being of health care professionals and trainees during the COVID-19 pandemic: A mixed method design

Background

During the coronavirus disease 2019 (COVID-19) pandemic, health professionals were pushed to the front line of a global health crisis unprepared and resource constrained, which affected their mental well-being.

Aim

This study aimed to investigate the effectiveness of a brief online mindfulness-based intervention (MBI) on stress and burnout for health professionals training and working in South Africa during the COVID-19 crisis.

Setting

The context of the study is the overburdened, under-resourced health care system in South Africa during a global pandemic.

Methods

A mixed method framework was adopted for this study. The quantitative data was analysed using descriptive analysis and the participants’ qualitative experiences were interpreted using interpretative phenomenological analysis.

Results

Forty-seven participants took part in this study. The study found a statistically significant (p < 0.05) reduction in stress levels and emotional exhaustion as well as an increase in mindful awareness and feelings of personal accomplishment after the intervention. The participants’ shared experiences were analysed in two parts. The pre-intervention analysis presented with central themes of loss of control and a sense of powerlessness because of COVID-19. The post-intervention analysis comprised themes of a sense of acquired control and empowerment through increased mindfulness.

Conclusions

The study found that a brief online MBI can be associated with reduced levels of stress and burnout as well as an increased sense of control and empowerment, felt both personally and professionally, during a global crisis.

Contribution

The impact of an online MBI for health care professionals amidst a pandemic has not been previously documented.

Barriers and promotors in receptivity and utilization of feedback in a pre-clinical simulation based clinical training setting

Objective: Many studies have explored feedback effectiveness using interventions focused on feedback delivery. It is equally important to consider how learners actively receive, engage with and interpret feedback. This study explores how medical students receive and use feedback in pre-clinical skills training. Method: Focus group data from 25 purposively selected thirdyear medical students was thematically analysed. Four major themes and eight sub-themes related to the facilitators and barriers to feedback receptivity and utilisation to feed forward emerged from the data. Results and Discussion: Students were receptive to feedback when its purpose and content aligned with their personal objectives, when it was consistent between tutors, and when it involved developing longitudinal relationships. The clinical skills formative logbook feedback culture with a learning focus was perceived to be predictive of their future performance and they were likely to take feedback on board, emphasising the role of reflection in this process. The depth and timing of actual feedback use varied among students, and language barriers hindered decoding feedback. Students’ self-regulatory focus on the feedback process had a dominant influence on their active use of feedback. Conclusion: Incorporating learner behaviour underlying feedback use should be considered when designing interventions to promote feedback engagement, feedback literacy skills and responsibility sharing in the feedback process. Establishing a learning culture that promotes shared responsibility between clinical educators and learners enable greater control by learners over assessment and feedback processes and a commitment to behaviour change. Bangladesh Journal of Medical Science Vol.20(3) 2021 p.594-607

Feasibility and perceptions of video tutorials for intern training in anaesthesia: a survey

Background: Effective intern training is essential for safe delivery of anaesthesia during community service. Interns are trained using both tutorials and clinical exposure. E-learning platforms using video tutorials may address millennial learner needs better, but feasibility is unknown in a particular local setting. This study aimed to explore the feasibility of using a video tutorial to facilitate intern training. Perceptions of the value of tutorials were also assessed. Methods: A cross-sectional survey was conducted amongst interns who rotated through anaesthesia at a regional hospital in KwaZulu-Natal. The interns had been exposed to face-to-face tutorials previously and provided with a video-based tutorial. Results: Thirty-seven of the interns consented (88%), watched the video and answered the questionnaire. All interns indicated a need for tutorials during internship. A majority of respondents (71%) felt underprepared by undergraduate training programmes. Most interns were prepared to watch the video tutorial outside of working hours; however, they were divided over the fairness of using internet access for tutorials. There was a preference for video tutorials over reading a textbook or lecture notes. A total of 88% watched the video from the Internet. One intern did not have Internet access or a computer but the video was watched using a free Wi-Fi hotspot. Conclusion: A video-based tutorial system is feasible but current infrastructure needs to be considered. Interns continue to feel underprepared for internship and request that tutorials continue in internship.

Using deliberate practice framework to assess the quality of feedback in undergraduate clinical skills training

BACKGROUND

In this research paper we report on the quality of feedback provided in the logbooks of pre-clinical undergraduate students based on a model of 'actionable feedback'. Feedback to clinical learners about their performance is crucial to their learning, which ultimately impacts on their development into competent clinicians. Due to students' concerns regarding the inconsistency and quality of feedback provided by clinicians, a structured feedback improvement strategy to move feedback forward was added to the clinical skills logbook. The instrument was also extended for peer assessment. This study aims to assess the quality of feedback using the deliberate practice framework.

METHODS

A feedback scoring system was used to retrospectively assess the quality of tutor and peer logbook feedback provided to second and third year medical students to identify deliberate practice components i.e. task, performance gap and action plan. The sample consisted of 425 second year and 600 third year feedback responses over a year.

RESULTS

All three deliberate practice components were observed in the majority of the written feedback for both classes. The frequency was higher in peer (83%, 89%) than tutor logbook assessments (51%, 67%) in both classes respectively. Average tutor and peer task, gap and action feedback scores ranged from 1.84-2.07 and 1.93-2.21 respectively. The overall quality of feedback provided by the tutor and peer was moderate and less specific (average score < or = 2). The absence of the three components was noted in only 1% of the feedback responses in both 2nd and 3rd year.

CONCLUSION

This study found that adding in a feed-forward strategy to the logbooks increased the overall quality of tutor and peer feedback as the task, gap and action plans were described. Deliberate practice framework provides an objective assessment of tutor and peer feedback quality and can be used for faculty development and training. The findings from our study suggest that the ratings from the tool can also be used as guidelines to provide feedback providers with feedback on the quality of feedback they provided. This includes specifically describing a task, performance gap and providing a learning plan as feed-forward to enhance feedback given.

"For most of us Africans, we don't just speak": a qualitative investigation into collaborative heterogeneous PBL group learning

Collaborative approaches such as Problem Based Learning (PBL) may provide the opportunity to bring together diverse students but their efficacy in practice and the complications that arise due to the mixed ethnicity needs further investigation. This study explores the key advantages and problems of heterogeneous PBL groups from the students' and teachers' opinions. Focus groups were conducted with a stratified sample of second year medical students and their PBL teachers. We found that students working in heterogeneous groupings interact with students with whom they don't normally interact with, learn a lot more from each other because of their differences in language and academic preparedness and become better prepared for their future professions in multicultural societies. On the other hand we found students segregating in the tutorials along racial lines and that status factors disempowered students and subsequently their productivity. Among the challenges was also that academic and language diversity hindered student learning. In light of these the recommendations were that teachers need special diversity training to deal with heterogeneous groups and the tensions that arise. Attention should be given to create 'the right mix' for group learning in diverse student populations. The findings demonstrate that collaborative heterogeneous learning has two sides that need to be balanced. On the positive end we have the 'ideology' behind mixing diverse students and on the negative the 'practice' behind mixing students. More research is needed to explore these variations and their efficacy in more detail.

Reliability and validity of a Tutorial Group Effectiveness Instrument

BACKGROUND

Tutorial group effectiveness is essential for the success of learning in problem-based learning (PBL). Less effective and dysfunctional groups compromise the quality of students learning in PBL.

AIMS

This article aims to report on the reliability and validity of an instrument aimed at measuring tutorial group effectiveness in PBL.

METHOD

The items within the instrument are clustered around motivational and cognitive factors based on Slavin's theoretical framework. A confirmatory factor analysis (CFA) was carried out to estimate the validity of the instrument. Furthermore, generalizability studies were conducted and alpha coefficients were computed to determine the reliability and homogeneity of each factor.

RESULTS

The CFA indicated that a three-factor model comprising 19 items showed a good fit with the data. Alpha coefficients per factor were high. The findings of the generalizability studies indicated that at least 9-10 student responses are needed in order to obtain reliable data at the tutorial group level.

CONCLUSION

The instrument validated in this study has the potential to provide faculty and students with diagnostic information and feedback about student behaviors that enhance and hinder tutorial group effectiveness.

Perceptions of problem-based learning (PBL) group effectiveness in a socially-culturally diverse medical student population

INTRODUCTION

A key aspect of the success of a PBL curriculum is the effective implementation of its small group tutorials. Diversity among students participating in tutorials may affect the effectiveness of the tutorials and may require different implementation strategies.

AIMS

To determine how students from diverse backgrounds perceive the effectiveness of the processes and content of the PBL tutorials. This study also aims to explore the relationship between students' perceptions of their PBL tutorials and their gender, age, language, prior educational training, and secondary schooling.

MATERIALS/METHODS

Data were survey results from 244 first-year student-respondents at the Nelson Mandela School of Medicine at the University of KwaZulu-Natal in South Africa. Exploratory factor analysis was conducted to verify scale constructs in the questionnaire. Relationships between independent and dependent variables were investigated in an analysis of variance.

RESULTS

The average scores for the items measured varied between 3.3 and 3.8 (scale value 1 indicated negative regard and 5 indicated positive regard). Among process measures, approximately two-thirds of students felt that learning in a group was neither frustrating nor stressful and that they enjoyed learning how to work with students from different social and cultural backgrounds. Among content measures, 80% of the students felt that they learned to work successfully with students from different social and cultural groups and 77% felt that they benefited from the input of other group members. Mean ratings on these measures did not vary with students' gender, age, first language, prior educational training, and the types of schools they had previously attended.

DISCUSSION AND CONCLUSION

Medical students of the University of KwaZulu-Natal, regardless of their backgrounds, generally have positive perceptions of small group learning. These findings support previous studies in highlighting the role that small group tutorials can play in overcoming cultural barriers and promoting unity and collaborative learning within diverse student groups.