COVID-19 Pivoted Virtual Skills Teaching Model: Project ECHO Ontario Skin and Wound Care Boot Camp

OBJECTIVE

To describe a virtual, competency-based skin and wound care (SWC) skills training model. The ECHO (Extension for Community Healthcare Outcomes) Ontario SWC pivoted from an in-person boot camp to a virtual format because of the COVID-19 pandemic.

METHODS

An outcome-based program evaluation was conducted. Participants first watched guided commentary and videos of experts performing in nine SWC multiskills videos, then practiced and video-recorded themselves performing those skills; these recordings were assessed by facilitators. Data were collected using pre-post surveys and rubric-based assessments. Descriptive statistics and thematic analysis were applied to data analysis.

RESULTS

Fifty-five healthcare professionals participated in the virtual boot camp, measured by the submission of at least one video. A total of 216 videos were submitted and 215 assessment rubrics were completed. Twenty-nine participants completed the pre-boot camp survey (53% response rate) and 26 responded to the post-boot camp survey (47% response rate). The strengths of the boot camp included the applicability of virtual learning to clinical settings, boot camp supplies, tool kits, and teaching strategies. The analysis of survey responses indicated that average proficiency scores were greater than 80% for three videos, 50% to 70% for three of the videos, and less than 50% for three of the videos. Participants received lower scores in local wound care and hand washing points of contact. The barriers of the boot camp included technical issues, time, level of knowledge required at times, and lack of equipment and access to interprofessional teams.

CONCLUSIONS

This virtual ECHO SWC model expanded access to practical skills acquisition. The professional development model presented here is generalizable to other healthcare domains.

Developing Faculty Developers: An Underexplored Realm in Professional Development

ABSTRACT

As faculty developers enter the field and grow in their roles, how do they keep up with ongoing changes and ensure that their knowledge remains relevant and up-to-date? In contrast to most of the studies which focused on the needs of faculty members, we focus on the needs of those who fulfill the needs of others. We highlight the knowledge gap and lack of adaptation of the field to consider the issue of professional development of faculty developers more broadly by studying how they identify their knowledge gaps and what approaches they use to address those gaps. The discussion of this problem sheds light on the professional development of faculty developers and offers several implications for practice and research. Our own piece of the solution indicates that faculty developers follow a multimodal approach to the development of their knowledge, including formal and informal approaches to addressing perceived gaps. Within this multimodal approach, our results suggest that the professional growth and learning of faculty developers is best characterized as a social practice. Based on our research, it would seem worthwhile for those in the field to become more intentional about the professional development of faculty developers and harness aspects of social learning in that process to better reflect faculty developers' learning habits. We also recommend applying these aspects more broadly to, in turn, enhance the development of educational knowledge and educational practices for the faculty members these educators support.

A Canadian survey of residency applicants' and interviewers' perceptions of the 2021 CaRMS R1 virtual interviews

BACKGROUND

All Canadian Residency Matching Service (CaRMS) R1 interviews were conducted virtually for the first time in 2021. We explored the facilitators, barriers, and implications of the virtual interview process for the CaRMS R1 match and provide recommendations for improvement.

METHODS

We conducted a cross-sectional survey study of CaRMS R1 residency applicants and interviewers across Canada in 2021. Surveys were distributed by email to the interviewers, and by email, social media, or newsletter to the applicants. Inductive thematic analysis was used for open-ended items. Recommendations were provided as frequencies to demonstrate strength. Close-ended items were described and compared across groups using Chi-Square Fisher's Exact tests.

RESULTS

A total of 127 applicants and 400 interviewers, including 127 program directors, responded to the survey. 193/380 (50.8%) interviewers and 90/118 (76.3%) applicants preferred virtual over in-person interview formats. Facilitators of the virtual interview format included cost and time savings, ease of scheduling, reduced environmental impact, greater equity, less stress, greater reach and participation, and safety. Barriers of the virtual interview format included reduced informal conversations, limited ability for applicants to explore programs at different locations, limited ability for programs to assess applicants' interest, technological issues, concern for interview integrity, limited non-verbal communication, and reduced networking. The most helpful media for applicants to learn about residency programs were program websites, the CaRMS/AFMC websites, and recruitment videos. Additionally, panel interviews were preferred by applicants for their ability to showcase themselves and build connections with multiple interviewers. Respondents provided recommendations regarding: (1) dissemination of program information, (2) the use of technology, and (3) the virtual interview format.

CONCLUSIONS

Perceptions of 2021 CaRMS R1 virtual interviews were favourable among applicants and interviewers. Recommendations from this study can help improve future iterations of virtual interviews.

Using Administrative Data in Primary Care to Evaluate the Effectiveness of a Continuing Professional Development Program Focused on the Management of Patients Living With Obesity

INTRODUCTION

There are guidelines for referral to medical and/or surgical weight loss interventions (MSWLI) in Ontario; however, only about one-third of eligible patients in our region are being referred for consideration of MSWLI.

METHODS

A planning committee, including a registered dietician, psychiatrist, endocrinologist, bariatric surgeon, family physician, and educationalists, developed an interdisciplinary continuing professional development (CPD) program focused on practical approaches to the management of patients living with obesity. The Kirkpatrick model was used to evaluate the educational outcomes of the CPD program specifically focusing on Level-2, -3, and -4 outcomes based on self-reported questionnaire and health administrative data.

RESULTS

Eighteen primary care providers from the CPD program agreed to participate in this study, and 16 primary care providers (89%) completed the postintervention questionnaire and granted us access to their MSWLI referral data; 94% of study participants reported changes to their knowledge, comfort, and confidence (Level 2), as well as expected change in their future behaviour (Level 3) following the CPD program. However, there was no change in Kirkpatrick Level-4 outcomes, despite more than 90% of participants indicating that they will be making changes to their practice after the program.

DISCUSSION

The CPD program in our study was overwhelmingly well received and participants reported knowledge (Level 2) and behavioural (Level 3) changes following participation; however, there was no detectable change in their clinical practice (Level 4). The methodology described in our proof-of-concept study can be modified and adopted to evaluate Level-4 outcomes in other studies of effectiveness of CPD interventions.

Internal medicine residents' and program directors' perception of virtual interviews during COVID-19: a national survey

PURPOSE

Due to the coronavirus disease 2019 pandemic, all Canadian Resident Matching Service interviews for internal medicine subspecialty programs were conducted virtually for the first time. This study explored the perceptions and experiences of internal medicine residents, subspecialty medicine program directors, and interviewers during virtual interviews.

METHODS

We invited all Canadian third-year IM residents, subspecialty program directors, and interviewers who participated in the 2020 medical subspecialty medicine interviews to complete a branching survey with a section for residents and one for program directors and interviewers. We distributed the anonymous survey after the submission of the rank order lists, to not affect residency match outcomes. Qualitative data were open-coded thematically and quantitative data were cleaned and then statistically analyzed using descriptive statistics and Analysis of Variance tests.

RESULTS

62 residents, 59 program directors, and 113 interviewers responded to the survey with representation from almost all Canadian medical faculties and medical subspecialties. Strengths of virtual interviews included reduced cost, stress, pandemic infection risk, and carbon footprint. Weaknesses of virtual interviews included decreased ability to connect personally and informally, and inability to tour medical facilities and cities. A majority of both resident respondents (59.6%) and program directors/interviewer respondents (54.6%) supported conducting interviews virtually in the future.

CONCLUSIONS

This study showed that the majority of both sampled residents and program directors/interviewers would prefer to conduct medicine subspecialty match interviews virtually in the future, and provides suggestions on how to improve the virtual interviews for the next iteration.

Systematic review support received and needed by researchers: a survey of libraries supporting Ontario medical schools

Introduction

Finding efficient ways to meet the growing demand for library systematic review support is imperative for facilitating the production of high-quality research. The objectives of this study were threefold: 1) to ascertain the systematic review support provided by health sciences libraries at Ontario medical schools and their affiliated hospitals, 2) to determine the perceived educational needs by researchers at these institutions, and 3) to assess the potential usefulness of freely available, online educational modules for researchers that discuss all stages of the systematic review process.

Methods

We conducted a cross-sectional survey in June and July of 2020. Data was analyzed and presented using median and interquartile range (IQR) for continuous measures, and in proportions for categorical measures.

Results

13 of 19 libraries invited provided usable data. Most libraries spent more time supporting systematic reviews via collaboration and participation than by providing educational support. The perceived needs of library users were contrary to the perceived gaps in researcher support provided by the library/institution. All libraries reported they would find freely available, online educational modules useful for training researchers.

Discussion

The next steps for our inter-professional research team will be to develop freely available, online education modules that introduce researchers to all stages of the systematic review process. These modules cannot replace the value that direct support from librarians, biostatisticians or methodology experts can provide, however, they may offer a more efficient way for libraries to familiarize researchers and trainees with best practices and universally accepted reporting guidelines for performing a high-quality review.

Environmental factors impacting the motivation to innovate: a systematic review

The environments where innovation occurs are often as varied as the areas of endeavors that aspiring innovators could pursue. This systematic review followed the guidelines of the Campbell Collaboration and PRISMA to consolidate the findings of 74 studies into the Expectancy-Value-Cost motivation theoretical framework as a means of usefully isolating for decision-makers the environmental factors that impact the motivation to innovate. The results of this review reveal that additional study of interdisciplinary samples is needed to gather deep narrative and case-driven data that considers the experiences of innovators in addition to organizations. Leaders, including decision-makers, teachers, and supervisors, can set a precedent for their learners and workers to use their past experiences and to feel safe to take intelligent risks and make reasonable mistakes in pursuit of innovating. Ensuring that project teams have a mix of experiences and backgrounds can make for more productive collaborations. Proactively addressing costs can increase workplaces' psychological safety and stability, which enables workers and learners to better focus on the endeavors at hand. The articles' evaluation illustrates that conversation about innovation promotion is dominated by business, which reduces the opportunity to learn from other innovation-driven disciplines or take truly interdisciplinary approaches.

Developing Academic Advisors and Competence Committees members: A community approach to developing CBME faculty leaders

INTRODUCTION

Implementing competency-based medical education (CBME) at the institutional level poses many challenges including having to rapidly enable faculty to be facilitators and champions of a new curriculum which utilizes feedback, coaching, and models of programmatic assessment. This study presents the necessary competencies required for Academic Advisors (AA) and Competence Committee (CC) members, as identified in the literature and as perceived by faculty members at Queen's University.

METHODS

This study integrated a review of available literature (n=26) yielding competencies that were reviewed by the authors followed by an external review consisting of CBME experts (n=5). These approved competencies were used in a cross-sectional community consultation survey distributed one year before (n=83) and one year after transitioning to CBME (n=144).

FINDINGS

Our newly identified competencies are a useful template for other institutions. Academic Advisor competencies focused on mentoring and coaching, whereas Competence Committee member's competencies focused on integrating assessments and institutional policies. Competency discrepancies between stakeholder groups existing before the transition had disappeared in the post-implementation sample.

CONCLUSIONS

We found value in taking an active community-based approach to developing and validating faculty leader competencies sooner rather than later when transitioning to CBME. The evolution of Competence Committees members and Academic Advisors requires the investment of specialized professional development and the sustained engagement of a collaborative community with shared concerns.

Assessing the quality of feedback to general internal medicine residents in a competency-based environment

CONSTRUCT

Competency Based Medical Education (CBME) is designed to use workplace-based assessment (WBA) tools to provide observed assessment and feedback on resident competence. Moreover, WBAs are expected to provide evidence beyond that of more traditional mid- or end-of-rotation assessments [e.g., In Training Evaluation Reports (ITERs)]. In this study, we investigated the quality of feedback in General Internal Medicine (GIM), by comparing WBA and ITER assessment tools.

BACKGROUND

WBAs are hypothesized to improve written and numerical feedback to support the development and documentation of competence. In this study, we investigated residents' and preceptors' perceptions of WBA validity, usability, and reliability and the extent to which WBAs differentiate residents' performance when compared to ITERs.

APPROACH

We used a mixed methods approach over a three-year period, including perspectives gathered from focus groups, interviews, along with numerical and narrative comparisons between WBA and ITERs in one GIM program.

RESULTS

Our quantitative analysis of feedback from seven residents' clinical assessments showed that overall rates of actionable feedback, for both ITERs and WBAs, were low (26%), with only 9% of the total providing an improvement strategy. The provision of quality feedback was not significantly different between tools; although WBAs provided more actionable feedback, ITERs provided more strategies. We found that residents and preceptors indicated the narrative component of feedback was more constructive and effective than numerical scores. Both groups perceived the focus on specific workplace-based feedback was more effective than ITERs.

CONCLUSIONS

Participants in this study viewed narrative, actionable, and specific feedback as essential, and an overall preference was found for written feedback over numerical assessments. However, our quantitative analyses showed that specific actionable feedback was rarely documented, despite finding an emphasis from both residents and preceptors of its importance for developing competency. Neither formative WBAs nor summative ITERs clearly provided better feedback, and both may still have a role in overall resident evaluation. Participant views differed in roles and responsibilities, with residents stating that preceptors should be responsible for initiating assessments and vice-versa. These results reveal an incongruence between resident and preceptor perceptions and practice around giving feedback and emphasize opportunities for programs adopting and implementing CBME to address how best to support residents and frontline clinical teachers.

Assessing the quality of feedback to general internal medicine residents in a competency-based environment

CONSTRUCT

Competency Based Medical Education (CBME) is designed to use workplace-based assessment (WBA) tools to provide observed assessment and feedback on resident competence. Moreover, WBAs are expected to provide evidence beyond that of more traditional mid- or end-of-rotation assessments [e.g., In Training Evaluation Reports (ITERs)]. In this study, we investigated the quality of feedback in General Internal Medicine (GIM), by comparing WBA and ITER assessment tools.

BACKGROUND

WBAs are hypothesized to improve written and numerical feedback to support the development and documentation of competence. In this study, we investigated residents' and preceptors' perceptions of WBA validity, usability, and reliability and the extent to which WBAs differentiate residents' performance when compared to ITERs.

APPROACH

We used a mixed methods approach over a three-year period, including perspectives gathered from focus groups, interviews, along with numerical and narrative comparisons between WBA and ITERs in one GIM program.

RESULTS

Our quantitative analysis of feedback from seven residents' clinical assessments showed that overall rates of actionable feedback, for both ITERs and WBAs, were low (26%), with only 9% of the total providing an improvement strategy. The provision of quality feedback was not significantly different between tools; although WBAs provided more actionable feedback, ITERs provided more strategies. We found that residents and preceptors indicated the narrative component of feedback was more constructive and effective than numerical scores. Both groups perceived the focus on specific workplace-based feedback was more effective than ITERs.

CONCLUSIONS

Participants in this study viewed narrative, actionable, and specific feedback as essential, and an overall preference was found for written feedback over numerical assessments. However, our quantitative analyses showed that specific actionable feedback was rarely documented, despite finding an emphasis from both residents and preceptors of its importance for developing competency. Neither formative WBAs nor summative ITERs clearly provided better feedback, and both may still have a role in overall resident evaluation. Participant views differed in roles and responsibilities, with residents stating that preceptors should be responsible for initiating assessments and vice-versa. These results reveal an incongruence between resident and preceptor perceptions and practice around giving feedback and emphasize opportunities for programs adopting and implementing CBME to address how best to support residents and frontline clinical teachers.