Living in a World of Change: Bridging the Gap From Competency-Based Medical Education Theory to Practice in Canada

The Surgical Education Checklist: A Novel Tool to Improve the Use of Entrustable Professional Activities in Operative Training in Competence by Design

OBJECTIVE

Surgical residents face significant challenges in meeting Entrustable Professional Activity (EPA) assessment requirements under the Competence By Design (CBD) framework, potentially impacting their surgical training and readiness for practice. This study seeks to assess the effectiveness of the Surgical Education Checklist (SEC) in increasing the proportion of completed EPA assessments within the Division of General Surgery.

DESIGN

This prospective cohort study implemented the SEC between January 1, 2023, and June 30, 2023. A pre and postchecklist survey collected experiences with the SEC and overall CBD framework. The primary outcome was the proportion of completed EPA assessments. Descriptive statistics were reported and subgroup analyses were performed based on resident postgraduate year (PGY).

SETTING

About 5 tertiary academic-affiliated teaching hospitals with McMaster University (Hamilton, Ontario) PARTICIPANTS: All general surgery residents and staff faculty were invited to utilize the survey, with an approximate sample size of 71 participants. The overall response rate was 34 (47.9%) for the prechecklist survey and 11 (15.5%) for the postchecklist survey.

RESULTS

The number of EPA assessments triggered increased from 485 to 639, with a significant increase in the absolute number of EPA assessments completed (400 vs 477, p < 0.01). There was an increase in the proportion of expired assessments (85 [17.5%] vs 162 [25.4%], p < 0.01) after the implementation of the SEC. In the subgroup analysis based on resident PGY, PGY1 to 3 general surgery residents triggered significantly more EPA assessments after the implementation of the SEC.

CONCLUSIONS

The SEC demonstrated its effectiveness as a perioperative prompt for triggering EPA assessments, particularly amongst junior general surgery residents, indicating its potential to facilitate early engagement with essential surgical training tasks. However, there remains gaps in follow-through and completion of triggered EPA assessments.

Competency based medical education implementation at the institutional level: A cross-discipline comparative program evaluation

INTRODUCTION

As an early adopter of competency-based medical education (CBME) our postgraduate institution was uniquely positioned to analyze implementation experience data across programs, while keeping institutional factors constant. We described participants' experiences related to CBME implementation across programs derived from early program evaluation efforts within our setting.

METHODS

This evaluation focused on eight residency programs at a medium-sized academic institution in Canada. Participants (n = 175) included program leaders, faculty, and residents. The study consisted of 3 phases: (1) describing intended implementation; (2) documenting enacted implementation; and (3) comparing intended with enacted implementation to inform adaptations. Each program's findings were summarized in technical reports which were then analyzed thematically. Cross program data were organized by themes.

RESULTS

Six themes were identified. All groups emphasized the need for ongoing refinement of CBME resulting from shared tensions such as increased assessment burden. However, there were some disparate CBME-related experiences between programs such as the experience with entrustable professional activities, the interpretation of retrospective entrustment anchors, and quality of feedback.

CONCLUSION

We detected several cross-program successes and important challenges related to CBME. Our experience can inform other programs engaging in implementation and evaluation of CBME.

Fixing disconnects: Exploring the emergence of principled adaptations in a competency-based curriculum

PURPOSE

Competency-based medical education (CBME) promises to improve medical education through curricular reforms to support learner development. This intention may be at risk in the case of a Canadian approach to CBME called Competence by Design (CBD), since there have been negative impacts on residents. According to Joseph Schwab, teachers, learners and milieu must be included in the process of curriculum-making to prevent misalignments between intended values and practice. This study considered what can be learned from the process of designing, enacting and adapting CBD to better support learners.

METHODS

This qualitative study explored the making of CBD through the perspectives of implementation leads (N = 18) at national, institutional and programme levels. A sociomaterial orientation to agency in curriculum-making guided the inductive approach to interviewing and analysis in phase one. A deductive analysis in phase two applied Schwab's theory to further understand sources of misalignments and the purpose of adaptive responses.

RESULTS

Misalignments occurred when the needs of teachers, learners and milieu were initially underestimated in the process of curriculum-making, disconnecting assessment practices from experiences of teaching, learning and entrustment. While technical and structural issues posed significant constraints on agency, some implementation leads were able to make changes to the curriculum or context to fix the disconnects. We identified six purposes for principled adaptations to align with CBME values of responsive teaching, individualised learning and meaningful entrustment.

CONCLUSION

Collectively, the adaptations we characterise demonstrate constructive alignment, a foundational principle of CBME in which assessment and teaching work together to support learning. This study proposes a model for making context-shaped, values-based adaptations to CBME to achieve its promise.

Was it all worth it? A graduating resident perspective on CBME

BACKGROUND

Our institution simultaneously transitioned all postgraduate specialty training programs to competency-based medical education (CBME) curricula. We explored experiences of CBME-trained residents graduating from five-year programs to inform the continued evolution of CBME in Canada.

METHODS

We utilized qualitative description to explore residents' experiences and inform continued CBME improvement. Data were collected from fifteen residents from various specialties through focus groups, interviews, and written responses. The data were analyzed inductively, using conventional content analysis.

RESULTS

We identified five overarching themes. Three themes provided insight into residents' experiences with CBME, describing discrepancies between the intentions of CBME and how it was enacted, challenges with implementation, and variation in residents' experiences. Two themes - adaptations and recommendations - could inform meaningful refinements for CBME going forward.

CONCLUSIONS

Residents graduating from CBME training programs offered a balanced perspective, including criticism and recognition of the potential value of CBME when implemented as intended. Their experiences provide a better understanding of residents' needs within CBME curricula, including greater balance and flexibility within programs of assessment and curricula. Many challenges that residents faced with CBME could be alleviated by greater accountability at program, institutional, and national levels. We conclude with actionable recommendations for addressing residents' needs in CBME.

Laying train tracks en route: How institutional education leaders navigate complexity during mandated curriculum change

INTRODUCTION

Institutional education leaders serve key roles in leading major curricular change within residency education, yet little is known about how they accomplish these goals on the ground. Change management principles have predominantly been developed and described in the hierarchical context of management science and corporate settings. However, the non-hierarchical, complex and adaptive features of health professions education may render these traditional change management models inadequate. We explored how institutional educational leaders navigate the complex residency education system in implementing a major curricular change.

METHODS

Using constructivist grounded theory, we conducted and iteratively analysed semi-structured interviews with 11 institutional education leaders from across Canada who were responsible for leading the nationally mandated curricular change to competency-based residency education. Thematic analysis was performed iteratively using constant comparison.

RESULTS

Leaders managing the change process focused on two priorities: steering the direction of the change process as it evolved and maintaining the momentum amongst stakeholders to move forward steadily. Four common threats and opportunities impacted the focus on direction and momentum: multiplicity of contexts, innovation, resistance and distractions. In response, leaders utilised various tactics and harnessed diverse leadership styles to manage these challenges accordingly.

CONCLUSIONS

We identified a change framework that offers a more contextually nuanced understanding of curricular change in residency education that has not been described in the change management literature generated by the management sector. Institutional education leaders focused on maintaining the direction and momentum, while constantly assessing and adapting to evolving, uncertain and complex conditions. Our findings provide a simple and practical foundation to support leadership education in curricular change as well as researchers in developing further change theories in complex adaptive health professions education systems.

Relevance of Competency Based Education for Architectural Education in India

Background

A holistic architectural education is the culmination of learning knowledge, skills, attitudes, and values, which eventually reflects in the quality of graduates. Even though different schools of thought have made various kinds of qualitative contributions towards the evolution of architectural education in India, it has largely been dominated by the quantitative and technical aspects of its regulating framework. Architects engage with the demanding contradictions between responsibilities of an ethical nature, the dynamic challenges of practice, and the intricacies of architectural imagination. The aesthetical and imaginative foundations of the field make it incumbent upon the architects to possess a balance of ecumenical proficiencies for accountability and personalization. The purpose of the study is to identify relevant attributes of Competency Based Education (CBE) that can be adopted for architectural education in India.

Methods

This research follows a narrative review approach and a descriptive-analytic method to broadly understand the attributes of CBE and its potential relevance to architectural education in India. 323 articles were searched on various search strings based on their relevance to the inquiry. 76 documents written in English language were included and appraised through the Scale of the Assessment for Narrative Review Articles (SANRA) tool to avoid any risk of bias. The PRISMA 2020 checklist and flow diagram has been used to report the findings of this narrative review.

Result

The study identifies eight critical parameters of CBE with respect to its definitions, origins, transitions, regulatory environment, characteristics, approaches and implications on teaching-learning, frameworks and models of assessment; and challenges, which makes a case for the relevance of CBE for architectural education in India, which hasn't been explored yet.

Conclusion

The broader expectations of 'being competent' can be addressed through a conscious adoption of strategies of relevant attributes of CBE which can encourage building attitudes and temperament for life-long learning.

Medical training to effectively support patients who use substances across practice settings: a scoping review of recommended competencies

Background

The responsibility for addressing the healthcare needs of PWUS is the responsibility of all physicians. Within the healthcare system, research consistently reveals inequitable experiences in healthcare with people who use substances (PWUS) reporting stigmatization, marginalization, and a lack of compassion.

Objectives

The aim of this scoping review was to find and describe competencies being taught, developed, and fostered within medical education and then to provide recommendations to improve care for this population of patients.

Results

Nineteen articles were included. Recommended knowledge competencies tend to promote understanding neurophysiological changes caused by substances, alongside knowing how to evaluate of 'risky' behaviours. Commonly recommended skills relate to the screening and management of substance use disorders. Recommended attitude competencies include identifying personal bias and establishing a patient-centered culture among practice teams. The disease model of addiction informed all papers, with no acknowledgement of potential beneficial or non-problematic experiences of substance use. To enhance knowledge-type competencies, medical education programs are advised to include addiction specialists as educators and prevent stigmatization through the hidden curriculum.

Conclusion

To reduce experiences of stigmatization and marginalization among patients who use illicit substances and to improve quality of care, knowledge, skills, and attitudes competencies can be more effectively taught in medical education programs.Résumé.

Picking Apart a Program Evaluation Committee: A Multiple Case Study Characterizing Primary Care Residency Program Evaluation Committee Structure, Program Improvement, and Outcomes

BACKGROUND

As of 2014, the Accreditation Council for Graduate Medical Education (ACGME) mandates initiating a Program Evaluation Committee (PEC) to guide ongoing program improvement. However, little guidance nor published reports exist about how individual PECs have undertaken this mandate.

OBJECTIVE

To explore how four primary care residency PECs configure their committees, review program goals and undertake program evaluation and improvement.

METHODS

We conducted a multiple case study between December 2022 and April 2023 of four purposively selected primary care residencies (e.g., family medicine, pediatrics, internal medicine). Data sources included semi-structured interviews with four PEC members per program and diverse program artifacts. Using a constructivist approach, we utilized qualitative coding to analyze participant interviews and content analysis for program artifacts. We then used coded transcripts and artifacts to construct logic models for each program guided by a systems theory lens.  Results: Programs adapt their PEC structure, execution, and outcomes to meet short- and long-term needs based on organizational and program-unique factors such as size and local practices. They relied on multiple data sources and sought diverse stakeholder participation to complete program evaluation and improvement. Identified deficiencies were often categorized as internal versus external to delineate PEC responsibility, boundaries, and feasibility of interventions.

CONCLUSION

The broad guidance provided by the ACGME for PEC configuration allows programs to adapt the committee based on individual needs. However, further instruction on program evaluation and organizational change principles would augment existing PEC efforts.

Effect evaluation of competency-based education (CBE) combined with multi-disciplinary team (MDT) teaching mode in respiratory rehabilitation nursing teaching：A randomized controlled trial

AIM

To explore the application effect of competency-based education (CBE) combined with multi-disciplinary team (MDT) teaching mode in respiratory rehabilitation nursing teaching.

BACKGROUND

Respiratory rehabilitation, as an important non drug treatment for chronic respiratory disease, started late in China, with low implementation rate in hospitals and insufficient knowledge of nursing staff. Therefore, it is urgent to cultivate high professional level nurses with strong professional ability.

DESIGN

A randomized controlled trial.

METHODS

We selected nurses from the respiratory and critical care department at a Grade III, Level A hospital in Beijing, who participated in the respiratory rehabilitation training program between March 2020 and August 2022, as our study participants. Participants were grouped using a random number method. Nurses who participated in the study from March 2020 to May 2021 were set as the control group, and the nurses who participated in the study from June 2021 to August 2022 were set as the test group. The control group used traditional teaching methods, and the test group used the CBE combined MDT teaching model. At the end of the test, the theory, operating skills, satisfaction and core competence were assessed.

RESULTS

After training, the theoretical examination score of the test group (93.71 ± 1.94) was higher than that of the control group (92.37 ± 1.92), the operational examination score of the test group (93.11 ± 2.12) was higher than that of the control group (91.61 ± 1.93), the overall teaching satisfaction of the test group (4.45 ± 0.50) was higher than that of the control group (4.13 ± 0.57), and the total score of the core competence of the test group (148.73 ± 7.08) was higher than that of the control group (141.02 ± 6.41). The difference between the two groups was statistically significant (P<0.05).

CONCLUSION

CBE combined with MDT teaching mode has a good effect in Pulmonary and Critical Care Medicine (PCCM) respiratory rehabilitation nursing teaching.

Lessons learned spanning 17 years of experience with three consecutive nationwide competency based medical education training plans

Introduction

Curricula for postgraduate medical education have transformed since the introduction of competency based medical education (CBME). Postgraduate training plans offer broader training with different competencies and an outcome-based approach, in addition to the medical technical aspects of training. However, CBME also has its challenges. Over the past years, critical views have been shared on the potential drawbacks of CBME, such as assessment burden and conflicts with practicality in the workplace. Recent studies identified a need for a better understanding of how the evolving concept of CBME has been translated to curriculum design and implemented in the practice of postgraduate training. The aim of this study was to describe the development of CBME translations to curriculum design, based on three consecutive postgraduate training programs spanning 17 years.

Method

We performed a document analysis of three consecutive Dutch gynecology and obstetrics training plans that were implemented in 2005, 2013, and 2021. We used template analysis to identify changes over time.

Results

Over time, CBME-based curriculum design changed in several domains. Assessment changed from a model with a focus on summative decision to one with an emphasis on formative, low-stakes assessments aimed at supporting learning. The training plans evolved in parallel to evolving educational insights, e.g., by placing increasing emphasis on personal development. The curricula focused on a competency-based concept by introducing training modules and personalized authorization based on feedback rather than on a set duration of internships. There was increasing freedom in personalized training trajectories in the training plans, together with increasing trust towards the resident.

Conclusion

The way CBME was translated into training plans has evolved in the course of 17 years of experience with CMBE-based education. The main areas of change were the structure of the training plans, which became increasingly open, the degree to which learning outcomes were mandatory or not, and the way these outcomes were assessed.

Education modalities for serious illness communication training: A scoping review on the impact on clinician behavior and patient outcomes

BACKGROUND

Several clinician training interventions have been developed in the past decade to address serious illness communication. While numerous studies report on clinician attitudes and confidence, little is reported on individual education modalities and their impact on actual behavior change and patient outcomes.

AIM

To examine what is known about the education modalities used in serious illness communication training and their impact on clinician behaviors and patient outcomes.

DESIGN

A scoping review using the Joanna Briggs Methods Manual for Scoping Reviews was conducted to examine studies measuring clinician behaviors or patient outcomes.

DATA SOURCES

Ovid MEDLINE and EMBASE databases were searched for English-language studies published between January 2011 and March 2023.

RESULTS

The search identified 1317 articles: 76 met inclusion criteria describing 64 unique interventions. Common education modalities used were: single workshop (n = 29), multiple workshops (n = 11), single workshop with coaching (n = 7), and multiple workshops with coaching (n = 5); though they were inconsistently structured. Studies reporting improved clinician skills tended to be in simulation settings with neither clinical practice nor patient outcomes explored. While some studies reported behavior changes or improved patient outcomes, they did not necessarily confirm improvements in clinician skills. As multiple modalities were commonly used and often embedded within quality improvement initiatives, the impact of individual modalities could not be determined.

CONCLUSION

This scoping review of serious illness communication interventions found heterogeneity among education modalities used and limited evidence supporting their effectiveness in impacting patient-centered outcomes and long-term clinician skill acquisition. Well-defined educational modalities and consistent measures of behavior change and standard patient-centered outcomes are needed.

Using a rapid-cycle approach to evaluate implementation of competency-based medical education in ophthalmology

OBJECTIVE

As competency-based medical education is being implemented across Canada, there is an increasing need to evaluate the progress to date, including identification of strengths and weaknesses, to inform program development. Ophthalmology is preparing for a national launch in coming years. The purpose of this study was to describe key stakeholders' lived experiences in the competency-based medical education foundation-of-discipline stage in one ophthalmology department.

DESIGN

Using a case-study approach, a qualitative rapid-cycle evaluation was conducted during the 2018-2019 academic year.

PARTICIPANTS

Residents, faculty, academic advisors, competence committee members, the program director, the program administrator, and the educational consultant were invited to participate in the program evaluation.

METHODS

The rapid-cycle evaluation consisted of 2 evaluation cycles, with the first round of interviews and focus groups occurring in October 2018 and the second round in March 2019. Recommendations were implemented in November 2019 and June 2019. All data were analyzed thematically using NVivo.

RESULTS

Three main themes emerged across all data sets: developing a shared understanding (e.g., role expectations and changes to assessment), refining assessment processes and tools (e.g., the need for streamlining and clarification), and feedback (e.g., perceived benefits and value of narrative comments).

CONCLUSIONS

Exploring lived experiences in this study resulted in positive and immediate improvements to the residency program. The recommendations and approach will be useful to other Canadian departments and institutions as they prepare for Competence by Design.

Exploring stakeholder perspectives regarding the implementation of competency-based medical education: a qualitative descriptive study

Introduction

Competency-based medical education (CBME) offers perceived advantages and benefits for postgraduate medical education (PGME) and the training of competent physicians. The purpose of our study was to gain insights from those involved in implementing CBME in two residency programs to inform ongoing implementation practices.

Methods

We conducted a qualitative descriptive study to explore the perspectives of multiple stakeholders involved in the implementation of CBME in two residency programs (the first cohort) to launch the Royal College's Competence by Design model at one Canadian university. Semi-structured interviews were conducted with 17 participants across six stakeholder groups including residents, department chairs, program directors, faculty, medical educators, and program administrators. Data collection and analysis were iterative and reflexive to enhance the authenticity of the results.

Results

The participants' perspectives organized around three key themes including: a) contextualizing curriculum and assessment practices with educational goals of CBME, b) coordinating new administrative requirements to support implementation, and c) adaptability toward a competency-based program structure, each with sub-themes.

Conclusion

By eliciting the perspectives of different stakeholder groups who experienced the implementation processes, we developed a common understanding regarding facilitators and challenges for program directors, program administrators and educational leaders across PGME. Results from our study contribute to the scholarly conversation regarding the key aspects related to CBME implementation and serve to inform its ongoing development and application in various educational contexts.

Unintended consequences of technology in competency-based education: a qualitative study of lessons learned in an OtoHNS program

BACKGROUND

Formative feedback and entrustment ratings on assessments of entrustable professional activities (EPAs) are intended to support learner self-regulation and inform entrustment decisions in competency-based medical education. Technology platforms have been developed to facilitate these goals, but little is known about their effects on these new assessment practices. This study investigates how users interacted with an e-portfolio in an OtoHNS surgery program transitioning to a Canadian approach to competency-based assessment, Competence by Design.

METHODS

We employed a sociomaterial perspective on technology and grounded theory methods of iterative data collection and analysis to study this OtoHNS program's use of an e-portfolio for assessment purposes. All residents (n = 14) and competency committee members (n = 7) participated in the study; data included feedback in resident portfolios, observation of use of the e-portfolio in a competency committee meeting, and a focus group with residents to explore how they used the e-portfolio and visualize interfaces that would better meet their needs.

RESULTS

Use of the e-portfolio to document, access, and interpret assessment data was problematic for both residents and faculty, but the residents faced more challenges. While faculty were slowed in making entrustment decisions, formative assessments were not actionable for residents. Workarounds to these barriers resulted in a "numbers game" residents played to acquire EPAs. Themes prioritized needs for searchable, contextual, visual, and mobile aspects of technology design to support use of assessment data for resident learning.

CONCLUSION

Best practices of technology design begin by understanding user needs. Insights from this study support recommendations for improved technology design centred on learner needs to provide OtoHNS residents a more formative experience of competency-based training.

Overcoming the barriers to implementation of competence-based medical education in post-graduate medical education: a narrative literature review

To ensure that residents are equipped with the necessary skills for practice, competence-based medical education (CBME) represents a transformative change in postgraduate medical education, which is being progressively introduced across Canadian specialty residency programs. Successful implementation will require adjustments to curriculum, assessment, and evaluation, with careful attention to the unique needs in the local context, including resident and faculty development. This narrative review of the literature aimed to determine the potential barriers to the successful implementation of CBME and the strategies by which they can be addressed, with a specific consideration of the author's program in pediatrics in Manitoba. Eleven articles were identified with a specific focus on the implementation of CBME in the post-graduate setting, and 10 were included in the review after critical appraisal. Three key themes emerged from the articles: the value of broad stakeholder engagement and leadership, the importance of faculty and resident development, and the development of specific support systems for the educational curriculum. Different strategies were considered and contrasted for addressing these important themes. This review provides important insights and practical approaches to the barriers that should be useful as programs prepare for the implementation of CBME.

Learning from the implementation of clinical empathy training: an explorative qualitative study in search of the barriers and facilitators

BACKGROUND

Amid concerns about the decline of empathy during the clinical training of medical clerks, evidence that empathy improves patient outcomes suggests some potential for teaching empathy in ways that will affect the knowledge, attitude and behaviour of medical clerks. This potential alone cannot, however, guarantee the success of educational innovations to introduce empathy to the medical curriculum. This research aims to identify the barriers and facilitators of the implementation of a specific clinical initiative to enhance the empathy skills of clerks, namely the training of clerks to act as a 'MedGezel' or 'medical coach'.

METHOD

We conducted an explorative qualitative study based on interview data collected and analyzed using reflexive thematic analysis and the readiness for change theory. We conducted semi-structured interviews with relevant stakeholders in this particular qualitative study. Thematic analysis was based on open and axial coding using ATLAS.ti 9, which facilitated the emergence of common themes of interest and meaning for the study.

RESULTS

A total of 13 relevant stakeholders participated as interviewees in our study. The data was collected from April to June 2021. Our analysis generated 6 main themes which can provide insights into why the implementation of the MedGezel educational innovation failed so far. The following themes emerged: the case for change: why change?; practical necessity; leadership; management and resources; staff culture; and alignment with the corporate strategy.

DISCUSSION

The implementation failure can be partially explained as resulting from the personal attitudes and choices of participants, who struggled to reconcile a vision that they liked with side effects that they feared. While participants repeatedly mentioned management and leadership issues, these organizational issues seemed less important as they could be easily resolved in practice. What was more important and fatal for the initiative was its lack of alignment with staff culture, despite its alignment with corporate strategy.

CONCLUSION

This investigation into the barriers and facilitators influencing the implementation of the MedGezel program identified 6 explanatory themes, the most impactful one being staff culture.

Exploring the relationship between social accountability and competency-based medical education: A narrative review

BACKGROUND

Social accountability (SA), a quintessential goal of medical education, has been discussed as a precipitant for the transition toward competency-based medical education (CBME). However, the relationship between SA and CBME remains unclear. A narrative review was conducted to systematically explore the relationship between SA and CBME as described in the literature.

METHODS

Electronic databases, select journals, and medical education organizations were systematically searched. 363 titles and abstracts were screened and 147 full texts were reviewed. The salient text was extracted from 36 records, which were then inductively coded before narrative synthesis and interpretation.

RESULTS

The relationship between SA and CBME was described in three manners: (1) CBME as a natural driver of SA where CBME was perceived to be inherently socially accountable, (2) CBME as an opportunistic mechanism for actively changing medical training to better meet standards of SA, and (3) CBME as a tool to measure SA relating to measurable outcomes data provided by CBME.

CONCLUSION

CBME has theoretical potential to assist programs in becoming more socially accountable if the communities they serve are considered key stakeholders in the design, implementation, and evaluation. A paucity of evidence remains which provides empirical evidence of SA within programs that have implemented CBME.

Creating a Competency-Based Medical Education Curriculum for Canadian Diagnostic Radiology Residency (Queen's Fundamental Innovations in Residency Education)-Part 2: Core of Discipline Stage

PURPOSE

All postgraduate residency programs in Canada are transitioning to a competency-based medical education (CBME) model divided into 4 stages of training. Queen's University has been the first Canadian institution to mandate transitioning to CBME across all residency programs, including Diagnostic Radiology. This study describes the implementation of CBME with a focus on the third developmental stage, Core of Discipline, in the Diagnostic Radiology residency program at Queen's University. We describe strategies applied and challenges encountered during the adoption and implementation process in order to inform the development of other CBME residency programs in Diagnostic Radiology.

METHODS

At Queen's University, the Core of Discipline stage was developed using the Royal College of Physicians and Surgeons of Canada's (RCPSC) competence continuum guidelines and the CanMEDS framework to create radiology-specific entrustable professional activities (EPAs) and milestones for assessment. New committees, administrative positions, and assessment strategies were created to develop these assessment guidelines. Currently, 2 cohorts of residents (n = 6) are enrolled in the Core of Discipline stage.

RESULTS

EPAs, milestones, and methods of evaluation for the Core of Discipline stage are described. Opportunities during implementation included tracking progress toward educational objectives and increased mentorship. Challenges included difficulty meeting procedural volume requirements, inconsistent procedural tracking, improving feedback mechanisms, and administrative burden.

CONCLUSION

The transition to a competency-based curriculum in an academic Diagnostic Radiology residency program is significantly resource and time intensive. This report describes challenges faced in developing the Core of Discipline stage and potential solutions to facilitate this process.

Developing a Competency Framework of Interprofessional Occupational Health Team: A First Step to Interprofessional Education in Occupational Health Field

OBJECTIVE

The present study aimed to develop a competency framework for interprofessional collaboration in an occupational health team.

METHODS

This study was carried out in three stages. In the first stage, the viewpoints of different stakeholders on the essential competencies of interprofessional team members in occupational health services (OHS) were assessed. In the second stage, extracted codes were converted into competencies in expert panels. After that, the extracted items were categorized by inductive content analysis approach into competency framework. In the third stage, the content validity of interprofessional competencies in OHS was evaluated.

RESULTS

The competency framework was finalized in six competency domains including prevention, occupational disease management, occupational health assessments, inter-disciplinary and interprofessional collaboration, education, and evidence-based practice and professionalism.

CONCLUSION

The framework could be developed in interprofessional education in OHS based on competency-based education approach.

Can case-based discussions in a group setting be used to assess residents' clinical skills?

OBJECTIVES

The purpose of this study was to explore residents' and assessors' perception of a new group assessment concept.

METHODS

This qualitative study consists of observations of four group assessment sessions, followed by semi-structured interviews with six residents and four assessors (specialists in internal medicine), who all volunteered to be interviewed. All residents at a medical department (eleven to fifteen each time) and four assessors participated in four group assessments, where the residents' clinical skills were assessed through case-based discussions. An external consultant (an anthropologist) performed the observations and the interviews. Notes from the observations and the interviews were analyzed using an inductive approach.

RESULTS

Eight of the ten interviewed participants preferred group assessment to individual assessment. Results from the interviews suggested that the group assessments were more consistent and that the level of discussion was perceived to be higher in the group discussions compared to the one-to-one discussions. All residents indicated that they had acquired new knowledge during their assessment and reported having learned from listening to the assessment of their peers. Assessors similarly reported gaining new knowledge.

CONCLUSIONS

The residents and assessors expressed very favourable attitudes toward the new group assessment concept. The assessment process was perceived to be higher in quality and more consistent, contributing to learning for all participating doctors in the department. Group assessment is feasible and acceptable, and provides a promising tool for assessment of clinical skills in the future.

Translating outcome frameworks to assessment programmes: Implications for validity

OBJECTIVES

Competency-based medical education (CBME) requires that educators structure assessment of clinical competence using outcome frameworks. Although these frameworks may serve some outcomes well (e.g. represent eventual practice), translating these into workplace-based assessment plans may undermine validity and, therefore, trustworthiness of assessment decisions due to a number of competing factors that may not always be visible or their impact knowable. Explored here is the translation process from outcome framework to formative and summative assessment plans in postgraduate medical education (PGME) in three Canadian universities.

METHODS

We conducted a qualitative study involving in-depth semi-structured interviews with leaders of PGME programmes involved in assessment and/or CBME implementation, with a focus on their assessment-based translational activities and evaluation strategies. Interviews were informed by Callon's theory of translation. Our analytical strategy involved directed content analysis, allowing us to be guided by Kane's validity framework, whilst still participating in open coding and analytical memo taking. We then engaged in axial coding to systematically explore themes across the dataset, various situations and our conceptual framework.

RESULTS

Twenty-four interviews were conducted involving 15 specialties across three universities. Our results suggest: (i) using outcomes frameworks for assessment is necessary for good assessment but are also viewed as incomplete constructs; (ii) there are a number of social and practical negotiations with competing factors that displace validity as a core influencer in assessment planning, including implementation, accreditation and technology; and (iii) validity exists as threatened, uncertain and assumed due to a number of unchecked assumptions and reliance on surrogates.

CONCLUSIONS

Translational processes in CBME involve negotiating with numerous influencing actors and institutions that, from an assessment perspective, provide challenges for assessment scientists, institutions and educators to contend with. These processes are challenging validity as a core element of assessment designs. Educators must reconcile these influences when preparing for or structuring validity arguments.