1
|
Wong L, Sacoransky E, Hopman W, Islam O, Chung AD, Kwan BYM. Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education. MEDICAL EDUCATION ONLINE 2024; 29:2357412. [PMID: 38810150 PMCID: PMC11138222 DOI: 10.1080/10872981.2024.2357412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 05/15/2024] [Indexed: 05/31/2024]
Abstract
INTRODUCTION Since 2022, all Canadian post-graduate medical programs have transitioned to a Competence by Design (CBD) model within a Competency-Based Medical Education (CBME) framework. The CBME model emphasized more frequent, formative assessment of residents to evaluate their progress towards predefined competencies in comparison to traditional medical education models. Faculty members therefore have increased responsibility for providing assessments to residents on a more regular basis, which has associated challenges. Our study explores faculty assessment behaviours within the CBD framework and assesses their openness to opportunities aimed at improving the quality of written feedback. Specifically, we explore faculty's receptiveness to routine metric performance reports that offer comprehensive feedback on their assessment patterns. METHODS Online surveys were distributed to all 28 radiology faculty at Queen's University. Data were collected on demographics, feedback practices, motivations for improving the teacher-learner feedback exchange, and openness to metric performance reports and quality improvement measures. Following descriptive statistics, unpaired t-tests and one-way analysis of variance were conducted to compare groups based on experience and subspecialty. RESULTS The response rate was 89% (25/28 faculty). 56% of faculty were likely to complete evaluations after working with a resident. Regarding the degree to which faculty felt written feedback is important, 62% found it at least moderately important. A majority (67%) believed that performance reports could influence their evaluation approach, with volume of written feedback being the most likely to change. Faculty expressed interest in feedback-focused development opportunities (67%), favouring Grand Rounds and workshops. CONCLUSION Assessment of preceptor perceptions reveals that faculty recognize the importance of offering high-quality written feedback to learners. Faculty openness to quality improvement interventions for curricular reform relies on having sufficient time, knowledge, and skills for effective assessments. This suggests that integrating routine performance metrics into faculty assessments could serve as a catalyst for enhancing future feedback quality.
Collapse
Affiliation(s)
- Laura Wong
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Ethan Sacoransky
- School of Medicine, Faculty of Health Sciences, Queen’s University, Kingston, ON, Canada
| | - Wilma Hopman
- Kingston General Hospital Research Institute, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Omar Islam
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Andrew D. Chung
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| | - Benjamin Y. M. Kwan
- Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston, ON, Canada
| |
Collapse
|
2
|
de Oliveira Filho GR, Soares Garcia JH. The Accuracy of the Learning-Curve Cumulative Sum Method in Assessing Brachial Plexus Block Competency. Anesth Analg 2024:00000539-990000000-00838. [PMID: 38861983 DOI: 10.1213/ane.0000000000006928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2024]
Abstract
BACKGROUND The learning-curve cumulative sum method (LC-CUSUM) and its risk-adjusted form (RA-LC-CUSUM) have been proposed as performance-monitoring methods to assess competency during the learning phase of procedural skills. However, scarce data exist about the method's accuracy. This study aimed to compare the accuracy of LC-CUSUM forms using historical data consisting of sequences of successes and failures in brachial plexus blocks (BPBs) performed by anesthesia residents. METHODS Using historical data from 1713 BPB performed by 32 anesthesia residents, individual learning curves were constructed using the LC-CUSUM and RA-LC-CUSUM methods. A multilevel logistic regression model predicted the procedure-specific risk of failure incorporated in the RA-LC-CUSUM calculations. Competency was defined as a maximum 15% cumulative failure rate and was used as the reference for determining the accuracy of both methods. RESULTS According to the LC-CUSUM method, 22 residents (84.61%) attained competency after a median of 18.5 blocks (interquartile range [IQR], 14-23), while the RA-LC-CUSUM assigned competency to 20 residents (76.92%) after a median of 17.5 blocks (IQR, 14-25, P = .001). The median failure rate at reaching competency was 6.5% (4%-9.75%) under the LC-CUSUM and 6.5% (4%-9%) for the RA-LC-CUSUM method (P = .37). The sensitivity of the LC-CUSUM (85%; 95% confidence interval [CI], 71%-98%) was similar to the RA-LC-CUSUM method (77%; 95% CI, 61%-93%; P = .15). Identical specificity values were found for both methods (67%; 95% CI, 29%-100%, P = 1). CONCLUSIONS The LC-CUSUM and RA-LC-CUSUM methods were associated with substantial false-positive and false-negative rates. Also, small lower limits for the 95% CIs around the accuracy measures were observed, indicating that the methods may be inaccurate for high-stakes decisions about resident competency at BPBs.
Collapse
|
3
|
Go MR, Traugott AL, Ejaz A, Collins C, Harzman AE, Ellison EC, Chen XP. Measuring Chief Resident Skill and Entrustment Progression in An Operative Coaching Program: Four Years' Experience. JOURNAL OF SURGICAL EDUCATION 2024; 81:457-464. [PMID: 38388313 DOI: 10.1016/j.jsurg.2023.12.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 11/27/2023] [Accepted: 12/22/2023] [Indexed: 02/24/2024]
Abstract
OBJECTIVE Operative coaching (OC) may facilitate improvement of surgery residents' competencies by optimizing learning and teaching. We investigated how residents' operative skills and prospective entrustment (PE) progress throughout the chief year in our OC program, how OC is perceived by participants, and how OC may facilitate learning and teaching. DESIGN, SETTING, AND PARTICIPANTS This is a mixed-methods study conducted within the Ohio State University Wexner Medical Center General Surgery residency. Validated performance evaluations with procedural-specific skill, general skill (GS), step-specific guidance required (SSG) (an autonomy measure), and PE measures completed by chiefs, faculty coaches, and attending surgeons from 7/2018 to 6/2022 were reviewed. We also interviewed OC participants to understand their experience. Descriptive statistical and qualitative content analysis were applied. RESULTS 441 evaluations from 147 OC cases completed by 22 chiefs, 5 faculty coaches, and 24 attendings were included. Overall, resident GS (p = 0.036), SSG (p = 0.023), and PE (p = 0.002) significantly improved throughout the year. PE significantly correlated (all p < 0.0001) with SSG (r = 0.73), followed by procedural-specific skill (r = 0.59), then GS (r = 0.57). On average, chiefs underestimated their surgical skills while attendings overestimated autonomy they permitted to residents. Chiefs, coaches, and attendings reached consensus on chiefs' PE upon graduation. Five graduated chiefs and 5 attendings were interviewed. Chiefs described OC as effective in improving their self-regulated learning and particularly valued 3 OC elements: neutral authentic feedback, third-party real-time observation, and actionable feedback. Attendings noted OC promoted their engagement in skills assessment and teaching. CONCLUSIONS Our findings suggest chief residents' skills, autonomy, and PE progress steadily along their OC journey. Despite differences in residents', coaches', and attendings' perceptions of skill, measures of autonomy reliably correlate with entrustment. OC promotes resident learning, faculty teaching, and assessment of resident skills, autonomy, and PE in the OR.
Collapse
Affiliation(s)
- Michael R Go
- Department of Surgery, The Ohio State University, Columbus, Ohio.
| | - Amber L Traugott
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Aslam Ejaz
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Courtney Collins
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | - Alan E Harzman
- Department of Surgery, The Ohio State University, Columbus, Ohio
| | | | | |
Collapse
|
4
|
Millar JK, Matusko N, Evans J, Baker SJ, Lindeman B, Jung S, Minter RM, Weinstein E, Goodstein F, Cook MR, Brasel KJ, Sandhu G. Faculty Entrustment and Resident Entrustability. JAMA Surg 2024; 159:277-285. [PMID: 38198146 PMCID: PMC10782383 DOI: 10.1001/jamasurg.2023.6915] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 09/17/2023] [Indexed: 01/11/2024]
Abstract
Importance As the surgical education paradigm transitions to entrustable professional activities, a better understanding of the factors associated with resident entrustability are needed. Previous work has demonstrated intraoperative faculty entrustment to be associated with resident entrustability. However, larger studies are needed to understand if this association is present across various surgical training programs. Objective To assess intraoperative faculty-resident behaviors and determine if faculty entrustment is associated with resident entrustability across 4 university-based surgical training programs. Design, Setting, and Participants This cross-sectional study was conducted at 4 university-based surgical training programs from October 2018 to May 2022. OpTrust, a validated tool designed to assess both intraoperative faculty entrustment and resident entrustability behaviors independently, was used to assess faculty-resident interactions. A total of 94 faculty and 129 residents were observed. Purposeful sampling was used to create variation in type of operation performed, case difficulty, faculty-resident pairings, faculty experience, and resident training level. Main Outcomes and Measures Observed resident entrustability scores (scale 1-4, with 4 indicating full entrustability) were compared with reported measures (faculty level, case difficulty, resident postgraduate year [PGY], resident gender, observation month) and observed faculty entrustment scores (scale 1-4, with 4 indicating full entrustment). Path analysis was used to explore direct and indirect effects of the predictors. Associations between resident entrustability and faculty entrustment scores were assessed by pairwise Pearson correlation coefficients. Results A total of 338 cases were observed. Cases observed were evenly distributed by faculty experience (1-5 years' experience: 67 [20.9%]; 6-14 years' experience: 186 [58%]; ≥15 years' experience: 67 [20.9%]), resident PGY (PGY 1: 28 [8%]; PGY 2: 74 [22%]; PGY 3: 64 [19%]; PGY 4: 40 [12%]; PGY 5: 97 [29%]; ≥PGY 6: 36 [11%]), and resident gender (female: 183 [54%]; male: 154 [46%]). At the univariate level, PGY (mean [SD] resident entrustability score range, 1.44 [0.46] for PGY 1 to 3.24 [0.65] for PGY 6; F = 38.92; P < .001) and faculty entrustment (2.55 [0.86]; R2 = 0.94; P < .001) were significantly associated with resident entrustablity. Path analysis demonstrated that faculty entrustment was associated with resident entrustability and that the association of PGY with resident entrustability was mediated by faculty entrustment at all 4 institutions. Conclusions and Relevance Faculty entrustment remained associated with resident entrustability across various surgical training programs. These findings suggest that efforts to develop faculty entrustment behaviors may enhance intraoperative teaching and resident progression by promoting resident entrustability.
Collapse
Affiliation(s)
- Jessica K. Millar
- Department of Surgery, University of Michigan, Ann Arbor
- Department of Cardiac Surgery, University of Michigan, Ann Arbor
| | - Niki Matusko
- Department of Surgery, University of Michigan, Ann Arbor
| | - Julie Evans
- Department of Surgery, University of Michigan, Ann Arbor
| | | | - Brenessa Lindeman
- Department of Surgery, University of Alabama at Birmingham, Birmingham
| | - Sarah Jung
- Department of Surgery, University of Wisconsin-Madison
| | | | - Emily Weinstein
- Department of Surgery, Oregon Health and Science University, Portland
| | | | - Mackenzie R. Cook
- Department of Surgery, Oregon Health and Science University, Portland
| | - Karen J. Brasel
- Department of Surgery, Oregon Health and Science University, Portland
| | - Gurjit Sandhu
- Department of Surgery, University of Michigan, Ann Arbor
| |
Collapse
|
5
|
Villa S, Caretta‐Weyer H, Yarris LM, Clarke SO, Coates WC, Sokol KA, Jurvis A, Papanagnou D, Ahn J, Hillman E, Camejo M, Deiorio N, Fischer KM, Wolff M, Estes M, Dimeo S, Jordan J. Development of entrustable professional activities for emergency medicine medical education fellowships: A modified Delphi study. AEM EDUCATION AND TRAINING 2024; 8:e10944. [PMID: 38504805 PMCID: PMC10950010 DOI: 10.1002/aet2.10944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 10/17/2023] [Accepted: 12/05/2023] [Indexed: 03/21/2024]
Abstract
Background It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows. Methods From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics. Results Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration. Conclusions We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd.
Collapse
Affiliation(s)
- Stephen Villa
- UCLA Department of Emergency MedicineUCLA David Geffen School of MedicineLos AngelesCaliforniaUSA
| | - Holly Caretta‐Weyer
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | | | - Samuel O. Clarke
- Department of Emergency MedicineUC Davis Health SystemSacramentoCaliforniaUSA
| | - Wendy C. Coates
- Harbor–UCLA Department of Emergency MedicineUCLA Geffen School of MedicineLos AngelesCaliforniaUSA
| | - Kimberly A. Sokol
- Department of Emergency MedicineKaweah Health Medical CenterVisaliaCaliforniaUSA
| | - Amanda Jurvis
- Department of Emergency MedicineHennepin HealthcareMinneapolisMinnesotaUSA
| | - Dimitrios Papanagnou
- Department of Emergency MedicineSidney Kimmel Medical College at Thomas Jefferson UniversityPhiladelphiaPennsylvaniaUSA
| | - James Ahn
- Section of Emergency MedicineUniversity of ChicagoChicagoIllinoisUSA
| | - Emily Hillman
- Department of Emergency MedicineUniversity Health‐Truman Medical Center, University of Missouri–Kansas City School of MedicineKansas CityMissouriUSA
| | - Melanie Camejo
- Department of Emergency MedicineUniversity Health‐Truman Medical Center, University of Missouri–Kansas City School of MedicineKansas CityMissouriUSA
| | - Nicole Deiorio
- Virginia Commonwealth University School of MedicineVirginia Commonwealth Department of Emergency MedicineRichmondVirginiaUSA
| | - Kathryn M. Fischer
- Department of Emergency MedicineBaylor College of MedicineHoustonTexasUSA
| | - Meg Wolff
- Department of Emergency MedicineUniversity of Michigan Medical SchoolAnn ArborMichiganUSA
| | - Molly Estes
- Department of Emergency MedicineLoma Linda University School of MedicineLoma LindaCaliforniaUSA
| | - Sara Dimeo
- Department of Emergency MedicineUniversity of South Carolina School of Medicine GreenvilleGreenvilleSouth CarolinaUSA
- Present address:
Dignity Health East Valley Emergency Medicine Residency ProgramChandlerAZUSA
| | - Jaime Jordan
- UCLA Department of Emergency MedicineUCLA David Geffen School of MedicineLos AngelesCaliforniaUSA
| |
Collapse
|
6
|
Louis AS, Lee C, Page AV, Ginsburg S. Anticipation or avoidance: internal medicine resident experiences performing invasive bedside procedures. CANADIAN MEDICAL EDUCATION JOURNAL 2023; 14:5-13. [PMID: 38045067 PMCID: PMC10689992 DOI: 10.36834/cmej.73122] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/05/2023]
Abstract
Background Internal Medicine (IM) residents are required to perform bedside procedures for diagnostic and therapeutic purposes. Residents' experiences with procedures vary widely, for unclear reasons. Objective To explore IM residents' experiences with performing bedside procedures and to identify barriers and facilitators to obtaining sufficient experience. Methods Using an inductive, thematic approach, we conducted five individual semi-structured interviews and one focus group with seven IM residents (12 residents in total) during the 2017-2018 academic year at a Canadian tertiary care centre. We used iterative, open-ended questions to elicit residents' experiences, and barriers and facilitators, to performing bedside procedures. Transcripts were analyzed for themes using Braun and Clarke's method. Results We identified four themes 1) Patient-specific factors such as body habitus and procedure urgency; 2) Systems factors such as time constraints and accessibility of materials; 3) Faculty factors including availability to supervise, comfort level, and referral preferences, and 4) Resident-specific factors including preparation, prior experiences, and confidence. Some residents expressed procedure-related anxiety and avoidance. Conclusion Educational interventions aimed to improve procedural efficiency and ensure availability of supervisors may help facilitate residents to perform procedures, yet may not address procedure-related anxiety. Further study is required to understand better how procedure-averse residents can gain confidence to seek out procedures.
Collapse
Affiliation(s)
- Alyssa S Louis
- Department of Medicine, Sinai Health, University of Toronto, Ontario, Canada; Division of Infectious Diseases
| | - Christie Lee
- Department of Medicine, Sinai Health, University of Toronto, Ontario, Canada; Division of Infectious Diseases
| | - Andrea V Page
- Department of Medicine, Sinai Health, University of Toronto, Ontario, Canada
- Temerty Faculty of Medicine, University of Toronto, Ontario, Canada
| | - Shiphra Ginsburg
- Department of Medicine, Sinai Health, University of Toronto, Ontario, Canada; Division of Infectious Diseases
- Wilson Centre for Research in Education, University Health Network, Ontario, Canada University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
7
|
Hamed R, Banks TM, Mahoney D, Simon P, Timmerberg JF, Nilsen DM. A Call to Shift to Competency-Based Education. Am J Occup Ther 2023; 77:7706347010. [PMID: 37938979 DOI: 10.5014/ajot.2023.050402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2023] Open
Abstract
This column explores the concept of competency-based education (CBE). A shift to CBE is a key trend for the future of health care education. Health care professions that have adopted, or started to adopt, a CBE framework include physical therapy, speech-language pathology, social work, medicine, nursing, pharmacology, and dentistry. Internationally, many occupational therapy programs are in the process of shifting to, or have shifted to, a CBE model. This column discusses how although select occupational therapy programs in the United States may individually be considering shifting to, or have shifted to, a CBE framework, there is no national movement to explore adopting the model for occupational therapy or a consensus on defined outcomes for the profession.
Collapse
Affiliation(s)
- Razan Hamed
- Razan Hamed, PhD, OTR/L, is Associate Professor, Department of Rehabilitation and Regenerative Medicine, and Associate Director, Programs in Occupational Therapy, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY;
| | - Tyra M Banks
- Tyra M. Banks, EdD, OTR/L, is Assistant Professor, Department of Rehabilitation and Regenerative Medicine, Programs in Occupational Therapy, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY
| | - Danielle Mahoney
- Danielle Mahoney, OTD, OTR/L, is Assistant Professor, Department of Rehabilitation and Regenerative Medicine, Programs in Occupational Therapy, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY
| | - Phyllis Simon
- Phyllis Simon, OTD, OTR/L, FNAP, is Assistant Professor, Department of Rehabilitation and Regenerative Medicine, Programs in Occupational Therapy, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY
| | - Jean Fitzpatrick Timmerberg
- Jean Fitzpatrick Timmerberg, PhD, PT, is Associate Professor and Vice Chair, Department of Rehabilitation and Regenerative Medicine; Director, Programs in Physical Therapy; and Assistant Dean, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY
| | - Dawn M Nilsen
- Dawn M. Nilsen, EdD, OTR/L, FAOTA, is Professor of Rehabilitation and Regenerative Medicine; Director, Programs in Occupational Therapy; Vice Chair, Department of Rehabilitation and Regenerative Medicine; and Assistant Dean, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center, New York, NY
| |
Collapse
|
8
|
Villagrán I, Rammsy F, Del Valle J, Gregorio de Las Heras S, Pozo L, García P, Torres G, Varas J, Mandrusiak A, Corvetto M, Fuentes-Cimma J. Remote, asynchronous training and feedback enables development of neurodynamic skills in physiotherapy students. BMC MEDICAL EDUCATION 2023; 23:267. [PMID: 37081551 PMCID: PMC10116106 DOI: 10.1186/s12909-023-04229-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Accepted: 04/03/2023] [Indexed: 05/03/2023]
Abstract
BACKGROUND During the COVID-19 pandemic, face-to-face teaching and learning of physiotherapy practical skills was limited. Asynchronous, remote training has been effective in development of clinical skills in some health professions. This study aimed to determine the effect of remote, asynchronous training and feedback on development of neurodynamic skills in physiotherapy students. METHODS Longitudinal repeated measurements study, across four training sessions. Participants engaged in a remote training program for development of upper limb neurodynamic techniques. In this sequential training, participants viewed the online tutorial, practiced independently, and uploaded a video of their performance for formative assessment and feedback from a trained instructor via a checklist and rubric. RESULTS Intra-subject analyses of 60 third-year physiotherapy students showed that the target standard of performance, with no further significant change in scores, was attained following session 2 for the checklist and session 3 for the rubric. This shows that two sessions are required to learn the procedures, and three sessions yield further improvements in performance quality. CONCLUSION The remote, asynchronous training and feedback model proved to be an effective strategy for students' development of neurodynamic testing skills and forms a viable alternative to in-person training. This study contributes to the future of acquiring physiotherapy clinical competencies when distance or hybrid practice is required.
Collapse
Affiliation(s)
- Ignacio Villagrán
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Francisca Rammsy
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Javiera Del Valle
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Sofía Gregorio de Las Heras
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Liliana Pozo
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Patricio García
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Gustavo Torres
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile
| | - Julián Varas
- Centro de Simulación y Cirugía experimental, Facultad de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Allison Mandrusiak
- School of Health and Rehabilitation Sciences, The University of Queensland, Brisbane, Australia
| | - Marcia Corvetto
- Centro de Simulación y Cirugía experimental, Facultad de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Javiera Fuentes-Cimma
- Carrera de Kinesiología, Departamento de Ciencias de la Salud, Facultad de Medicina, Pontificia Universidad Católica de Chile, Vicuña Mackenna 4860, Santiago, Chile.
- School of Health Professions Education (SHE), Maastricht University, Maastricht, the Netherlands.
| |
Collapse
|
9
|
Quach S, Veitch A, Zaccagnini M, West A, Nonoyama ML. Underrepresentation of Respiratory Therapists as Experts in Delphi Studies on Respiratory Practices and Research Priorities. Respir Care 2022; 67:1609-1632. [PMID: 36442987 PMCID: PMC9994035 DOI: 10.4187/respcare.10012] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Delphi survey techniques are a common consensus method used to collect feedback from an expert panel to inform practices, establish guidelines, and identify research priorities. Collecting respiratory therapists' (RT) expertise and experiences as part of consensus-building methodologies is one way to ensure that they align with RT practices and to better influence respiratory care practice. This narrative review aimed to report the RT representation in expert panels of Delphi studies focused on respiratory therapy practices and research priorities. The research question that guided this review is: to what extent are RTs included as expert participants among published Delphi studies relate to respiratory therapy and research topics? We conducted a structured search of the literature and identified 23 papers that reported Delphi studies related to respiratory care practices and 15 that reported on respiratory-related research priorities. Delphi studies that focused on reporting consensus on respiratory care practices included the following: (1) mechanical ventilation, (2) high-flow nasal cannula therapy, (3) COVID-19 respiratory management, (4) home oxygen therapy, (5) cardiopulmonary monitoring, and (6) disease-specific guidelines. Delphi studies that focused on establishing respiratory research priorities included the following: (1) theory and practice-orientated knowledge gaps, and (2) priority research topics for empirical investigation. The results of this review suggest that RTs were rarely included as expert participants and, when involved, were minimally represented (5% to 33%). Given RTs' diverse and relevant experience in respiratory care, incorporating their perspectives to inform future education, respiratory care practices, and research priorities would allow evidence to better align with knowledge gaps deemed important for the respiratory therapy profession.
Collapse
Affiliation(s)
- Shirley Quach
- School of Rehabilitation Sciences, McMaster University, Hamilton, Ontario, Canada
- Faculty of Health Sciences, Ontario Tech University, Oshawa, Ontario, Canada
- Respiratory Therapy Department, Hospital of Sick Children, Toronto, Ontario, Canada
| | - Alanna Veitch
- Faculty of Health Sciences, Ontario Tech University, Oshawa, Ontario, Canada
| | - Marco Zaccagnini
- School of Physical and Occupational Therapy, McGill University, Montréal, Quebec, Canada
- Department Respiratory Therapy, McGill University Health Centre, Montréal, Quebec, Canada
| | - Andrew West
- Canadian Society of Respiratory Therapists, Saint John, New Brunswick, Canada
| | - Mika L Nonoyama
- Faculty of Health Sciences, Ontario Tech University, Oshawa, Ontario, Canada.
- Respiratory Therapy Department, Hospital of Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
10
|
Dunne D, Gielissen K, Slade M, Park YS, Green M. WBAs in UME-How Many Are Needed? A Reliability Analysis of 5 AAMC Core EPAs Implemented in the Internal Medicine Clerkship. J Gen Intern Med 2022; 37:2684-2690. [PMID: 34561828 PMCID: PMC9411433 DOI: 10.1007/s11606-021-07151-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Accepted: 09/09/2021] [Indexed: 01/07/2023]
Abstract
BACKGROUND Reliable assessments of clinical skills are important for undergraduate medical education, trustworthy handoffs to graduate medical programs, and safe, effective patient care. Entrustable professional activities (EPAs) for entering residency have been developed; research is needed to assess reliability of such assessments in authentic clinical workspaces. DESIGN A student-driven mobile assessment platform was developed and used for clinical supervisors to record ad hoc entrustment decisions using the modified Ottawa scale on 5 core EPAs in an 8-week internal medicine (IM) clerkship. After a 12-month period, generalizability (G) theory analysis was performed to estimate the reliability of entrustment scores and determine the proportion of variance attributable to the student and the other facets, including particular EPA, evaluator type (attending versus resident), or case complexity. Decision (D) theory analysis determined the expected reliability based on the number of hypothetical observations. A g-coefficient of 0.7 was used as a generally agreed upon minimum reliability threshold. KEY RESULTS A total of 1368 ratings over the 5 EPAs were completed on 94 students. Variance attributed to person (true variance) was high for all EPAs; EPA-5 had the lowest person variance (9.8% across cases and four blocks). Across cases, reliability ranged from 0.02 to 0.60. Applying this to the Decision study, the estimated number of observations needed to reach a reliability index of 0.7 ranged between 9 and 11 for all EPAs except EPA5 which was sensitive to case complexity. CONCLUSIONS Work place-based clinical skills in IM clerkship students were assessed and logged using a convenient mobile platform. Our analysis suggests that 9-11 observations are needed for these EPA workplace-based assessments (WBAs) to achieve a reliability index of 0.7. Note writing was very sensitive to case complexity. Further reliability analyses of core EPAs are needed before US medical schools consider wider adoption into summative entrustment processes and GME handoffs.
Collapse
Affiliation(s)
- Dana Dunne
- Department of Internal Medicine, Section of Infectious Diseases, Yale School of Medicine, 15 York Street LMP 1074, New Haven, CT, 065111, USA.
| | - Katherine Gielissen
- Department of Internal Medicine, Section of General Medicine, Yale School of Medicine, New Haven, CT, 06511, USA
| | - Martin Slade
- Occupational Medicine, Yale School of Medicine, New Haven, CT, 06511, USA
| | | | - Michael Green
- Department of Internal Medicine, Section of General Medicine, Yale School of Medicine, New Haven, CT, 06511, USA
| |
Collapse
|
11
|
Jeyalingam T, Brydges R, Ginsburg S, McCreath GA, Walsh CM. How Clinical Supervisors Conceptualize Procedural Entrustment: An Interview-Based Study of Entrustment Decision Making in Endoscopic Training. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:586-592. [PMID: 34935727 DOI: 10.1097/acm.0000000000004566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
PURPOSE Entrustment is central to assessment in competency-based medical education (CBME). To date, little research has addressed how clinical supervisors conceptualize entrustment, including factors they consider in making entrustment decisions. The aim of this study was to characterize supervisors' decision making related to procedural entrustment, using gastrointestinal endoscopy as a test case. METHOD Using methods from constructivist grounded theory, the authors interviewed 29 endoscopy supervisors in the United States and Canada across multiple specialties (adult and pediatric gastroenterology, surgery, and family medicine). Semistructured interviews, conducted between April and November 2019, focused on how supervisors conceptualize procedural entrustment, how they make entrustment decisions, and what factors they consider. Transcripts were analyzed using constant comparison to generate an explanatory framework and themes. RESULTS Three themes were identified from the analysis of interview transcripts: (1) entrustment occurs in varying degrees and fluctuates over time; (2) entrustment decisions can transfer within and across procedural and nonprocedural contexts; (3a) persistent static factors (e.g., supervisor competence, institutional culture, legal considerations) influence entrustment decisions, as do (3b) fluctuating, situated dynamic factors (e.g., trainee skills, patient acuity, time constraints), which tend to change from one training encounter to the next. CONCLUSIONS In the process of making procedural entrustment decisions, clinical supervisors appear to synthesize multiple dynamic factors against a background of static factors, culminating in a decision of whether to entrust. Entrustment decisions appear to fluctuate over time, and assessors may transfer decisions about specific trainees across settings. Understanding which factors supervisors perceive as influencing their decision making has the potential to inform faculty development, as well as competency committees seeking to aggregate faculty judgments about trainee unsupervised practice. Those leading CBME programs may wish to invest in optimizing the observed static factors, such that these foundational factors are tuned to facilitate trainee learning and achievement of entrustment.
Collapse
Affiliation(s)
- Thurarshen Jeyalingam
- T. Jeyalingam is an advanced fellow in luminal therapeutic endoscopy, University of Calgary, Calgary, Alberta, Canada; ORCID: http://orcid.org/0000-0002-7254-9639
| | - Ryan Brydges
- R. Brydges is a scientist and holds the Professorship in Technology-Enabled Education, St. Michael's Hospital, Unity Health Toronto, and is associate professor, Department of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Shiphra Ginsburg
- S. Ginsburg is professor of medicine, Department of Medicine, Sinai Health System and Faculty of Medicine, a scientist, Wilson Centre for Research in Education, and Canada Research Chair in Health Professions Education, University of Toronto, Toronto, Ontario, Canada; ORCID: http://orcid.org/0000-0002-4595-6650
| | - Graham A McCreath
- G.A. McCreath is clinical research project coordinator, SickKids Research Institute, Hospital for Sick Children, Toronto, Ontario, Canada; ORCID: http://orcid.org/0000-0002-9312-8665
| | - Catharine M Walsh
- C.M. Walsh is staff gastroenterologist, Division of Gastroenterology, Hepatology, and Nutrition, educational researcher, SickKids Learning Institute, scientist, Child Health Evaluative Sciences, SickKids Research Institute, Hospital for Sick Children, scientist, Wilson Centre for Research in Education, and associate professor of paediatrics, University of Toronto, Toronto, Ontario, Canada; ORCID: http://orcid.org/0000-0003-3928-703X
| |
Collapse
|
12
|
Woodworth GE, Marty AP, Tanaka PP, Ambardekar AP, Chen F, Duncan MJ, Fromer IR, Hallman MR, Klesius LL, Ladlie BL, Mitchell SA, Miller Juve AK, McGrath BJ, Shepler JA, Sims C, Spofford CM, Van Cleve W, Maniker RB. Development and Pilot Testing of Entrustable Professional Activities for US Anesthesiology Residency Training. Anesth Analg 2021; 132:1579-1591. [PMID: 33661789 DOI: 10.1213/ane.0000000000005434] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
BACKGROUND Modern medical education requires frequent competency assessment. The Accreditation Council for Graduate Medical Education (ACGME) provides a descriptive framework of competencies and milestones but does not provide standardized instruments to assess and track trainee competency over time. Entrustable professional activities (EPAs) represent a workplace-based method to assess the achievement of competency milestones at the point-of-care that can be applied to anesthesiology training in the United States. METHODS Experts in education and competency assessment were recruited to participate in a 6-step process using a modified Delphi method with iterative rounds to reach consensus on an entrustment scale, a list of EPAs and procedural skills, detailed definitions for each EPA, a mapping of the EPAs to the ACGME milestones, and a target level of entrustment for graduating US anesthesiology residents for each EPA and procedural skill. The defined EPAs and procedural skills were implemented using a website and mobile app. The assessment system was piloted at 7 anesthesiology residency programs. After 2 months, faculty were surveyed on their attitudes on usability and utility of the assessment system. The number of evaluations submitted per month was collected for 1 year. RESULTS Participants in EPA development included 18 education experts from 11 different programs. The Delphi rounds produced a final list of 20 EPAs, each differentiated as simple or complex, a defined entrustment scale, mapping of the EPAs to milestones, and graduation entrustment targets. A list of 159 procedural skills was similarly developed. Results of the faculty survey demonstrated favorable ratings on all questions regarding app usability as well as the utility of the app and EPA assessments. Over the 2-month pilot period, 1636 EPA and 1427 procedure assessments were submitted. All programs continued to use the app for the remainder of the academic year resulting in 12,641 submitted assessments. CONCLUSIONS A list of 20 anesthesiology EPAs and 159 procedural skills assessments were developed using a rigorous methodology to reach consensus among education experts. The assessments were pilot tested at 7 US anesthesiology residency programs demonstrating the feasibility of implementation using a mobile app and the ability to collect assessment data. Adoption at the pilot sites was variable; however, the use of the system was not mandatory for faculty or trainees at any site.
Collapse
Affiliation(s)
- Glenn E Woodworth
- From the Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon
| | - Adrian P Marty
- Institute of Anesthesiology, University Hospital Zurich, Zurich, Switzerland
| | - Pedro P Tanaka
- Department of Anesthesiology, Stanford University, Stanford, California
| | - Aditee P Ambardekar
- Department of Anesthesiology, University of Texas, Southwestern Medical Center, Dallas, Texas
| | - Fei Chen
- Department of Anesthesiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina
| | - Michael J Duncan
- Department of Anesthesiology, University of Missouri-Kansas City, Kansas City, Missouri
| | - Ilana R Fromer
- Department of Anesthesiology, University of Minnesota, Minneapolis, Minnesota
| | - Matthew R Hallman
- Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, Washington
| | - Lisa L Klesius
- Department of Anesthesiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Beth L Ladlie
- Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota
| | | | - Amy K Miller Juve
- Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon
| | - Brian J McGrath
- Department of Anesthesiology, University of Florida College of Medicine - Jacksonville, Jacksonville, Florida
| | - John A Shepler
- Department of Anesthesiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Charles Sims
- Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota
| | - Christina M Spofford
- Department of Anesthesiology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Wil Van Cleve
- Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, Washington
| | - Robert B Maniker
- Department of Anesthesiology, Columbia University, New York, New York
| |
Collapse
|
13
|
Kouzmina E, Mann S, Chaplin T, Zevin B. An Evaluation of the Surgical Foundations Curriculum: A National Study. JOURNAL OF SURGICAL EDUCATION 2021; 78:914-926. [PMID: 33109493 DOI: 10.1016/j.jsurg.2020.10.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Revised: 09/12/2020] [Accepted: 10/02/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND Canadian Surgical Foundations (SF) residency programs transitioned to competency-based medical education in 2018. It is unknown how well the SF curriculum prepares surgery residents to recognize and manage common perioperative patient presentations. We sought to evaluate the national SF curriculum using the Kirkpatrick model of curriculum evaluation. METHODS We administered online surveys to 300 first-year English-speaking surgery residents across Canada to assess self-reported confidence in recognizing and managing 7 common perioperative patient presentations at 3 timepoints: pre-SF (July 2019), mid-SF (December 2019), and post-SF (May 2020). We conducted multistation simulation-based objective structured clinical examinations for surgery residents at our institution pre-SF (August 2019) and mid-SF (December 2019), and collected workplace-based assessment (WBA) data, including entrustment scores and narrative feedback, for 6 preselected entrustable professional activities (EPAs) (July 2019 to May 2020). RESULTS Fifty-five residents (18%) completed pre-SF, 31 (10%) completed mid-SF, and 52 (17%) completed post-SF surveys. Residents' confidence in recognizing 6 out of 7 patient presentations was high pre-SF and did not improve significantly during the SF curriculum except for recognizing poor glycemic control (p < 0.01). Residents' confidence in managing 7 out of 7 patient presentations improved significantly (p < 0.05). Objective structured clinical examinations performance did not change significantly between pre-SF and mid-SF (4 [3.5-4.5] vs 4 [3-4]; p = 0.28). Analysis of WBA data showed that residents received high entrustment scores from the start of the SF curriculum. Entrustment scores improved significantly during the SF curriculum for 2 out of 6 EPAs. Only 56% of WBA assessments had narrative feedback, 16% of which had somewhat constructive feedback. CONCLUSION Participation in the SF curriculum was associated with improved confidence of surgery residents in managing common perioperative patient presentations, and greater level of entrustment for some EPAs. Consideration should be given to further faculty development to increase the quantity and quality of narrative feedback in the SF curriculum.
Collapse
Affiliation(s)
| | - Stephen Mann
- Department of Surgery, Queen's University, Kingston, Ontario, Canada
| | - Timothy Chaplin
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| | - Boris Zevin
- Department of Surgery, Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
14
|
Dehghani Poudeh M, Mohammadi A, Mojtahedzadeh R, Yamani N. Entrustability levels of general internal medicine residents. BMC MEDICAL EDUCATION 2021; 21:185. [PMID: 33766005 PMCID: PMC7995576 DOI: 10.1186/s12909-021-02624-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 03/19/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Entrustable professional activities (EPAs) are those activities that a health professional can perform without direct supervision in a defined environment. Bridging the gap between competencies and learning objectives, EPAs have made assessing the performances of health professional more realistic. The main objective of the present study was developing and customizing EPAs for Iranian Internal Medicine Residency Programs. RESULTS After reviewing the publications, residency curricula and logbooks, and collecting experts' ideas, the initial list of EPAs was developed. Then, in a focus group, the list was refined, the entrustability level of each residency year was determined, and finally, the EPA-competency cross-tab was established, and in the next step, through a one- round Delphi, the results were validated. Twenty-eight EPAs were developed. Some of them were definitely suitable for the higher levels of residency, such that they had to be accomplished under direct supervision until the end of the program. On the other hand, some of EPAs were those that residents, even from the first year, are expected to perform independently or under indirect supervision. Most of the EPAs cover a wide range of competencies. CONCLUSION Determining the entrustability level of each residency year in each EPA as well as the competency- EPA matrix has crucial effect on the quality of the graduates. It seems that our findings are applicable in developing countries like Iran.
Collapse
Affiliation(s)
- Mostafa Dehghani Poudeh
- Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Aeen Mohammadi
- Department of E-learning in Medical Education, Virtual School, Center for Excellence in E-learning in Medical Education, Tehran University of Medical Sciences, Tehran, Iran
| | - Rita Mojtahedzadeh
- Department of E-learning in Medical Education, Virtual School, Center for Excellence in E-learning in Medical Education, Tehran University of Medical Sciences, Tehran, Iran
| | - Nikoo Yamani
- Department of Medical Education, Isfahan University of Medical Sciences, Isfahan, Iran
| |
Collapse
|
15
|
St-Onge C, Vachon Lachiver É, Langevin S, Boileau E, Bernier F, Thomas A. Lessons from the implementation of developmental progress assessment: A scoping review. MEDICAL EDUCATION 2020; 54:878-887. [PMID: 32083743 DOI: 10.1111/medu.14136] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Revised: 01/21/2020] [Accepted: 02/06/2020] [Indexed: 06/10/2023]
Abstract
OBJECTIVES Educators and researchers recently implemented developmental progress assessment (DPA) in the context of competency-based education. To reap its anticipated benefits, much still remains to be understood about its implementation. In this study, we aimed to determine the nature and extent of the current evidence on DPA, in an effort to broaden our understanding of the major goals and intended outcomes of DPA as well as the lessons learned from how it has been executed in, or applied across, educational contexts. METHODS We conducted a scoping study based on the methodology of Arksey and O'Malley. Our search strategy yielded 2494 articles. These articles were screened for inclusion and exclusion (90% agreement), and numerical and qualitative data were extracted from 56 articles based on a pre-defined set of charting categories. The thematic analysis of the qualitative data was completed with iterative consultations and discussions until consensus was achieved for the interpretation of the results. RESULTS Tools used to document DPA include scales, milestones and portfolios. Performances were observed in clinical or standardised contexts. We identified seven major themes in our qualitative thematic analysis: (a) underlying aims of DPA; (b) sources of information; (c) barriers; (d) contextual factors that can act as barriers or facilitators to the implementation of DPA; (e) facilitators; (f) observed outcomes, and (g) documented validity evidences. CONCLUSIONS Developmental progress assessment seems to fill a need in the training of future competent health professionals. However, moving forward with a widespread implementation of DPA, factors such as lack of access to user-friendly technology and time to observe performance may render its operationalisation burdensome in the context of competency-based medical education.
Collapse
Affiliation(s)
- Christina St-Onge
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Élise Vachon Lachiver
- Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Serge Langevin
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Elisabeth Boileau
- Department of Family and Emergency Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Frédéric Bernier
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
- Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
- Research Center - Sherbrooke University Hospital Center (CHUS), Integrated Health and Social Service Centers (CISSS) and Integrated University Health and Social Service Centres (CIUSSS), Sherbrooke, Québec, Canada
| | - Aliki Thomas
- School of Physical and Occupational Therapy, McGill University, Montreal, Québec, Canada
| |
Collapse
|
16
|
Goldenberg M, Ordon M, Honey JRD, Andonian S, Lee JY. Objective Assessment and Standard Setting for Basic Flexible Ureterorenoscopy Skills Among Urology Trainees Using Simulation-Based Methods. J Endourol 2020; 34:495-501. [PMID: 32059622 DOI: 10.1089/end.2019.0626] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023] Open
Abstract
Objective: To objectively assess the performance of graduating urology residents performing flexible ureterorenoscopy (fURS) using a simulation-based model and to set an entrustability standard or benchmark for use across the educational spectrum. Methods: Chief urology residents and attending endourologists performed a standardized fURS task (ureterorenoscopy and repositioning of stones) using a Boston Scientific© Lithovue ureteroscope on a Cook Medical© URS model. All performances were video-recorded and blindly scored by both endourology experts and crowd-workers (C-SATS) using the Ureteroscopic Global Rating Scale, plus an overall entrustability score. Validity evidence supporting the scores was collected and categorized. The Borderline Group (BG) method was used to set absolute performance standards for the expert and crowdsourced ratings. Results: A total of 44 participants (40 chief residents, 4 faculties) completed testing. Eighty-three percent of participants had performed >50 fURS cases at the time of the study. Only 47.7% (mean score 12.6/20) and 61.4% (mean score 12.4/20) of participants were deemed "entrustable" by experts and crowd-workers, respectively. The BG method produced entrustability benchmarks of 11.8/20 for experts and 11.4/20 for crowd-worker ratings, resulting in pass rates of 56.9% and 61.4%. Conclusion: Using absolute standard setting methods, benchmark scores were set to identify trainees who could safely carry out fURS in the simulated setting. Only 60% of residents in our cohort were rated as entrustable. These findings support the use of benchmarks to earlier identify trainees requiring remediation.
Collapse
Affiliation(s)
- Mitchell Goldenberg
- Division of Urology, Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada
| | - Michael Ordon
- Division of Urology, Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada
| | - John R D'A Honey
- Division of Urology, Department of Surgery, St. Michael's Hospital, University of Toronto, Toronto, Canada
| | - Sero Andonian
- Division of Urology, McGill University Health Centre, McGill University, Quebec, Canada
| | - Jason Y Lee
- Division of Urology, Department of Surgery, University Health Network-Toronto General Hospital, University of Toronto, Toronto, Canada
| |
Collapse
|
17
|
O'Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. MEDICAL EDUCATION 2019; 53:234-249. [PMID: 30609093 DOI: 10.1111/medu.13792] [Citation(s) in RCA: 101] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Revised: 11/05/2018] [Accepted: 11/20/2018] [Indexed: 05/13/2023]
Abstract
PURPOSE This review aimed to synthesise some of the extant work on the use of entrustable professional activities (EPAs) for postgraduate physicians, to assess the quality of the work and provide direction for future research and practice. METHOD Systematic searches were conducted within five electronic databases (Medline, Scopus, Web of Science, PsycINFO and CINAHL) in September 2018. Reference lists, Google Scholar and Google were also searched. Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). RESULTS In total, 49 studies were included, classified as Development of EPAs (n = 37; 76% of total included), Implementation and/or assessment of EPAs (n = 10; 20%), or both (n = 2; 4%). EPAs were described for numerous specialties, including internal medicine (n = 14; 36%), paediatrics (n = 8; 21%) and psychiatry (n = 4; 10%). Of the development studies, 92% utilised more than one method to generate EPAs. The two most commonly used methods were developing initial EPAs in a working group, (n = 27; 69%) and revising through deliberation (n = 21; 54%). Development papers were of variable quality (mean QATSDD score = 20, range 6-41). Implementation and assessment studies utilised methods that included observing trainee performance (n = 6; 50%) and enrolling trainees in competency-based curricula, which included EPAs (n = 4; 33%). The methodological quality of these implementation studies varied (mean QATSDD score = 19.5, range = 6-32). CONCLUSIONS This review highlighted a need for: (i) consideration of best practice guidelines for EPA development; (ii) focus on the methodological quality of research on EPA development and of EPAs, and (iii) further work investigating the implementation of EPAs in the curriculum.
Collapse
Affiliation(s)
- Emily O'Dowd
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Sinéad Lydon
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| | - Paul O'Connor
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Caoimhe Madden
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Dara Byrne
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| |
Collapse
|
18
|
Timmerberg JF, Dole R, Silberman N, Goffar SL, Mathur D, Miller A, Murray L, Pelletier D, Simpson MS, Stolfi A, Thompson A, Utzman R. Physical Therapist Student Readiness for Entrance Into the First Full-Time Clinical Experience: A Delphi Study. Phys Ther 2019; 99:131-146. [PMID: 30561680 DOI: 10.1093/ptj/pzy134] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/15/2017] [Accepted: 09/16/2018] [Indexed: 11/14/2022]
Abstract
BACKGROUND To consistently prepare physical therapist students for their first full-time clinical experience, the entry-level curriculum must provide and assess competency in the essential knowledge, skills, attitudes, and professional behaviors. OBJECTIVE The purpose of this Delphi study was to develop consensus on a core set of elements that should be demonstrated by physical therapist students prior to entry into their first full-time clinical experience. A second aim was to obtain the recommended competency levels and assessment methods. DESIGN The study was conducted using the Delphi method. METHODS Purposive selection and snowball sampling techniques were used to recruit clinical instructors, recent graduates, directors or academic coordinators of clinical education, and academic faculty. Four web-based survey rounds were used to achieve consensus, defined as agreement among ≥80% of respondents. The first round gathered demographic information on respondents and identified elements that were deemed essential; the second collected information about clarity and redundancy in the elements provided; the third asked participants to rank their agreement with elements and themes; and the fourth gathered the level of competency that physical therapist students should demonstrate prior to beginning a first full-time clinical experience. RESULTS Consensus revealed 95 elements, categorized under 14 themes, which were deemed essential for readiness for the first clinical experience. Levels of competency for each element were identified. LIMITATIONS Participants might not have represented all academic programs, practice settings, and geographic locations. CONCLUSION This study identified the specific knowledge, skills, attitudes, and professional behaviors in which all physical therapist students in the United States need to demonstrate competency before their first clinical experience, regardless of school or setting, which would allow learning experiences to be tailored appropriately.
Collapse
Affiliation(s)
- Jean F Timmerberg
- Department of Physical Therapy, Columbia University, 710 West 168th St, 8th Floor, New York, NY 10032 (USA)
| | - Robin Dole
- Institute for Physical Therapy Education, Widener University, Chester, Pennsylvania
| | - Nicki Silberman
- Department of Physical Therapy, Hunter College, New York, New York
| | - Stephen L Goffar
- School of Physical Therapy, University of the Incarnate Word, San Antonio, Texas
| | - Divya Mathur
- Department of Physical Therapy, Rusk Rehabilitation at NYU Langone Orthopedic Hospital, New York, New York
| | - Amy Miller
- Department of Physical Therapy, Arcadia University, Glenside, Pennsylvania
| | - Leigh Murray
- Division of Physical Therapy, Walsh University, North Canton, Ohio
| | - Deborah Pelletier
- Department of Physical Therapy, Springfield College, Springfield, Massachusetts
| | - Michael S Simpson
- Department of Physical Therapy, University of Southern California, Los Angeles, California
| | - Angela Stolfi
- Department of Physical Therapy, Rusk Rehabilitation at NYU Langone Orthopedic Hospital
| | - Anne Thompson
- Department of Physical Therapy, Armstrong Atlantic State University, Savannah, Georgia
| | - Ralph Utzman
- Division of Physical Therapy, West Virginia University, Morgantown, West Virginia
| |
Collapse
|
19
|
Chen HE, Sonntag CC, Pepley DF, Prabhu RS, Han DC, Moore JZ, Miller SR. Looks can be deceiving: Gaze pattern differences between novices and experts during placement of central lines. Am J Surg 2018; 217:362-367. [PMID: 30514436 DOI: 10.1016/j.amjsurg.2018.11.007] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2018] [Revised: 10/02/2018] [Accepted: 11/12/2018] [Indexed: 11/25/2022]
Abstract
BACKGROUND The objective of this study was to determine whether gaze patterns could differentiate expertise during simulated ultrasound-guided Internal Jugular Central Venous Catheterization (US-IJCVC) and if expert gazes were different between simulators of varying functional and structural fidelity. METHODS A 2017 study compared eye gaze patterns of expert surgeons (n = 11), senior residents (n = 4), and novices (n = 7) during CVC needle insertions using the dynamic haptic robotic trainer (DHRT), a system which simulates US-IJCVC. Expert gaze patterns were also compared between a manikin and the DHRT. RESULTS Expert gaze patterns were consistent between the manikin and DHRT environments (p = 0.401). On the DHRT system, CVC experience significantly impacted the percent of time participants spent gazing at the ultrasound screen (p < 0.0005) and the needle and ultrasound probe (p < 0.0005). CONCLUSION Gaze patterns differentiate expertise during ultrasound-guided IJCVC placement and the fidelity of the simulator does not impact gaze patterns.
Collapse
Affiliation(s)
- Hong-En Chen
- Department of Industrial and Manufacturing Engineering, Penn State, University Park, PA, 16802, USA
| | - Cheyenne C Sonntag
- Department of Surgery, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, 17033, USA
| | - David F Pepley
- Department of Mechanical and Nuclear Engineering, Penn State, University Park, PA, 16802, USA
| | - Rohan S Prabhu
- Department of Mechanical and Nuclear Engineering, Penn State, University Park, PA, 16802, USA
| | - David C Han
- Department of Surgery, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, 17033, USA; Penn State Heart and Vascular Institute, Penn State Health Milton S. Hershey Medical Center, Hershey, PA, 17033, USA
| | - Jason Z Moore
- Department of Mechanical and Nuclear Engineering, Penn State, University Park, PA, 16802, USA
| | - Scarlett R Miller
- Department of Industrial and Manufacturing Engineering, Penn State, University Park, PA, 16802, USA; School of Engineering Design, Technology, and Professional Programs, Penn State, University Park, PA, 16802, USA.
| |
Collapse
|