1
|
Langhan ML, Boyer DL, Hsu D, Moffatt ME, Pitts SA, Atlas MP, Aye T, Chess P, Curran ML, Czaja AS, Dammann CEL, Fussell J, George RP, Herman BE, High P, James SH, Kamin DS, Karnik R, Kesselheim J, Lopez MA, Mahan JD, McFadden V, McGann KA, Mehta JJ, Rama J, Robinson BW, Sauer C, Stafford DEJ, Turner DA, Weiss P, Yussman SM, Schwartz A, Mink R. Implementing Entrustable Professional Activities in Pediatric Fellowships: Facilitating the Process. Pediatrics 2024; 153:e2023065024. [PMID: 38757175 DOI: 10.1542/peds.2023-065024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 02/02/2024] [Accepted: 02/20/2024] [Indexed: 05/18/2024] Open
Abstract
BACKGROUND AND OBJECTIVES Entrustable professional activities (EPAs) will be used for initial certification by the American Board of Pediatrics by 2028. Less than half of pediatric fellowships currently use EPAs for assessment, yet all will need to adopt them. Our objectives were to identify facilitators and barriers to the implementation of EPAs to assess pediatric fellows and to determine fellowship program directors' (FPD) perceptions of EPAs and Milestones. METHODS We conducted a survey of FPDs from 15 pediatric subspecialties. EPA users were asked about their implementation of EPAs, barriers encountered, and perceptions of EPAs. Nonusers were queried about deterrents to using EPAs. Both groups were asked about potential facilitators of implementation and their perceptions of Milestones. RESULTS The response rate was 65% (575/883). Of these, 344 (59.8%) were EPA users and 231 (40.2%) were nonusers. Both groups indicated work burden as a barrier to implementation. Nonusers reported more barriers than users (mean [SD]: 7 [3.8] vs 5.8 [3.4], P < .001). Both groups identified training materials and premade assessment forms as facilitators to implementation. Users felt that EPAs were easier to understand than Milestones (89%) and better reflected what it meant to be a practicing subspecialty physician (90%). In contrast, nonusers felt that Milestones were easy to understand (57%) and reflected what it meant to be a practicing subspecialist (58%). CONCLUSIONS Implementing EPA-based assessment will require a substantial investment by FPDs, facilitated by guidance and easily accessible resources provided by multiple organizations. Perceived barriers to be addressed include FPD time constraints, a need for additional assessment tools, and outcomes data.
Collapse
Affiliation(s)
| | - Donald L Boyer
- Children's Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Deborah Hsu
- Stanford University School of Medicine, Stanford, California
| | - Mary E Moffatt
- University of Missouri Kansas City School of Medicine, Kansas City, Missouri
| | - Sarah A Pitts
- Boston Children's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Mark P Atlas
- Zucker School of Medicine at Hofstra-Northwell, Cohen Children's Medical Center, Queens, New York
| | - Tandy Aye
- Stanford University School of Medicine, Stanford, California
| | - Patricia Chess
- University of Rochester Medical Center, Rochester, New York
| | - Megan L Curran
- University of Colorado School of Medicine, Aurora, Colorado
| | - Angela S Czaja
- University of Colorado School of Medicine, Aurora, Colorado
| | | | - Jill Fussell
- University of Arkansas for Medical Sciences, Little Rock, Arkansas
| | - Roshan P George
- Emory University and Children's Healthcare of Atlanta, Atlanta, Georgia
| | - Bruce E Herman
- University of Utah School of Medicine, Salt Lake City, Utah
| | - Pamela High
- Warren Alpert Medical School of Brown University, Providence, Rhode Island
| | - Scott H James
- University of Alabama at Birmingham, Birmingham, Alabama
| | - Daniel S Kamin
- Boston Children's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Ruchika Karnik
- Yale University School of Medicine, New Haven, Connecticut
| | - Jennifer Kesselheim
- Boston Children's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Michelle A Lopez
- Baylor College of Medicine, Texas Children's Hospital, Houston, Texas
| | - John D Mahan
- The Ohio State University College of Medicine, Columbus, Ohio
| | | | | | - Jay J Mehta
- Children's Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Jennifer Rama
- Baylor College of Medicine, Texas Children's Hospital, Houston, Texas
| | - Bradley W Robinson
- Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, Pennsylvania
| | - Cary Sauer
- Emory University and Children's Healthcare of Atlanta, Atlanta, Georgia
| | | | - David A Turner
- American Board of Pediatrics, Chapel Hill, North Carolina
| | - Pnina Weiss
- Yale University School of Medicine, New Haven, Connecticut
| | | | - Alan Schwartz
- University of Illinois College of Medicine at Chicago, Chicago, Illinois
| | - Richard Mink
- Harbor-UCLA Medical Center and The Lundquist Institute for Biomedical Innovation, Torrance, California
| |
Collapse
|
2
|
Schmidbauer ML, Pinilla S, Kunst S, Biesalski AS, Bösel J, Niesen WD, Schramm P, Wartenberg K, Dimitriadis K. Fit for Service: Preparing Residents for Neurointensive Care with Entrustable Professional Activities: A Delphi Study. Neurocrit Care 2024; 40:645-653. [PMID: 37498455 PMCID: PMC10959831 DOI: 10.1007/s12028-023-01799-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 06/30/2023] [Indexed: 07/28/2023]
Abstract
BACKGROUND Although the relevance of neurointensive medicine and high-quality training of corresponding physicians is increasingly recognized, there is high heterogeneity in the nature, duration, and quality of neurointensive care curricula around the world. Thus, we aimed to identify, define, and establish validity evidence for entrustable professional activities (EPAs) for postgraduate training in neurointensive care to determine trainees' readiness for being on-call. METHODS After defining EPAs through an iterative process by an expert group, we used a modified Delphi approach with a single-center development process followed by a national consensus and a single-center validation step. EPAs were evaluated by using the EQual rubric (Queen's EPA Quality Rubric). Interrater reliability was measured with Krippendorff's α. RESULTS The expert group defined seven preliminary EPAs for neurointensive care. In two consecutive Delphi rounds, EPAs were adapted, and consensus was reached for level of entrustment and time of expiration. Ultimately, EPAs reached a high EQual score of 4.5 of 5 and above. Interrater reliability for the EQual scoring was 0.8. CONCLUSIONS Using a multistep Delphi process, we defined and established validity evidence for seven EPAs for neurointensive medicine with a high degree of consensus to objectively describe readiness for on-call duty in neurointensive care. This operationalization of pivotal clinical tasks may help to better train clinical residents in neurointensive care across sites and health care systems and has the potential to serve as a blueprint for training in general intensive care medicine. It also represents a starting point for further research and development of medical curricula.
Collapse
Affiliation(s)
| | - Severin Pinilla
- University Hospital for Old Age Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
- Institute for Medical Education (IML), University of Bern, Bern, Switzerland
| | - Stefan Kunst
- Department of Neurology, LMU University Hospital, LMU Munich, Munich, Germany
| | - Anne-Sophie Biesalski
- Department of Neurology, Ruhr-Universität Bochum, St. Josef Hospital, Bochum, Germany
| | - Julian Bösel
- Department of Neurology, Heidelberg University Hospital, Heidelberg, Germany
| | - Wolf-Dirk Niesen
- Department of Neurology and Neurophysiology, University Medical Center Freiburg, Freiburg, Germany
| | - Patrick Schramm
- Department of Neurology, Universitätsklinikum Giessen und Marburg, Standort Giessen, Justus-Liebig-University, Giessen, Germany
| | - Katja Wartenberg
- Department of Neurology, University of Leipzig, Leipzig, Germany
| | - Konstantinos Dimitriadis
- Department of Neurology, LMU University Hospital, LMU Munich, Munich, Germany.
- Institute for Stroke and Dementia Research (ISD), LMU University Hospital, LMU Munich, Munich, Germany.
| |
Collapse
|
3
|
Yan F, Yang X, Zhang L, Cheng H, Bai L, Yang F. Establishing entrustable professional activities for psychiatry residents in China. BMC MEDICAL EDUCATION 2023; 23:623. [PMID: 37658351 PMCID: PMC10474625 DOI: 10.1186/s12909-023-04583-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 08/11/2023] [Indexed: 09/03/2023]
Abstract
PURPOSE The authors established entrustable professional activities for psychiatry residents in China. METHODS The authors conducted a literature research and two expert consultation rounds following the Delphi method in 2022 to screen and optimize entrustable professional activities for psychiatry residents. RESULTS The effective questionnaire recovery rate in the two consultation rounds was 100% (44/44). The expert authority coefficients of the first and second consultation rounds were 0.861 and 0.881, respectively. The Kendall harmony coefficients of the first and second expert consultation rounds were 0.279 (χ2 = 405.43, P < .001) and 0.389 (χ2 = 3456.83, P < .001), respectively. The arithmetic means of the various indicators' evaluation results in the two consultation rounds ranged between 3.61 and 4.93, and the full score rates were between 13.6% and 93.2%. The authors established 17 entrustable professional activities for psychiatry residents and their contents with phase-based modularization and formulated the entrustable level of each at various stages. CONCLUSIONS Combined with standardized psychiatry training characteristics, the authors preliminarily established phase-specific and modular entrustable professional activities for psychiatry residents. The formulated entrustable professional activities are suitable for the practice and clinical environment of standardized psychiatry training in China. The devised system has good observability and measurability and provides a simple and feasible competency evaluation method for standardized psychiatry resident training.
Collapse
Affiliation(s)
- Feng Yan
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Xu Yang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Ligang Zhang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Huaqin Cheng
- Institute of Medical Education &National center for Health Professions Education Development, PeKing University, Beijing, 100083, China
| | - Luyuan Bai
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Fude Yang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China.
| |
Collapse
|
4
|
Chan TM, Sebok-Syer SS, Yilmaz Y, Monteiro S. The Impact of Electronic Data to Capture Qualitative Comments in a Competency-Based Assessment System. Cureus 2022; 14:e23480. [PMID: 35494923 PMCID: PMC9038604 DOI: 10.7759/cureus.23480] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/24/2022] [Indexed: 11/23/2022] Open
Abstract
Introduction Digitalizing workplace-based assessments (WBA) holds the potential for facilitating feedback and performance review, wherein we can easily record, store, and analyze data in real time. When digitizing assessment systems, however, it is unclear what is gained and lost in the message as a result of the change in medium. This study evaluates the quality of comments generated in paper vs. electronic media and the influence of an assessor’s seniority. Methods Using a realist evaluation framework, a retrospective database review was conducted with paper-based and electronic medium comments. A sample of assessments was examined to determine any influence of the medium on the word count and the Quality of Assessment for Learning (QuAL) score. A correlation analysis evaluated the relationship between word count and QuAL score. Separate univariate analyses of variance (ANOVAs) were used to examine the influence of the assessor's seniority and medium on word count, QuAL score, and WBA scores. Results The analysis included a total of 1,825 records. The average word count for the electronic comments (M=16) was significantly higher than the paper version (M=12; p=0.01). Longer comments positively correlated with QuAL score (r=0.2). Paper-based comments received lower QuAL scores (0.41) compared to electronic (0.51; p<0.01). Years in practice was negatively correlated with QuAL score (r=-0.08; p<0.001) as was word count (r=-0.2; p<0.001). Conclusion Digitization of WBAs increased the length of comments and did not appear to jeopardize the quality of WBAs; these results indicate higher-quality assessment data. True digital transformation may be possible by harnessing trainee data repositories and repurposing them to analyze for faculty-relevant metrics.
Collapse
|
5
|
Chauhan VS, Chatterjee K, Prakash J, Singh YM, Dangi A, Dubey A, Chail A, Sharma R. Identifying entrustable professional activities for postgraduation in psychiatry: What should a psychiatrist be able to do? Ind Psychiatry J 2022; 31:49-55. [PMID: 35800858 PMCID: PMC9255628 DOI: 10.4103/ipj.ipj_124_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Revised: 06/23/2021] [Accepted: 07/02/2021] [Indexed: 11/04/2022] Open
Abstract
BACKGROUND Competency-based medical education (CBME) has been the mandate by regulatory bodies of medical education in India, and implementation is an evolving process. This study aimed to identify and propose a set of entrustable professional activities (EPAs) for a master's degree in psychiatry. An attempt has been made to present it in the form of a portfolio. METHODOLOGY Faculty experts in psychiatry went through phases of review to select appropriate EPAs for postgraduate residents from divergent thinking to development of portfolio. RESULTS One hundred and sixty-seven EPAs were identified and are presented. These were divided according to expertise required. Of these, 54% were considered vital, 30% essential, and 16% desirable. CONCLUSION It is envisaged that the use of EPAs and portfolios will be instrumental in implementation and success of CBME in psychiatry. It is expected that medical universities will incorporate them in their curricula. It is also expected that data will be generated to identify challenges, which shall guide future refinement of the EPAs.
Collapse
Affiliation(s)
- Vinay Singh Chauhan
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Kaushik Chatterjee
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Jyoti Prakash
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Yujal Man Singh
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Ankit Dangi
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Amresh Dubey
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Amit Chail
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| | - Rachit Sharma
- Department of Psychiatry, Armed Forces Medical College, Pune, Maharashtra, India
| |
Collapse
|
6
|
Young JQ, Frank JR, Holmboe ES. Advancing Workplace-Based Assessment in Psychiatric Education: Key Design and Implementation Issues. Psychiatr Clin North Am 2021; 44:317-332. [PMID: 34049652 DOI: 10.1016/j.psc.2021.03.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
With the adoption of competency-based medical education, assessment has shifted from traditional classroom domains of knows and knows how to the workplace domain of doing. This workplace-based assessment has 2 purposes; assessment of learning (summative feedback) and the assessment for learning (formative feedback). What the trainee does becomes the basis for identifying growth edges and determining readiness for advancement and ultimately independent practice. High-quality workplace-based assessment programs require thoughtful choices about the framework of assessment, the tools themselves, the platforms used, and the contexts in which the assessments take place, with an emphasis on direct observation.
Collapse
Affiliation(s)
- John Q Young
- Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and, Zucker Hillside Hospital at Northwell Health, 75-59 263rd Street, Kaufman Building, Glen Oaks, NY 11004, USA.
| | - Jason R Frank
- Department of Emergency Medicine, University of Ottawa, Royal College of Physicians and Surgeons of Canada, 774 Echo Drive, Ottawa, Ontario K15 5NB, Canada
| | - Eric S Holmboe
- Accreditation Council for Graduate Medical Education, ACGME, 401 North Michigan Avenue, Chicago, IL 60611, USA
| |
Collapse
|