1
|
Cifra N, Pitts S, Mink R, Schwartz A, Herman B, Turner DA, Yussman S. Analysis of fellowship program director opinions of entrustable professional activities in adolescent medicine fellowship. Int J Adolesc Med Health 2024; 36:237-242. [PMID: 38522004 DOI: 10.1515/ijamh-2023-0154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 03/08/2024] [Indexed: 03/25/2024]
Abstract
OBJECTIVES This study aimed to explore the minimum entrustable professional activity (EPA) supervision levels at which pediatric fellowship program directors (FPDs) would be willing to graduate fellows and the levels deemed necessary for safe and effective practice for each of the common pediatric subspecialty and the four adolescent medicine-specific EPAs. METHODS This cross-sectional study utilized survey data from pediatric FPDs in 2017. FPDs indicated the minimum level of supervision (LOS) for fellows at graduation and for safe and effective practice. RESULTS 82 percent (23/28) of adolescent medicine FPDs completed the survey. For each EPA, there were differences (p<0.05) between LOS expected for graduation and for safe and effective practice. There was also variability in the level at which FPDs would graduate fellows. CONCLUSIONS This study summarizes pediatric FPD opinions regarding the minimum levels of supervision required for fellows at the time of graduation as well as the levels deemed necessary for safe and effective practice. The difference between the minimum LOS at which FPDs would graduate a fellow and that deemed appropriate for safe and effective practice, along with variability in minimum LOS for graduation, highlight the need for clearer standards for fellowship graduation as well as more structured early career support for ongoing learning. These data highlight variability in FPD opinion regarding such expectations and both the need to better define desired training outcomes and potential need for post-graduation supervision in clinical practice.
Collapse
Affiliation(s)
- Nicole Cifra
- Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, USA
| | - Sarah Pitts
- Department of Pediatrics, Boston Children's Hospital, Boston, USA
| | | | - Alan Schwartz
- Department of Pediatrics, University of Utah School of Medicine, Salt Lake City, USA
| | | | | | - Susan Yussman
- Department of Pediatrics, University of Rochester School of Medicine and Dentistry, Rochester, USA
| |
Collapse
|
2
|
Gibbon LM, Buck L, Schmidt L, Bogetz JF, Trowbridge A. "It's a Heavy Thing to Carry:" Internal Medicine and Pediatric Resident Experiences Caring for Dying Patients. Am J Hosp Palliat Care 2024; 41:492-500. [PMID: 37288486 DOI: 10.1177/10499091231181567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/09/2023] Open
Abstract
BACKGROUND Residents often feel unprepared to care for dying patients and may benefit from more training. Little is known about factors in the clinical setting that promote resident learning about end of life (EOL) care. OBJECTIVES This qualitative study aimed to characterize the experiences of residents caring for dying patients and elucidate the impact of emotional, cultural, and logistical factors on learning. METHODS 6 US internal medicine and 8 pediatric residents who had cared for at least 1 dying patient completed a semi-structured one-on-one interview between 2019 and 2020. Residents described an experience caring for a dying patient including their confidence in clinical skills, emotional experience, role within the interdisciplinary team, and perspective on how to improve their education. Interviews were transcribed verbatim and investigators conducted content analysis to generate themes. RESULTS 3 themes (with subthemes) emerged: (1) experiencing strong emotion or tension (loss of patient personhood, emerging professional identity, emotional dissonance); (2) processing the experience (innate resilience, team support); and (3) recognition of a new perspective or skill (bearing witness, meaning making, recognizing biases, emotional work of doctoring). CONCLUSIONS Our data suggests a model for the process by which residents learn affective skills critical to EOL care: residents (1) notice strong emotion, (2) reflect on the meaning of the emotion, and (3) crystallize this reflection into a new perspective or skill. Educators can use this model to develop educational methods that emphasize normalization of physician emotions and space for processing and professional identity formation.
Collapse
Affiliation(s)
- Lindsay M Gibbon
- Department of Medicine, University of Washington School of Medicine, Seattle, WA, USA
- Harborview Medical Center, Seattle, WA, USA
| | - Laura Buck
- Department of Medicine, University of Washington School of Medicine, Seattle, WA, USA
| | - Lauren Schmidt
- Department of Social Work, University of Washington Medical Center, Seattle, WA, USA
| | - Jori F Bogetz
- Department of Pediatrics, University of Washington School of Medicine, Seattle, WA, USA
- Center for Clinical and Translational Research, Seattle Children's Research Institute, Seattle, WA, USA
| | - Amy Trowbridge
- Department of Pediatrics, University of Washington School of Medicine, Seattle, WA, USA
- Center for Clinical and Translational Research, Seattle Children's Research Institute, Seattle, WA, USA
| |
Collapse
|
3
|
Tu CY, Huang KM, Cheng CH, Lin WJ, Liu CH, Yang CW. Development, implementation, and evaluation of entrustable professional activities (EPAs) for medical radiation technologists in Taiwan: a nationwide experience. BMC MEDICAL EDUCATION 2024; 24:95. [PMID: 38287396 PMCID: PMC10826224 DOI: 10.1186/s12909-024-05088-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Accepted: 01/23/2024] [Indexed: 01/31/2024]
Abstract
BACKGROUND Competency-based medical education (CBME) is an outcomes-oriented approach focused on developing competencies that translate into clinical practice. Entrustable professional activities (EPAs) bridge competency assessment and clinical performance by delineating essential day-to-day activities that can be entrusted to trainees. EPAs have been widely adopted internationally, but not yet implemented for medical radiation professionals in Taiwan. MATERIALS AND METHODS A nationwide consensus process engaged 97 experts in radiation technology education representing diagnostic radiography, radiation therapy, and nuclear medicine. Preliminary EPAs were developed through the focus group discussion and the modified Delphi method. The validity of these EPAs was evaluated using the QUEPA and EQual tools. RESULTS Through iterative consensus building, six core EPAs with 18 component observable practice activities (OPAs) in total were developed, encompassing routines specific to each radiation technology specialty. QUEPA and EQual questionnaire data verified these EPAs were valid, and of high quality for clinical teaching and evaluation. CONCLUSION The consensus development of tailored EPAs enables rigorous competency assessment during medical radiation technology education in Taiwan. Further expansion of EPAs and training of clinical staff could potentially enhance care quality by producing competent professionals.
Collapse
Affiliation(s)
- Chun-Yuan Tu
- Taiwan Association of Medical Radiation Technologists, Taipei, Taiwan
- Department of Medical Imaging and Radiological Sciences, I-Shou University, Kaohsiung, Taiwan
| | - Kuo-Ming Huang
- Taiwan Association of Medical Radiation Technologists, Taipei, Taiwan
- Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan
| | - Ching-Hsueh Cheng
- Taiwan Association of Medical Radiation Technologists, Taipei, Taiwan
- Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan
| | - Wei-Jou Lin
- Taiwan Association of Medical Radiation Technologists, Taipei, Taiwan
- Division of Radiation Oncology, Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan
| | - Cheng-Heng Liu
- Department of Medical Education, National Taiwan University Hospital, Taipei, Taiwan
- Department of Emergency Medicine, National Taiwan University Hospital, Taipei, Taiwan
- Department and Graduate Institute of Medical Education and Bioethics, College of Medicine, National Taiwan University, No. 1, Sec. 1, Ren'ai Rd., Zhongzheng Dist., Taipei, 100, Taiwan
| | - Chih-Wei Yang
- Department of Medical Education, National Taiwan University Hospital, Taipei, Taiwan.
- Department of Emergency Medicine, National Taiwan University Hospital, Taipei, Taiwan.
- Department and Graduate Institute of Medical Education and Bioethics, College of Medicine, National Taiwan University, No. 1, Sec. 1, Ren'ai Rd., Zhongzheng Dist., Taipei, 100, Taiwan.
| |
Collapse
|
4
|
Li S, Qi X, Li H, Zhou W, Jiang Z, Qi J. Exploration of validity evidence for core residency entrustable professional activities in Chinese pediatric residency. Front Med (Lausanne) 2024; 10:1301356. [PMID: 38259855 PMCID: PMC10801054 DOI: 10.3389/fmed.2023.1301356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Accepted: 12/12/2023] [Indexed: 01/24/2024] Open
Abstract
Introduction This study seeks to explore validity and reliability evidence for core residency entrustable professional activities (CR-EPAs) that were developed by Peking University First Hospital (PKUFH) in 2020. Methods A prospective cohort study was conducted in PKUFH. Trainers (raters) assessed pediatric residents on CR-EPAs over 1 academic year, bi-annually. Critical components within a validity evidence framework were examined: response process (rater perceptions), the internal structure (reliability and contributions of different variance sources), and consequences (potential use of a cutoff score). Results In total, 37 residents were enrolled, and 111 and 99 trainers' ratings were collected in Fall 2020 and Spring 2021, respectively. For rater perceptions, all the raters considered CR-EPAs highly operational and convenient. In all ratings, individual EPAs correlate with total EPA moderately, with Spearman correlation coefficients spanning from 0.805 to 0.919. EPA 2 (select and interpret the auxiliary examinations), EPA 5 (prepare and complete medical documents), EPA 6 (provide an oral presentation of a case or a clinical encounter), and EPA 7 (identify and manage the general clinical conditions) were EPAs correlated with other EPAs significantly. The results of the generalizability theory indicated that the variability due to residents is the highest (nearly 78.5%), leading to a large size of the reliability estimates. The matching results indicate that the lowest error locates at 5.933. Conclusion The rating showed good validity and reliability. The ratings were reliable based on G-theory. CR-EPAs have a magnificent internal structure and have promising consequences. Our results indicate that CR-EPAs are a robust assessment tool in workplace-based training in a carefully designed setting.
Collapse
Affiliation(s)
- Shan Li
- Department of Paediatrics, Peking University First Hospital, Beijing, China
| | - Xin Qi
- Department of Plastic Surgery and Burns, Peking University First Hospital, Beijing, China
| | - Haichao Li
- Department of Respiratory and Critical Medicine, Peking University First Hospital, Beijing, China
| | - Wenjing Zhou
- School of Public Health, Peking University, Beijing, China
| | - Zhehan Jiang
- Institute of Medical Education and National Center for Health Professions Education Department, Peking University, Beijing, China
| | - Jianguang Qi
- Department of Paediatrics, Peking University First Hospital, Beijing, China
| |
Collapse
|
5
|
Verstegen PMB, Kole JJ(J, Groenewoud AS, van den Hoogen FJA. Virtues in Competency-Based Assessment Frameworks: A Text Analysis. PERSPECTIVES ON MEDICAL EDUCATION 2023; 12:418-426. [PMID: 37868074 PMCID: PMC10588518 DOI: 10.5334/pme.996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 09/08/2023] [Indexed: 10/24/2023]
Abstract
Introduction Official documentation of specialty training provides comprehensive and elaborate criteria to assess residents. These criteria are commonly described in terms of competency roles and entrustable professional activities (EPA's), but they may also implicitly encompass virtues. Virtues are desirable personal qualities that enable a person, in this case, a medical specialist, to make and act on the right decisions. We articulate these virtues and explore the resulting implied ideal of a medical professional. Method We applied a two-staged virtue ethical content analysis to analyze documents, specific to the Dutch training program of the Ear, Nose, and Throat (ENT) specialty. First, we identified explicit references to virtues. Next, we articulated implicit virtues through interpretation. The results were categorized into cardinal, intellectual, moral, and professional virtues. Results Thirty virtues were identified in the ENT- training program. Amongst them, practical wisdom, temperance, and commitment. Furthermore, integrity, curiosity, flexibility, attentiveness, trustworthiness and calmness are often implicitly assumed. Notable findings are the emphasis on efficiency and effectiveness. Together, these virtues depict an ideal of a future medical specialist. Conclusion Our findings suggest that competency-frameworks and EPA's implicitly appeal to virtues and articulate a specific ideal surgeon. Explicit attention for virtue development and discussion of the role and relevance of implied ideal professionals in terms of virtues could further improve specialty training.
Collapse
Affiliation(s)
- Pleuntje M. B. Verstegen
- Radboud university medical center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - J. J. (Jos) Kole
- Radboud university medical center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - A. Stef Groenewoud
- Radboud university medical center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | | |
Collapse
|
6
|
Mink RB, Carraccio CL, Herman BE, Weiss P, Turner DA, Stafford DEJ, McGann KA, Kesselheim J, Hsu DC, High PC, Fussell JJ, Curran ML, Chess PR, Sauer C, Pitts S, Myers AL, Mahan JD, Dammann CEL, Aye T, Schwartz A. Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment. BMC MEDICAL EDUCATION 2023; 23:720. [PMID: 37789289 PMCID: PMC10548580 DOI: 10.1186/s12909-023-04689-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 09/15/2023] [Indexed: 10/05/2023]
Abstract
BACKGROUND Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment. OBJECTIVES We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME). METHODS In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale. RESULTS One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05). CONCLUSIONS We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.
Collapse
Affiliation(s)
- Richard B Mink
- Department of Pediatrics, David Geffen School of Medicine at UCLA and the Lundquist Institute for Biomedical Innovation at Harbor-UCLA Medical Center, 1124 West Carson Street, Torrance, CA, 90502, USA.
| | | | - Bruce E Herman
- University of Utah School of Medicine, Salt Lake, UT, USA
| | - Pnina Weiss
- Department of Pediatrics, Yale School of Medicine, New Haven, CT, USA
| | | | - Diane E J Stafford
- Division of Endocrinology, Department of Pediatrics, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Kathleen A McGann
- Department of Pediatrics, Duke University Medical Center, Durham, NC, USA
| | - Jennifer Kesselheim
- Dana-Farber/Boston Children's Cancer and Blood Disorders Center, Boston, MA, USA
| | | | - Pamela C High
- Alpert Medical School of Brown University, Providence, RI, USA
- Developmental-Behavioral Pediatrics, Hasbro Children's Hospital, Providence, RI, USA
| | - Jill J Fussell
- University of Arkansas for Medical Sciences and Arkansas Children's Hospital, Little Rock, AR, USA
| | - Megan L Curran
- Department of Pediatrics, University of Colorado School of Medicine, Aurora, CO, USA
| | | | - Cary Sauer
- Department of Pediatrics, Emory University School of Medicine and Children's Healthcare of Atlanta, Atlanta, GA, USA
| | - Sarah Pitts
- Division of Adolescent/Young Adult Medicine, Boston Children's Hospital, Boston, MA, USA
| | - Angela L Myers
- Center for Wellbeing, Children's Mercy Hospital and University of Missouri-Kansas City School of Medicine, Kansas City, MO, USA
| | - John D Mahan
- Department of Pediatrics, Nationwide Children's Hospital and The Ohio State University College of Medicine, Columbus, OH, USA
| | | | - Tandy Aye
- Division of Endocrinology, Department of Pediatrics, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Alan Schwartz
- University of Illinois College of Medicine at Chicago, Chicago, IL, USA
| |
Collapse
|
7
|
Kinnear B, Weber DE, Schumacher DJ, Edje L, Warm EJ, Anderson HL. Reconstructing Neurath's Ship: A Case Study in Reevaluating Equity in a Program of Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:S50-S56. [PMID: 37071695 DOI: 10.1097/acm.0000000000005249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Inequity in assessment has been described as a "wicked problem"-an issue with complex roots, inherent tensions, and unclear solutions. To address inequity, health professions educators must critically examine their implicit understandings of truth and knowledge (i.e., their epistemologies) with regard to educational assessment before jumping to solutions. The authors use the analogy of a ship (program of assessment) sailing on different seas (epistemologies) to describe their journey in seeking to improve equity in assessment. Should the education community repair the ship of assessment while sailing or should the ship be scrapped and built anew? The authors share a case study of a well-developed internal medicine residency program of assessment and describe efforts to evaluate and enable equity using various epistemological lenses. They first used a postpositivist lens to evaluate if the systems and strategies aligned with best practices, but found they did not capture important nuances of what equitable assessment entails. Next, they used a constructivist approach to improve stakeholder engagement, but found they still failed to question the inequitable assumptions inherent to their systems and strategies. Finally, they describe a shift to critical epistemologies, seeking to understand who experiences inequity and harm to dismantle inequitable systems and create better ones. The authors describe how each unique sea promoted different adaptations to their ship, and challenge programs to sail through new epistemological waters as a starting point for making their own ships more equitable.
Collapse
Affiliation(s)
- Benjamin Kinnear
- B. Kinnear is associate professor of internal medicine and pediatrics, Departments of Pediatrics and Internal Medicine, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0003-0052-4130
| | - Danielle E Weber
- D.E. Weber is assistant professor of internal medicine and pediatrics, Departments of Pediatrics and Internal Medicine, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-4857-6936
| | - Daniel J Schumacher
- D.J. Schumacher is tenured professor of pediatrics, Department of Pediatrics, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0001-5507-8452
| | - Louito Edje
- L. Edje is professor of family and community medicine, Department of Medical Education and Family and Community Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Eric J Warm
- E.J. Warm is professor of internal medicine and program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6088-2434
| | - Hannah L Anderson
- H.L. Anderson is clinical research associate, Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; ORCID: https://orcid.org/0000-0002-9435-1535
| |
Collapse
|
8
|
Kinnear B, Santen SA, Kelleher M, Martini A, Ferris S, Edje L, Warm EJ, Schumacher DJ. How Does TIMELESS Training Impact Resident Motivation for Learning, Assessment, and Feedback? Evaluating a Competency-Based Time-Variable Training Pilot. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2023; 98:828-835. [PMID: 36656286 DOI: 10.1097/acm.0000000000005147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
PURPOSE As competency-based medical education has become the predominant graduate medical education training model, interest in time-variable training has grown. Despite multiple competency-based time-variable training (CBTVT) pilots ongoing in the United States, little is known about how this training approach impacts learners. The authors aim to explore how their CBTVT pilot program impacted resident motivation for learning, assessment, and feedback. METHOD The authors performed a qualitative educational case study on the Transitioning in Internal Medicine Education Leveraging Entrustment Scores Synthesis (TIMELESS) program at the University of Cincinnati from October 2020 through March 2022. Semistructured interviews were conducted with TIMELESS residents (n = 9) approximately every 6 months to capture experiences over time. The authors used inductive thematic analysis to develop themes and compared their findings with existing theories of learner motivation. RESULTS The authors developed 2 themes: TIMELESS had variable effects on residents' motivation for learning and TIMELESS increased resident engagement with and awareness of the program of assessment. Participants reported increased motivation to learn and seek assessment, though some felt a tension between performance (e.g., advancement through the residency program) and growth (e.g., improvement as a physician). Participants became more aware of the quality of assessments they received, in part due to TIMELESS increasing the perceived stakes of assessment, and reported being more deliberate when assessing other residents. CONCLUSIONS Resident motivation for learning, assessment, and feedback was impacted in ways that the authors contextualize using current theories of learner motivation (i.e., goal orientation theory and attribution theory). Future research should investigate how interventions, such as coaching, guided learner reflection, or various CBTVT implementation strategies, can help keep learners oriented toward mastery learning rather than toward performance.
Collapse
Affiliation(s)
- Benjamin Kinnear
- B. Kinnear is associate professor of internal medicine and pediatrics, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0003-0052-4130
| | - Sally A Santen
- S.A. Santen is professor of emergency medicine, Department of Emergency Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, and University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-8327-8002
| | - Matthew Kelleher
- M. Kelleher is assistant professor of internal medicine and pediatrics, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6400-1745
| | - Abigail Martini
- A. Martini is a clinical research coordinator with emergency medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio
| | - Sarah Ferris
- S. Ferris is a research administrator, Clinical Trials Unit, Michigan Medicine Research, University of Michigan, Ann Arbor, Michigan
| | - Louito Edje
- L. Edje is professor of family and community medicine, Departments of Medical Education and of Family and Community Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Eric J Warm
- E.J. Warm is professor of internal medicine and program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6088-2434
| | - Daniel J Schumacher
- D.J. Schumacher is professor of pediatrics, Cincinnati Children's Hospital Medical Center and Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0001-5507-8452
| |
Collapse
|
9
|
A Tool to Assess Competence in Critical Care Ultrasound Based on Entrustable Professional Activities. ATS Sch 2023; 4:61-75. [PMID: 37089679 PMCID: PMC10117444 DOI: 10.34197/ats-scholar.2022-0063oc] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 11/09/2022] [Indexed: 01/26/2023] Open
Abstract
Background Existing assessment tools for competence in critical care ultrasound (CCUS) have limited scope and interrupt clinical workflow. The framework of entrustable professional activities (EPAs) is well suited to developing an assessment tool that is comprehensive and readily integrated into the intensive care unit (ICU) training environment. Objective This study sought to design an EPA-based tool to assess competence in CCUS for pulmonary and critical care fellows and to assess the validity and reliability of the tool. Methods Eight experts in CCUS met to define the core EPAs for CCUS. A nominal group technique was used to reach consensus. An assessment tool was created based on the EPAs with a modified Ottawa entrustability scale. Trained faculty evaluated pulmonary and critical care fellows using this tool in the ICU over a 6-month study period at a single institution. An assessment of validity of the EPA-based tool is made with four sources of validity evidence: content, response process, reliability, and relation to other variables. Reliability and response process data were generated using generalizability theory analysis to estimate sources of variance in entrustment scores. Analysis of response process validity and validity by relation to other variables was performed using regression models. Results Fifty-four assessments were recorded during the study period, conducted on 23 trainees by 13 faculty. Content validity of the tool was demonstrated using expert consensus and published guidelines from critical care societies to define the EPAs. Response process validity was demonstrated by the low variance in entrustment scores due to evaluators (0.086 or 6%) and high agreement between score and trainee self-assessment (regression coefficient, 0.82; P < 0.0001). Reliability was demonstrated by the high "true" variance in entrustment score attributable to the trainee: 0.674 or 45%. Validity by relation to other variables was demonstrated using regression analysis to show correlation between entrustment score and the number of times a fellow has performed an EPA (regression coefficient, 0.023; P < 0.0001). Conclusion An EPA-based assessment tool for competence in CCUS was created. We obtained sufficient validity evidence on three of the diagnostic EPAs. Procedural EPAs were infrequently assessed, limiting generalizability in this subgroup.
Collapse
|
10
|
Valding B, Monti M, Junod Perron N, Frick S, Jaques C, Nendaz M, Gachoud D. Entrustable professional activities for residency in general internal medicine: a systematic review. Swiss Med Wkly 2022; 152:40002. [PMID: 36473152 DOI: 10.57187/smw.2022.40002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
CONTEXT Entrustable Professional Activities (EPAs) are observable tasks that are regular parts of a physician's daily clinical work. Before being permitted to accomplish these tasks independently, trainees must gain their supervisors' trust. Defining the list of EPAs that should be mastered by the end of a residency is critical to setting clear expectations about autonomous practice. OBJECTIVE To collect all the lists of EPAs defined for residencies in general internal medicine and synthesise them into a reference work useful for developing new lists of EPAs or improving existing ones. METHOD This systematic review searched five databases and relevant grey literature using keywords related to EPAs and postgraduate education, from 2005, when the first article on EPAs was published, to April 2022. Inclusion criteria were the availability of an EPAs list and a focus on general internal medicine. Two reviewers independently selected the studies, extracted data and performed a quality assessment using QATSDD and AACODS tools. Mean values and inter-rater reliability were calculated. RESULTS The review yielded 3292 records, with 16 articles meeting the inclusion criteria, mostly from North America. Synthesising their 16 lists generated 395 EPAs. The reviewers then inductively categoried those EPAs, 308 of which fell into 6 domains, 14 themes and 24 subthemes. The domains were: (1) care and management of the general adult population (n = 103 EPAs); (2) care and management of patients with specific needs (n = 67); (3) care coordination and communication (n = 52); (4) management and leadership (N = 21); (5) healthcare quality, education, and research (n = 41); and (6) miscellaneous (n = 24). The remaining 87 EPAs were disease-specific and did not fit into this categorisation. CONCLUSIONS Categorising EPAs created a unique synthesis of the existing lists of EPAs for educating residents in general internal medicine. This synthesis could be used as a reference for anyone tasked with developing new lists of EPAs or improving existing ones.
Collapse
Affiliation(s)
- Bastien Valding
- Department of Internal Medicine, Lausanne University Hospital, Lausanne, Switzerland
| | - Matteo Monti
- Department of Internal Medicine, Lausanne University Hospital, Lausanne, Switzerland.,Medical Education Unit, University of Lausanne, Switzerland
| | - Noëlle Junod Perron
- Institute of Primary Care, Geneva University Hospitals, Geneva, Switzerland.,Unit for Development and Research in Medical Education, Faculty of Medicine, Geneva, Switzerland
| | - Sonia Frick
- Department of Internal Medicine, Lachen Hospital, Lachen, Switzerland.,School of Medicine, University of Zürich, Switzerland
| | - Cécile Jaques
- Medical Library, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Mathieu Nendaz
- Unit for Development and Research in Medical Education, Faculty of Medicine, Geneva, Switzerland.,Division of Internal Medicine, Geneva University Hospitals, Geneva, Switzerland
| | - David Gachoud
- Department of Internal Medicine, Lausanne University Hospital, Lausanne, Switzerland.,Medical Education Unit, University of Lausanne, Switzerland
| |
Collapse
|
11
|
Schumacher DJ, Michelson C, Winn AS, Turner DA, Elshoff E, Kinnear B. Making prospective entrustment decisions: Knowing limits, seeking help and defaulting. MEDICAL EDUCATION 2022; 56:892-900. [PMID: 35263474 DOI: 10.1111/medu.14797] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 02/28/2022] [Accepted: 03/02/2022] [Indexed: 06/14/2023]
Abstract
INTRODUCTION Many studies focused on entrustment have not considered prospective entrustment decisions, where clinical competency and entrustment committees determine how much supervision trainees will require in the future for tasks occurring in not completely known contexts. The authors sought to explore factors that influence and determine prospective entrustment decisions made by members of such committees in graduate medical education (GME) and undergraduate medical education (UME). METHODS The authors conducted a constructivist grounded theory study with 23 faculty participants from GME and UME clinical competency and entrustment committees in the United States between October 2020 and March 2021. Interviews sought to explore factors and considerations participants weigh in making prospective entrustment decisions about trainees. Data collection and analysis occurred in an iterative fashion, ensuring constant comparison. Theoretical sampling was used to confirm, disconfirm and elaborate on the evolving results. RESULTS Trainees' ability to know limits and seek help is the foundation of participants' prospective entrustment decision making. Most participants, however, describe a presumption of trainee readiness to progress and describe commonly making default prospective entrustment decisions unless 'red flags' in performance are present. Although participants desire sufficient and trusted data about trainee performance to inform decisions, they often lack it. Finally, the perceived permanence and consequences of prospective entrustment decisions influence how participants weigh other factors contributing to prospective entrustment decisions. CONCLUSION Trainees' ability to know limits and seek help appears to be the foundation of prospective entrustment decision making. Training programmes should strive to collect and employ robust data supporting and questioning the presence of these attributes.
Collapse
Affiliation(s)
- Daniel J Schumacher
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | - Catherine Michelson
- Boston Medical Center, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Ariel S Winn
- Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - David A Turner
- American Board of Pediatrics, Chapel Hill, North Carolina, USA
| | | | - Benjamin Kinnear
- Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| |
Collapse
|
12
|
Yun Z, Jing L, Junfei C, Wenjing Z, Jinxiang W, Tong Y, Aijun Z. Entrustable Professional Activities for Chinese Standardized Residency Training in Pediatric Intensive Care Medicine. Front Pediatr 2022; 10:919481. [PMID: 35859946 PMCID: PMC9289143 DOI: 10.3389/fped.2022.919481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Accepted: 06/10/2022] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND Entrustable professional activities (EPAs) were first introduced by Olle ten Cate in 2005. Since then, hundreds of applications in medical research have been reported worldwide. However, few studies discuss the use of EPAs for residency training in pediatric intensive care medicine. We conducted a pilot study of EPA for pediatric intensive care medicine to evaluate the use of EPAs in this subspecialty. MATERIALS AND METHODS A cross-sectional study was implemented in pediatric intensive care medicine standardized residency training at the Qilu Hospital of Shandong University. An electronic survey assessing EPA performance using eight scales composed of 15 categories were distributed among residents and directors. RESULTS A total of 217 director-assessment and 44 residents' self-assessment questionnaires were collected, both demonstrating a rising trend in scores across postgraduate years. There were significant differences in PGY1-vs.-PGY2 and PGY1-vs.-PGY3 director-assessment scores, while there were no differences in PGY2-vs.-PGY3 scores. PGY had a significant effect on the score of each EPA, while position significantly affected the scores of all EPAs except for EPA1 (Admit a patient) and EPA2 (Select and interpret auxiliary examinations). Gender only significantly affected the scores of EPA6 (Report a case), EPA12 (Perform health education), and EPA13 (Inform bad news). CONCLUSION This study indicates that EPA assessments have a certain discriminating capability among different PGYs in Chinese standardized residency training in pediatric intensive care medicine. Postgraduate year, gender, and resident position affected EPA scores to a certain extent. Given the inconsistency between resident-assessed and director-assessed scores, an improved feedback program is needed in the future.
Collapse
Affiliation(s)
- Zhang Yun
- Department of Pediatrics, Qilu Hospital of Shandong University, Jinan, China
| | - Liu Jing
- Department of Biostatistics, School of Public Health, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Chen Junfei
- Department of Pediatric Surgery, Qilu Hospital of Shandong University, Jinan, China
| | - Zhang Wenjing
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan, China
| | - Wu Jinxiang
- Department of Pulmonary and Critical Care Medicine, Qilu Hospital of Shandong University, Jinan, China
| | - Yue Tong
- Medical Training Office, Qilu Hospital of Shandong University, Jinan, China
| | - Zhang Aijun
- Department of Pediatrics, Qilu Hospital of Shandong University, Jinan, China
| |
Collapse
|
13
|
Young JQ, Thakker K, John M, Friedman K, Sugarman R, van Merriënboer JJG, Sewell JL, O'Sullivan PS. Exploring the relationship between emotion and cognitive load types during patient handovers. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2021; 26:1463-1489. [PMID: 34037906 DOI: 10.1007/s10459-021-10053-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Accepted: 05/05/2021] [Indexed: 06/12/2023]
Abstract
Cognitive Load Theory has emerged as an important approach to improving instruction in the health professions workplace, including patient handovers. At the same time, there is growing recognition that emotion influences learning through numerous cognitive processes including motivation, attention, working memory, and long-term memory. This study explores how emotion influences the cognitive load experienced by trainees performing patient handovers. From January to March 2019, 693 (38.7%) of 1807 residents and fellows from a 24-hospital health system in New York city completed a survey after performing a handover. Participants rated their emotional state and cognitive load. The survey included questions about features of the learner, task, and instructional environment. The authors used factor analysis to identify the core dimensions of emotion. Regression analyses explored the relationship between the emotion factors and cognitive load types. Two emotion dimensions were identified representing invigoration and tranquility. In regression analyses, higher levels of invigoration, tranquility, and their interaction were independently associated with lower intrinsic load and extraneous load. The interaction of invigoration and tranquility predicted lower germane load. The addition of the emotion variables to multivariate models including other predictors of cognitive load types significantly increased the amount of variance explained. The study provides a model for measuring emotions in workplace learning. Because emotion appears to have a significant influence on cognitive load types, instructional designers should consider strategies that help trainees regulate emotion in order to reduce cognitive load and improve learning and performance.
Collapse
Affiliation(s)
- John Q Young
- Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | - Krima Thakker
- Division of Education and Training, Zucker Hillside Hospital at Northwell Health, 75-59 263rd Street, Glen Oaks, NY, 10543, USA
| | - Majnu John
- Division of Research, Zucker Hillside Hospital at Northwell Health, Glen Oaks, NY, USA
| | - Karen Friedman
- Department of Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwel, Hempstead, NY, USA
| | | | - Jeroen J G van Merriënboer
- School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Justin L Sewell
- Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| | - Patricia S O'Sullivan
- Department of Medicine & Office of Research and Development in Medical Education, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| |
Collapse
|
14
|
Younas A, Khan RA, Yasmin R. Entrustment in physician-patient communication: a modified Delphi study using the EPA approach. BMC MEDICAL EDUCATION 2021; 21:497. [PMID: 34544394 PMCID: PMC8454034 DOI: 10.1186/s12909-021-02931-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 09/08/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Competency based curricula across the globe stress on the importance of effective physician patient communication. A variety of courses have been used to train physicians for this purpose. However, few of them link competencies with practice resulting in confusions in implementation and assessment. This issue can be resolved by treating certain specific patient communication related tasks as acts of entrustment or entrustable professional activities (EPAs). In this study, we aimed to define a competency-based framework for assessing patient physician communication using the language of EPAs. METHODS A modified Delphi study was conducted in three stages. The first stage was an extensive literature review to identify and elaborate communication related tasks which could be treated as EPAs. The second stage was content validation by medical education experts for clarity and representativeness. The third stage was three iterative rounds of modified Delphi with predefined consensus levels. The McNemar test was used to check response stability in the Delphi Rounds. RESULTS Expert consensus resulted in development of 4 specific EPAs focused on physician-patient communication with their competencies and respective assessment strategies all aiming for level 5 of unsupervised practice. These include Providing information to the patient or their family about diagnosis or prognosis; Breaking Bad news to the patient or their family; Counseling a patient regarding their disease or illness; Resolving conflicts with patients or their families. CONCLUSIONS The EPAs for Physician-patient communication are a step toward an integrative, all-inclusive competency-based assessment framework for patient-centered care. They are meant to improve the quality of physician patient interaction by standardizing communication as a decision of entrustment. The EPAs can be linked to competency frameworks around the world and provide a useful assessment framework for effective training in patient communication. They can be integrated into any post graduate curriculum and can also serve as a self-assessment tool for postgraduate training programs across the globe to improve their patient communication curricula.
Collapse
Affiliation(s)
- Ayesha Younas
- Department of Medical and Dental Education, Shifa College of Dentistry, Shifa Tameer-e-Millat University, Islamabad, Pakistan.
| | - Rehan Ahmed Khan
- Islamic International Medical College, Riphah International University, Rawalpindi, Pakistan
| | - Raheela Yasmin
- RARE Department, Islamic International Medical College, Riphah International University, Rawalpindi, Pakistan
| |
Collapse
|
15
|
van Loon KA, Bonnie LHA, van Dijk N, Scheele F. Benefits of EPAs at risk? The influence of the workplace environment on the uptake of EPAs in EPA-based curricula. PERSPECTIVES ON MEDICAL EDUCATION 2021; 10:200-206. [PMID: 33788161 PMCID: PMC8368760 DOI: 10.1007/s40037-021-00658-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/04/2020] [Revised: 02/18/2021] [Accepted: 02/25/2021] [Indexed: 06/12/2023]
Abstract
INTRODUCTION Entrustable Professional Activities (EPAs) have been applied differently in many postgraduate medical education (PGME) programmes, but the reasons for and the consequences of this variation are not well known. Our objective was to investigate how the uptake of EPAs is influenced by the workplace environment and to what extent the benefits of working with EPAs are at risk when the uptake of EPAs is influenced. This knowledge can be used by curriculum developers who intend to apply EPAs in their curricula. METHOD For this qualitative study, we selected four PGME programmes: General Practice, Clinical Geriatrics, Obstetrics & Gynaecology, and Radiology & Nuclear Medicine. A document analysis was performed on the national training plans, supported by the AMEE Guide for developing EPA-based curricula and relevant EPA-based literature. Interviews were undertaken with medical specialists who had specific involvement in the development of the curricula. Content analysis was employed and illuminated the possible reasons for variation in the uptake of EPAs. RESULTS An important part of the variation in the uptake of EPAs can be explained by environmental factors, such as patient population, the role of the physician in the health-care system, and the setup of local medical care institutions where the training programme takes place. The variation in uptake of EPAs is specifically reflected in the number and breadth of the EPAs, and in the way the entrustment decision is executed within the PGME programme. DISCUSSION Due to variation in uptake of EPAs, the opportunities for trainees to work independently during the training programme might be challenging. EPAs can be implemented in the curriculum of PGME programmes in a meaningful way, but only if the quality of an EPA is assessed, future users are involved in the development, and the key feature of EPAs (the entrustment decision) is retained.
Collapse
Affiliation(s)
- Karsten Arthur van Loon
- School of Medical Sciences of the Amsterdam University Medical Center, Amsterdam, The Netherlands.
| | | | - Nynke van Dijk
- Department of General Practice of the Academic Medical Centre, Amsterdam, The Netherlands
| | - Fedde Scheele
- Health Systems Innovation and Education at the VU University in Amsterdam and Amsterdam UMC, OLVG, Amsterdam, The Netherlands
| |
Collapse
|
16
|
Woodworth GE, Marty AP, Tanaka PP, Ambardekar AP, Chen F, Duncan MJ, Fromer IR, Hallman MR, Klesius LL, Ladlie BL, Mitchell SA, Miller Juve AK, McGrath BJ, Shepler JA, Sims C, Spofford CM, Van Cleve W, Maniker RB. Development and Pilot Testing of Entrustable Professional Activities for US Anesthesiology Residency Training. Anesth Analg 2021; 132:1579-1591. [PMID: 33661789 DOI: 10.1213/ane.0000000000005434] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
BACKGROUND Modern medical education requires frequent competency assessment. The Accreditation Council for Graduate Medical Education (ACGME) provides a descriptive framework of competencies and milestones but does not provide standardized instruments to assess and track trainee competency over time. Entrustable professional activities (EPAs) represent a workplace-based method to assess the achievement of competency milestones at the point-of-care that can be applied to anesthesiology training in the United States. METHODS Experts in education and competency assessment were recruited to participate in a 6-step process using a modified Delphi method with iterative rounds to reach consensus on an entrustment scale, a list of EPAs and procedural skills, detailed definitions for each EPA, a mapping of the EPAs to the ACGME milestones, and a target level of entrustment for graduating US anesthesiology residents for each EPA and procedural skill. The defined EPAs and procedural skills were implemented using a website and mobile app. The assessment system was piloted at 7 anesthesiology residency programs. After 2 months, faculty were surveyed on their attitudes on usability and utility of the assessment system. The number of evaluations submitted per month was collected for 1 year. RESULTS Participants in EPA development included 18 education experts from 11 different programs. The Delphi rounds produced a final list of 20 EPAs, each differentiated as simple or complex, a defined entrustment scale, mapping of the EPAs to milestones, and graduation entrustment targets. A list of 159 procedural skills was similarly developed. Results of the faculty survey demonstrated favorable ratings on all questions regarding app usability as well as the utility of the app and EPA assessments. Over the 2-month pilot period, 1636 EPA and 1427 procedure assessments were submitted. All programs continued to use the app for the remainder of the academic year resulting in 12,641 submitted assessments. CONCLUSIONS A list of 20 anesthesiology EPAs and 159 procedural skills assessments were developed using a rigorous methodology to reach consensus among education experts. The assessments were pilot tested at 7 US anesthesiology residency programs demonstrating the feasibility of implementation using a mobile app and the ability to collect assessment data. Adoption at the pilot sites was variable; however, the use of the system was not mandatory for faculty or trainees at any site.
Collapse
Affiliation(s)
- Glenn E Woodworth
- From the Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon
| | - Adrian P Marty
- Institute of Anesthesiology, University Hospital Zurich, Zurich, Switzerland
| | - Pedro P Tanaka
- Department of Anesthesiology, Stanford University, Stanford, California
| | - Aditee P Ambardekar
- Department of Anesthesiology, University of Texas, Southwestern Medical Center, Dallas, Texas
| | - Fei Chen
- Department of Anesthesiology, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina
| | - Michael J Duncan
- Department of Anesthesiology, University of Missouri-Kansas City, Kansas City, Missouri
| | - Ilana R Fromer
- Department of Anesthesiology, University of Minnesota, Minneapolis, Minnesota
| | - Matthew R Hallman
- Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, Washington
| | - Lisa L Klesius
- Department of Anesthesiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Beth L Ladlie
- Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota
| | | | - Amy K Miller Juve
- Department of Anesthesiology and Perioperative Medicine, Oregon Health & Science University, Portland, Oregon
| | - Brian J McGrath
- Department of Anesthesiology, University of Florida College of Medicine - Jacksonville, Jacksonville, Florida
| | - John A Shepler
- Department of Anesthesiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Charles Sims
- Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota
| | - Christina M Spofford
- Department of Anesthesiology, Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Wil Van Cleve
- Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, Washington
| | - Robert B Maniker
- Department of Anesthesiology, Columbia University, New York, New York
| |
Collapse
|
17
|
Schwartz A, Borman-Shoap E, Carraccio C, Herman B, Hobday PM, Kaul P, Long M, O'Connor M, Mink R, Schumacher DJ, Turner DA, West DC. Learner Levels of Supervision Across the Continuum of Pediatrics Training. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S42-S49. [PMID: 34183601 DOI: 10.1097/acm.0000000000004095] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE To describe trajectories in level of supervision ratings for linked entrustable professional activities (EPAs) among pediatric learners in medical school, residency, fellowship. METHOD The authors performed secondary analyses of 3 linked datasets of level of supervision ratings for the Core EPAs for Entering Residency, the General Pediatrics EPAs, and the Subspecialty Pediatrics EPAs. After identifying 9 activities in common across training stages and aligning the level of entrustment-supervision scales across the datasets, piecewise ordinal and linear mixed effects models were fitted to characterize trajectories of supervision ratings. RESULTS Within each training period, learners were rated as needing less supervision over time in each activity. When transitioning from medical school to residency or during the first year of residency, learners were rated as needing greater supervision in activities related to patient management, teamwork, emergent care, and public health/QI than in earlier periods. When transitioning from residency to fellowship, learners were always rated as needing greater supervision than they had been accorded at the end of residency and sometimes even more than they had been accorded at the start of residency. CONCLUSIONS Although development over training is often imagined as continuous and monotonically increasing competence, this study provides empirical evidence supporting the idea that entrustment is a set of discrete decisions. The relaxation of supervision in training is not a linear process. Even with a seamless curriculum, supervision is tightly bound to the training setting. Several explanations for these findings are discussed.
Collapse
Affiliation(s)
- Alan Schwartz
- A. Schwartz is Michael Reese Endowed Professor of Medical Education and research professor, pediatrics, University of Illinois College of Medicine, Chicago, Illinois, and network director, Association of Pediatric Program Directors (APPD) Longitudinal Educational Assessment Research Network (LEARN), McLean, Virginia; ORCID: http://orcid.org/0000-0003-3809-6637
| | - Emily Borman-Shoap
- E. Borman-Shoap is associate professor, residency program director, and vice chair of education, Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota; ORCID: https://orcid.org/0000-0001-7514-7793
| | - Carol Carraccio
- C. Carraccio was vice president for competency-based medical education, American Board of Pediatrics, Chapel Hill, North Carolina, at the time the work was completed; ORCID: https://orcid.org/0000-0001-5473-8914
| | - Bruce Herman
- B. Herman is professor and vice chair of education, Department of Pediatrics, University of Utah, Salt Lake City, Utah
| | - Patricia M Hobday
- P.M. Hobday is assistant professor and director, Education in Pediatrics Across the Continuum (EPAC), Department of Pediatrics, University of Minnesota, Minneapolis, Minnesota
| | - Paritosh Kaul
- P. Kaul is professor and associate director, EPAC, Department of Pediatrics, University of Colorado, School of Medicine, Denver, Colorado; ORCID: https://orcid.org/0000-0003-4155-7406
| | - Michele Long
- M. Long is professor and director, EPAC, Department of Pediatrics, University of California, San Francisco, San Francisco, California; ORCID: https://orcid.org/0000-0002-8399-5589
| | - Meghan O'Connor
- M. O'Connor is assistant professor and director, EPAC, Department of Pediatrics, University of Utah, Salt Lake City, Utah; ORCID: https://orcid.org/0000-0002-5403-923X
| | - Richard Mink
- R. Mink is professor of pediatrics, David Geffen School of Medicine at the University of California, Los Angeles (UCLA), Los Angeles, California, pediatric critical care fellowship program director, Harbor-UCLA Medical Center, Torrance, California, and director, APPD Subspecialty Pediatrics Investigator Network, McLean, Virginia; ORCID: http://orcid.org/0000-0002-7998-4713
| | - Daniel J Schumacher
- D.J. Schumacher is associate professor, Department of Pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0001-5507-8452
| | - David A Turner
- D.A. Turner was associate professor, Division of Pediatric Critical Care, Department of Pediatrics, and associate director, Graduate Medical Education, Duke University Hospital and Health System, Durham, North Carolina, at the time the work was completed
| | - Daniel C West
- D.C. West is professor and senior director of medical education, Department of Pediatrics, Children's Hospital of Philadelphia and The Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania; ORCID: https://orcid.org/0000-0002-0909-4213
| |
Collapse
|
18
|
Schumacher DJ, Turner DA. Entrustable Professional Activities: Reflecting on Where We Are to Define a Path for the Next Decade. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S1-S5. [PMID: 34183594 DOI: 10.1097/acm.0000000000004097] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Affiliation(s)
- Daniel J Schumacher
- D.J. Schumacher is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center and University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - David A Turner
- D.A. Turner is vice president for competency-based medical education, American Board of Pediatrics, Chapel Hill, North Carolina
| |
Collapse
|
19
|
Kinnear B, Warm EJ, Caretta-Weyer H, Holmboe ES, Turner DA, van der Vleuten C, Schumacher DJ. Entrustment Unpacked: Aligning Purposes, Stakes, and Processes to Enhance Learner Assessment. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S56-S63. [PMID: 34183603 DOI: 10.1097/acm.0000000000004108] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Educators use entrustment, a common framework in competency-based medical education, in multiple ways, including frontline assessment instruments, learner feedback tools, and group decision making within promotions or competence committees. Within these multiple contexts, entrustment decisions can vary in purpose (i.e., intended use), stakes (i.e., perceived risk or consequences), and process (i.e., how entrustment is rendered). Each of these characteristics can be conceptualized as having 2 distinct poles: (1) purpose has formative and summative, (2) stakes has low and high, and (3) process has ad hoc and structured. For each characteristic, entrustment decisions often do not fall squarely at one pole or the other, but rather lie somewhere along a spectrum. While distinct, these continua can, and sometimes should, influence one another, and can be manipulated to optimally integrate entrustment within a program of assessment. In this article, the authors describe each of these continua and depict how key alignments between them can help optimize value when using entrustment in programmatic assessment within competency-based medical education. As they think through these continua, the authors will begin and end with a case study to demonstrate the practical application as it might occur in the clinical learning environment.
Collapse
Affiliation(s)
- Benjamin Kinnear
- B. Kinnear is associate professor of internal medicine and pediatrics, Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0003-0052-4130
| | - Eric J Warm
- E.J. Warm is professor of internal medicine and program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6088-2434
| | - Holly Caretta-Weyer
- H. Caretta-Weyer is assistant professor of emergency medicine, Department of Emergency Medicine, Stanford University School of Medicine, Palo Alto, California; ORCID: https://orcid.org/0000-0002-9783-5797
| | - Eric S Holmboe
- E.S. Holmboe is chief, research, milestones development and evaluation officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0108-6021
| | - David A Turner
- D.A. Turner is vice president, Competency-Based Medical Education, American Board of Pediatrics, Chapel Hill, North Carolina
| | - Cees van der Vleuten
- C. van der Vleuten is professor of education, Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands; ORCID: https://orcid.org/0000-0001-6802-3119
| | - Daniel J Schumacher
- D.J. Schumacher is associate professor of pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0001-5507-8452
| |
Collapse
|
20
|
Schumacher DJ, Martini A, Kinnear B, Kelleher M, Balmer DF, Wurster-Ovalle V, Carraccio C. Facilitators and Inhibitors to Assessing Entrustable Professional Activities in Pediatric Residency. Acad Pediatr 2021; 21:735-741. [PMID: 33221495 DOI: 10.1016/j.acap.2020.11.013] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 11/12/2020] [Accepted: 11/14/2020] [Indexed: 01/09/2023]
Abstract
OBJECTIVE Research on entrustable professional activities (EPAs) has focused on EPA development with little attention paid to implementation experiences. This constructivist grounded theory study sought to begin filling this gap by exploring the experiences of pediatric residency programs with implementing EPA-based assessment. METHODS Interviews with 19 program leader and clinical competency committee participants from 13 sites were held between January and July 2019. Participants were asked about their experiences with implementing EPA-based assessment. Data collection and analysis were iterative. RESULTS Participants described a range of facilitators and inhibitors that influenced their efforts to implement EPA-based assessment. These fell into 4 thematic areas: 1) alignment of EPA construct with local views of performance and assessment, 2) assessing EPAs illuminates holes in the residency curriculum, 3) clinical competency committee structure and process impacts EPA-based assessment, and 4) faculty engagement and development drives ability to assess EPAs. Areas described as facilitators by some participants were noted to be inhibitors for others. The sum of a program's facilitators and inhibitors led to more or less ability to assess EPAs on the whole. Finally, the first area functions differently from the others; it can shift the entire balance toward or away from the ability to assess EPAs overall. CONCLUSION This study helps fill a void in implementation evidence for EPA-based assessment through better understanding of facilitators and inhibitors to such efforts.
Collapse
Affiliation(s)
- Daniel J Schumacher
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine (DJ Schumacher, A Martini, and V Wurster-Ovalle), Cincinnati, Ohio.
| | - Abigail Martini
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine (DJ Schumacher, A Martini, and V Wurster-Ovalle), Cincinnati, Ohio
| | - Benjamin Kinnear
- Departments of Pediatrics and Medicine, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine (B Kinnear and M Kelleher), Cincinnati, Ohio
| | - Matthew Kelleher
- Departments of Pediatrics and Medicine, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine (B Kinnear and M Kelleher), Cincinnati, Ohio
| | - Dorene F Balmer
- Children's Hospital of Philadelphia and University of Pennsylvania Perelman School of Medicine (DF Balmer), Philadelphia, Pa
| | - Victoria Wurster-Ovalle
- Department of Pediatrics, Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine (DJ Schumacher, A Martini, and V Wurster-Ovalle), Cincinnati, Ohio
| | | |
Collapse
|
21
|
Watson A, Leroux T, Ogilvie-Harris D, Nousiainen M, Ferguson PC, Murnahan L, Dwyer T. Entrustable Professional Activities in Orthopaedics. JB JS Open Access 2021; 6:JBJSOA-D-20-00010. [PMID: 34056510 PMCID: PMC8154482 DOI: 10.2106/jbjs.oa.20.00010] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Background An entrustable professional activity (EPA) is defined as a core task of a specialty that is entrusted to a trainee once sufficient competence has been reached. A group of EPAs reflects the activities that clinicians commonly do on a day-to-day basis. Lists of EPAs have been created for most medical subspecialties, but not orthopaedic surgery. The aim of this study was to create a peer-reviewed list of essential EPAs that a resident must perform independently before completing orthopaedic residency training. Methods A focus group of 7 orthopaedic surgeons from the University of Toronto developed a comprehensive list of 285 EPAs. For each subspecialty group, the opinions of at least 15 academic and nonacademic surgeons, as well as subspecialty-trained and non-subspecialty-trained surgeons, were used. The modified Delphi method was used to rank EPAs on a five-point scale from not important to mandatory for a resident to competent before exiting training. Two Delphi rounds were used, using a threshold of >50% of surgeons considering the EPA as mandatory before being considered for the next round. A final list of EPAs was ratified using the focus group of academic surgeons involved in the study. Results Seventy-five (75) of 107 (70%) surgeons invited responded to the survey. Nearly half (129) of the 285 EPAs were discarded after the first round of Delphi. A further 118 EPAs were discarded after the second Delphi round, leaving 49 final EPAs, across 9 subspecialties in orthopaedic surgery. Conclusions Expert consensus was used to create a list of EPAs considered mandatory for completion of resident training in orthopaedics in our province. The final 49 peer-reviewed EPAs will be a valuable benchmark in curriculum design and assessment in orthopaedic surgery in the competency-based era for other programs.
Collapse
Affiliation(s)
- Adam Watson
- Royal Hobart Hospital, Hobart, Tasmania, Australia.,Women's College Hospital, Toronto, Ontario, Canada
| | - Timothy Leroux
- Women's College Hospital, Toronto, Ontario, Canada.,Toronto Western Hospital, Toronto, Ontario, Canada
| | - Darrell Ogilvie-Harris
- Women's College Hospital, Toronto, Ontario, Canada.,Toronto Western Hospital, Toronto, Ontario, Canada
| | - Markku Nousiainen
- Women's College Hospital, Toronto, Ontario, Canada.,Sunnybrook Health Science Centre, Toronto, Ontario, Canada
| | | | - Lucas Murnahan
- Women's College Hospital, Toronto, Ontario, Canada.,Mount Sinai Hospital, Toronto, Ontario, Canada
| | - Tim Dwyer
- Women's College Hospital, Toronto, Ontario, Canada.,Mount Sinai Hospital, Toronto, Ontario, Canada
| |
Collapse
|
22
|
Lin D. Hospitalist Readiness to Assess and Evaluate Resident Progress. South Med J 2021; 114:215-217. [PMID: 33787934 DOI: 10.14423/smj.0000000000001227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Affiliation(s)
- Doris Lin
- From the Department of Medicine, Baylor College of Medicine, Houston, Texas
| |
Collapse
|
23
|
Dehghani Poudeh M, Mohammadi A, Mojtahedzadeh R, Yamani N. Entrustability levels of general internal medicine residents. BMC MEDICAL EDUCATION 2021; 21:185. [PMID: 33766005 PMCID: PMC7995576 DOI: 10.1186/s12909-021-02624-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 03/19/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Entrustable professional activities (EPAs) are those activities that a health professional can perform without direct supervision in a defined environment. Bridging the gap between competencies and learning objectives, EPAs have made assessing the performances of health professional more realistic. The main objective of the present study was developing and customizing EPAs for Iranian Internal Medicine Residency Programs. RESULTS After reviewing the publications, residency curricula and logbooks, and collecting experts' ideas, the initial list of EPAs was developed. Then, in a focus group, the list was refined, the entrustability level of each residency year was determined, and finally, the EPA-competency cross-tab was established, and in the next step, through a one- round Delphi, the results were validated. Twenty-eight EPAs were developed. Some of them were definitely suitable for the higher levels of residency, such that they had to be accomplished under direct supervision until the end of the program. On the other hand, some of EPAs were those that residents, even from the first year, are expected to perform independently or under indirect supervision. Most of the EPAs cover a wide range of competencies. CONCLUSION Determining the entrustability level of each residency year in each EPA as well as the competency- EPA matrix has crucial effect on the quality of the graduates. It seems that our findings are applicable in developing countries like Iran.
Collapse
Affiliation(s)
- Mostafa Dehghani Poudeh
- Department of Medical Education, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Aeen Mohammadi
- Department of E-learning in Medical Education, Virtual School, Center for Excellence in E-learning in Medical Education, Tehran University of Medical Sciences, Tehran, Iran
| | - Rita Mojtahedzadeh
- Department of E-learning in Medical Education, Virtual School, Center for Excellence in E-learning in Medical Education, Tehran University of Medical Sciences, Tehran, Iran
| | - Nikoo Yamani
- Department of Medical Education, Isfahan University of Medical Sciences, Isfahan, Iran
| |
Collapse
|
24
|
Young JQ, Thakker K, John M, Friedman K, Sugarman R, Sewell JL, O'Sullivan PS. Features of the learner, task, and instructional environment that predict cognitive load types during patient handoffs: Implications for instruction. APPLIED COGNITIVE PSYCHOLOGY 2021. [DOI: 10.1002/acp.3803] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- John Q. Young
- Department of Psychiatry Donald and Barbara Zucker School of Medicine at Hofstra/Northwell Hempstead New York USA
- Department of Psychiatry Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Krima Thakker
- Department of Psychiatry Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Majnu John
- Division of Research Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Karen Friedman
- Department of Medicine Donald and Barbara Zucker School of Medicine at Hofstra/Northwell Hempstead New York USA
| | - Rebekah Sugarman
- School of Medicine University of Michigan Ann Arbor Michigan USA
| | - Justin L. Sewell
- Department of Medicine School of Medicine, University of California, San Francisco San Francisco California USA
| | - Patricia S. O'Sullivan
- Department of Medicine School of Medicine, University of California, San Francisco San Francisco California USA
| |
Collapse
|
25
|
White K, Qualtieri J, Courville EL, Beck RC, Alobeid B, Czuchlewski DR, Teruya-Feldstein J, Soma LA, Prakash S, Gratzinger D. Entrustable Professional Activities in Hematopathology Pathology Fellowship Training: Consensus Design and Proposal. Acad Pathol 2021; 8:2374289521990823. [PMID: 33644302 PMCID: PMC7894592 DOI: 10.1177/2374289521990823] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Revised: 11/24/2020] [Accepted: 12/20/2020] [Indexed: 12/16/2022] Open
Abstract
Hematopathology fellowship education has grown in complexity as patient-centered treatment plans have come to depend on integration of clinical, morphologic, immunophenotypic, molecular, and cytogenetic variables. This complexity is in competition with the need for timely hematopathology care with stewardship of patient, laboratory, and societal resources. Accreditation Council for Graduate Medical Education Milestones provide a guidance document for hematopathology training, but fellows and their educators are in need of a simple framework that allows assessment and feedback of growth toward independent hematopathology practice. Entrustable professional activities provide one such framework, and herein, we provide proposed Hematopathology Fellowship Entrustable Professional Activities based on review of pertinent guidelines and literature, with multiple rounds of expert and stakeholder input utilizing a modified mini-Delphi approach. Ten core entrustable professional activities deemed essential for graduating hematopathology fellows were developed together with skills and knowledge statements, example scenarios, and corresponding Accreditation Council for Graduate Medical Education Milestones. Application of these entrustable professional activities in program design, fellow evaluation, and decisions regarding level of supervision is discussed with consideration of benefits and barriers to implementation. These entrustable professional activities may be used by hematopathology fellowship directors and faculty to provide fellows with timely constructive feedback, determine entrustment decisions, provide the Clinical Competency Committee with granular data to support Milestone evaluations, and provide insight into areas of potential improvement in fellowship training. Fellows will benefit from a clear roadmap to independent hematopathology practice with concrete and timely feedback.
Collapse
Affiliation(s)
- Kristie White
- Department of Laboratory Medicine, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| | - Julianne Qualtieri
- Department of Pathology and Laboratory Medicine, University of Pennsylvania School of Medicine, Philadelphia, PA, USA
| | - Elizabeth L. Courville
- Department of Pathology, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Rose C. Beck
- Department of Pathology, University Hospitals of Cleveland/Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Bachir Alobeid
- Department of Pathology and Cell Biology, Columbia University, New York, NY, USA
| | - David R. Czuchlewski
- Department of Pathology, University of New Mexico School of Medicine, Albuquerque, NM, USA
| | - Julie Teruya-Feldstein
- Department of Pathology, Molecular, and Cell-Based Medicine, Icahn School of Medicine, Mount Sinai Health System, New York, NY, USA
| | - Lorinda A. Soma
- Department of Laboratory Medicine and Pathology, University of Washington School of Medicine, Seattle, WA, USA
| | - Sonam Prakash
- Department of Laboratory Medicine, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| | - Dita Gratzinger
- Stanford University School of Medicine, Stanford, CA, USA
- Dita Gratzinger, Department of Pathology, Stanford University School of Medicine, 300 Pasteur Drive, L235, Stanford, CA 94305, USA.
| |
Collapse
|
26
|
Taylor D, Park YS, Smith C, Cate OT, Tekian A. Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice. TEACHING AND LEARNING IN MEDICINE 2021; 33:89-97. [PMID: 32634323 DOI: 10.1080/10401334.2020.1784740] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Issue: Entrustable Professional Activities (EPAs) describe the core tasks health professionals must be competent performing prior to promotion and/or moving into unsupervised practice. When used for learner assessment, they serve as gateways to increased responsibility and autonomy. It follows that identifying and describing EPAs is a high-stakes form of work analysis aiming to describe the core work of a profession. However, hasty creation and adoption of EPAs without rigorous attention to content threatens the quality of judgments subsequently made from using EPA-based assessment tools. There is a clear need for approaches to identify validity evidence for EPAs themselves prior to their deployment in workplace-based assessment. Evidence: For EPAs to realize their potential in health professions education, they must first be constructed to reflect accurately the work of that profession or specialty. If the EPAs fail to do so, they cannot predict a graduate's readiness for or future performance in professional practice. Evaluating the methods used for identification, description, and adoption of EPAs through a construct validity lens helps give leaders and stakeholders of EPA development confidence that the EPAs constructed are, in fact, an accurate representation of the profession's work. Implications: Application of a construct validity lens to EPA development impacts all five commonly followed steps in EPA development: selection of experts; identification of candidate EPAs; iterative revisions; evaluation of proposed EPAs; and formal adoption of EPAs into curricula. It allows curricular developers to avoid pitfalls, bias, and common mistakes. Further, construct validity evidence for EPA development provides assurance that the EPAs adopted are appropriate for use in workplace-based assessment and entrustment decision-making.
Collapse
Affiliation(s)
- David Taylor
- Department of Medicine, Queen's University, Kingston, Canada
| | - Yoon Soo Park
- Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois, USA
| | | | - Olle Ten Cate
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Ara Tekian
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
27
|
St-Onge C, Vachon Lachiver É, Langevin S, Boileau E, Bernier F, Thomas A. Lessons from the implementation of developmental progress assessment: A scoping review. MEDICAL EDUCATION 2020; 54:878-887. [PMID: 32083743 DOI: 10.1111/medu.14136] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Revised: 01/21/2020] [Accepted: 02/06/2020] [Indexed: 06/10/2023]
Abstract
OBJECTIVES Educators and researchers recently implemented developmental progress assessment (DPA) in the context of competency-based education. To reap its anticipated benefits, much still remains to be understood about its implementation. In this study, we aimed to determine the nature and extent of the current evidence on DPA, in an effort to broaden our understanding of the major goals and intended outcomes of DPA as well as the lessons learned from how it has been executed in, or applied across, educational contexts. METHODS We conducted a scoping study based on the methodology of Arksey and O'Malley. Our search strategy yielded 2494 articles. These articles were screened for inclusion and exclusion (90% agreement), and numerical and qualitative data were extracted from 56 articles based on a pre-defined set of charting categories. The thematic analysis of the qualitative data was completed with iterative consultations and discussions until consensus was achieved for the interpretation of the results. RESULTS Tools used to document DPA include scales, milestones and portfolios. Performances were observed in clinical or standardised contexts. We identified seven major themes in our qualitative thematic analysis: (a) underlying aims of DPA; (b) sources of information; (c) barriers; (d) contextual factors that can act as barriers or facilitators to the implementation of DPA; (e) facilitators; (f) observed outcomes, and (g) documented validity evidences. CONCLUSIONS Developmental progress assessment seems to fill a need in the training of future competent health professionals. However, moving forward with a widespread implementation of DPA, factors such as lack of access to user-friendly technology and time to observe performance may render its operationalisation burdensome in the context of competency-based medical education.
Collapse
Affiliation(s)
- Christina St-Onge
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Élise Vachon Lachiver
- Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Serge Langevin
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Elisabeth Boileau
- Department of Family and Emergency Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
| | - Frédéric Bernier
- Department of Medicine, Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
- Faculty of Medicine and Health Sciences, University of Sherbrooke, Sherbrooke, Québec, Canada
- Research Center - Sherbrooke University Hospital Center (CHUS), Integrated Health and Social Service Centers (CISSS) and Integrated University Health and Social Service Centres (CIUSSS), Sherbrooke, Québec, Canada
| | - Aliki Thomas
- School of Physical and Occupational Therapy, McGill University, Montreal, Québec, Canada
| |
Collapse
|
28
|
Pagano MB, Treml A, Stephens LD, Joshi S, Li Y, Lopez-Plaza I, Poyyapakkam S, Schwartz J, Tanhehco Y, Zantek ND. Entrustable professional activities for apheresis medicine education. Transfusion 2020; 60:2432-2440. [PMID: 32757215 DOI: 10.1111/trf.15983] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Revised: 06/24/2020] [Accepted: 06/24/2020] [Indexed: 11/26/2022]
Abstract
BACKGROUND Entrustable professional activities (EPAs) are well-defined, executable, observable, and measurable activities that are performed by a trainee and can be performed independently as training progresses. The purpose of this study is to develop EPAs specific for the practice of apheresis medicine (AM). METHODS Members of the American Society for Apheresis Graduate Medical Education subcommittee developed a list of 28 apheresis medical activities linked to Accreditation Council for Graduate Medical Education milestones and competencies in five areas: (a) consultation, (b) clinical care for therapeutic apheresis, (c) clinical care for donor collections, (d) test optimization, and (e) vascular access. Ten AM experts using a validated tool to measure the quality of the EPAs (QUEPA) evaluated these activities with use of a Likert scale. Per group consensus, an activity was considered acceptable for each domain if it had received an average score greater than 3.7, and it was rated 4 or 5 (agree or strongly agree) by at least 70% of experts. RESULTS Of the 28 activities, 11 did not have acceptable QUEPA scores: 7 activities were rated as unobservable, 4 were rated unfocused, 2 were rated unrealistic and not generalizable, and 2 were rated as not addressing multiple competencies. Four activities had unacceptable scores in more than one domain. Subcommittee members edited these 11 activities over two review cycles to produce a final list of 26 activities. CONCLUSION A set of practical, focused, and observable EPAs in AM were systematically developed. These EPAs can be used to assess and support trainee performance in AM.
Collapse
Affiliation(s)
- Monica B Pagano
- Department of Laboratory Medicine, University of Washington, Seattle, Washington, USA
| | - Angela Treml
- Department of Pathology, Versiti WI, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Laura D Stephens
- Department of Pathology, University of Arizona College of Medicine, Tucson, Arizona, USA
| | - Sarita Joshi
- Department of Medicine, Seattle Children's Hospital, Seattle, Washington, USA
| | - Yanhua Li
- Department of Pathology and Laboratory Medicine, New York University School of Medicine, New York, New York, USA
| | - Ileana Lopez-Plaza
- Department of Pathology and Laboratory Medicine, Henry Ford Health System, Detroit, Michigan, USA
| | | | - Joseph Schwartz
- Transfusion Medicine and Cellular Therapy, Department of Pathology and Cell Biology, Columbia University Irving Medical Center, New York, New York, USA
| | - Yvette Tanhehco
- Transfusion Medicine and Cellular Therapy, Department of Pathology and Cell Biology, Columbia University Irving Medical Center, New York, New York, USA
| | - Nicole D Zantek
- Department of Laboratory Medicine and Pathology, University of Minnesota, Minneapolis, Minnesota, USA
| |
Collapse
|
29
|
Young JQ, Sugarman R, Schwartz J, McClure M, O'Sullivan PS. A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement. PERSPECTIVES ON MEDICAL EDUCATION 2020; 9:210-219. [PMID: 32504446 PMCID: PMC7459074 DOI: 10.1007/s40037-020-00587-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2023]
Abstract
INTRODUCTION Mobile apps that utilize the framework of entrustable professional activities (EPAs) to capture and deliver feedback are being implemented. If EPA apps are to be successfully incorporated into programmatic assessment, a better understanding of how they are experienced by the end-users will be necessary. The authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with an EPA app. METHODS Structured interviews of faculty and residents were conducted with an interview guide based on the CFIR. Transcripts were independently coded by two study authors using directed content analysis. Differences were resolved via consensus. The study team then organized codes into themes relevant to the domains of the CFIR. RESULTS Eight faculty and 10 residents chose to participate in the study. Both faculty and residents found the app easy to use and effective in facilitating feedback immediately after the observed patient encounter. Faculty appreciated how the EPA app forced brief, distilled feedback. Both faculty and residents expressed positive attitudes and perceived the app as aligned with the department's philosophy. Barriers to engagement included faculty not understanding the EPA framework and scale, competing clinical demands, residents preferring more detailed feedback and both faculty and residents noting that the app's feedback should be complemented by a tool that generates more systematic, nuanced, and comprehensive feedback. Residents rarely if ever returned to the feedback after initial receipt. DISCUSSION This study identified key enablers and barriers to engagement with the EPA app. The findings provide guidance for future research and implementation efforts focused on the use of mobile platforms to capture direct observation feedback.
Collapse
Affiliation(s)
- John Q Young
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA.
| | - Rebekah Sugarman
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Jessica Schwartz
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Matthew McClure
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Patricia S O'Sullivan
- Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, USA
| |
Collapse
|
30
|
Levine D, Gadivemula J, Kutaimy R, Kamatam S, Sarvadevabatla N, Lohia P. Analysis of patient safety messages delivered and received during clinical rounds. BMJ Open Qual 2020; 9:bmjoq-2019-000869. [PMID: 32719084 PMCID: PMC7388879 DOI: 10.1136/bmjoq-2019-000869] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Revised: 05/28/2020] [Accepted: 07/03/2020] [Indexed: 11/16/2022] Open
Abstract
Background Multiple modalities are available to introduce patient safety training to healthcare professionals. In internal medicine, clinical rounds have always played an important role in education; however, the patient safety content taught at the point of care is not well studied. We studied, both quantitatively and qualitatively, the number and nature of patient safety messages delivered by attending physicians to determine what is taught at the point of care and how well this is recognised and recalled by attending physicians, residents and medical students. Methods This prospective mixed methods study was conducted on the medicine teaching service. Clinical rounds were audio-recorded. Immediately after rounds, attending physicians, residents and students completed a short survey card identifying the number and type of educational messages they immediately recalled teaching or hearing. Independent t-test was used to compare differences in the number of messages delivered by attendings and recalled by trainees. One-way analysis of variance was used to compare differences in messages delivered by attending physicians compared with trainees. Recordings were transcribed and analysed qualitatively for patient safety content. Results Trainees recalled more educational messages than attendings recalled teaching in all educational domains. Safety messages comprised 17.5% of educational messages. The average number of patient safety messages recalled per session was 1.08 per attending physicians, 1.84 per resident and 2.50 per student. Residents recalled 56.4% of safety messages delivered; students recalled 76.7% of safety messages. Conclusion Patient safety is a focus of teaching during clinical rounds and provides meaningful opportunities to train students and residents to practice safe patient care.
Collapse
Affiliation(s)
- Diane Levine
- Internal Medicine, Wayne State University, Detroit, Michigan, USA .,Internal Medicine, Detroit Medical Center, Wayne State University, Detroit, MI, USA
| | - Jaya Gadivemula
- Internal Medicine, Detroit Medical Center, Wayne State University, Detroit, MI, USA
| | - Raya Kutaimy
- Internal Medicine, Detroit Medical Center, Wayne State University, Detroit, MI, USA
| | - Srinivasa Kamatam
- Internal Medicine, Detroit Medical Center, Wayne State University, Detroit, MI, USA
| | | | - Prateek Lohia
- Internal Medicine, Detroit Medical Center, Wayne State University, Detroit, MI, USA
| |
Collapse
|
31
|
Marty AP, Schmelzer S, Thomasin RA, Braun J, Zalunardo MP, Spahn DR, Breckwoldt J. Agreement between trainees and supervisors on first-year entrustable professional activities for anaesthesia training. Br J Anaesth 2020; 125:98-103. [DOI: 10.1016/j.bja.2020.04.009] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2019] [Revised: 04/07/2020] [Accepted: 04/11/2020] [Indexed: 11/28/2022] Open
|
32
|
Soran C, Laponis R, Summerville S, Thompson V, Eastburn A, O'Sullivan P, Julian KA. Identifying Entrustable Professional Activities for Internal Medicine Residents in Ambulatory Continuity Practice. J Gen Intern Med 2020; 35:1917-1919. [PMID: 31637647 PMCID: PMC7280367 DOI: 10.1007/s11606-019-05430-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/16/2019] [Revised: 08/27/2019] [Accepted: 09/24/2019] [Indexed: 10/25/2022]
Affiliation(s)
- Christine Soran
- Division of General Internal Medicine, Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, CA, USA.
| | - Ryan Laponis
- Division of General Internal Medicine, University of California San Francisco, San Francisco, CA, USA.,Internal Medicine Residency Program, University of California San Francisco, San Francisco, CA, USA
| | - Sarah Summerville
- Division of General Internal Medicine, University of California San Francisco, San Francisco, CA, USA
| | - Vanessa Thompson
- Division of General Internal Medicine, Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, CA, USA.,Internal Medicine Residency Program, University of California San Francisco, San Francisco, CA, USA
| | - Abigail Eastburn
- Division of General Internal Medicine, San Francisco Veterans Affairs Medical Center, University of California San Francisco, San Francisco, CA, USA
| | - Patricia O'Sullivan
- Department of Medicine, University of California San Francisco, San Francisco, CA, USA.,Department of Surgery, University of California San Francisco, San Francisco, CA, USA.,Research and Development in Medical Education, University of California San Francisco, San Francisco, CA, USA
| | - Katherine A Julian
- Division of General Internal Medicine, Zuckerberg San Francisco General Hospital, University of California San Francisco, San Francisco, CA, USA.,Internal Medicine Residency Program, University of California San Francisco, San Francisco, CA, USA
| |
Collapse
|
33
|
Lau ST, Ang E, Samarasekera DD, Shorey S. Development of undergraduate nursing entrustable professional activities to enhance clinical care and practice. NURSE EDUCATION TODAY 2020; 87:104347. [PMID: 32004948 DOI: 10.1016/j.nedt.2020.104347] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Revised: 11/22/2019] [Accepted: 01/20/2020] [Indexed: 05/22/2023]
Abstract
BACKGROUND Nursing education adopts a time-based approach to assess the multifaceted competencies of student nurses. The competency-based approach is preferred historically as it is practical and ensures that individuals deliver effective healthcare practice. However, there remains a gap on how these competencies are actually applied in nursing practice. To facilitate the connection between competencies, competency-based education, and nursing practice, entrustable professional activities (EPAs) can be utilized to translate competencies into clinical practice. EPAs have shown promising results across multiple healthcare specialties and have become the current driving force to facilitate nursing care and practice. Given the limited information of EPAs in nursing education, it is an opportune time to develop EPAs specific to nursing care and practice. OBJECTIVES To provide a detailed breakdown on the development of EPAs in nursing education to inform clinical care and practice. METHODS The development stages of EPAs included: i) the formation of a team, ii) the development of the conceptual framework, and iii) the pooling, reviewing, and revising of core EPAs. RESULTS A total of ten core EPAs were developed, with sub-EPAs nested within these core EPAs. The EPAs include: 1) patient engagement, 2) patient care and practice, 3) care management, 4) common procedures, 5) safety, 6) urgent care, 7) transition care, 8) patient education, 9) interprofessional collaboration, and 10) palliative care. CONCLUSION The development of EPAs specific to nursing care and practice may offer nursing programs a guide to assist with curricula planning and a basis for developing entrustment assessment tools. The unfamiliarity of EPAs in nursing education may pose as implementation challenges to EPAs. Future research is warranted to evaluate and improve the developed EPAs.
Collapse
Affiliation(s)
- Siew Tiang Lau
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Level 2, Clinical Research Centre, Block MD11, 10 Medical Drive, Singapore 117597, Singapore.
| | - Emily Ang
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Level 2, Clinical Research Centre, Block MD11, 10 Medical Drive, Singapore 117597, Singapore.
| | - Dujeepa D Samarasekera
- Centre for Medical Education, National University of Singapore, Level 5, Clinical Research Centre, 10 Medical Drive, Singapore 117597, Singapore.
| | - Shefaly Shorey
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Level 2, Clinical Research Centre, Block MD11, 10 Medical Drive, Singapore 117597, Singapore.
| |
Collapse
|
34
|
Schumacher DJ, West DC, Schwartz A, Li ST, Millstein L, Griego EC, Turner T, Herman BE, Englander R, Hemond J, Hudson V, Newhall L, McNeal Trice K, Baughn J, Giudice E, Famiglietti H, Tolentino J, Gifford K, Carraccio C. Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities. JAMA Netw Open 2020; 3:e1919316. [PMID: 31940042 PMCID: PMC6991321 DOI: 10.1001/jamanetworkopen.2019.19316] [Citation(s) in RCA: 61] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
IMPORTANCE Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. OBJECTIVE To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. DESIGN, SETTING, AND PARTICIPANTS Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. INTERVENTIONS Longitudinal, prospective assessment using EPAs. MAIN OUTCOMES AND MEASURES Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. RESULTS Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). CONCLUSIONS AND RELEVANCE This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.
Collapse
Affiliation(s)
- Daniel J. Schumacher
- Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
- University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Daniel C. West
- Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
- Department of Pediatrics, University of Pennsylvania, Philadelphia
| | - Alan Schwartz
- Department of Medical Education, University of Illinois at Chicago
- Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network, McLean, Virginia
- Department of Pediatrics, University of Illinois at Chicago
| | - Su-Ting Li
- Department of Pediatrics at the University of California Davis Health, Sacramento
| | - Leah Millstein
- Department of Pediatrics, University of Maryland School of Medicine, Baltimore
| | - Elena C. Griego
- Department of Pediatrics, Seattle Children’s Hospital, Seattle, Washington
| | - Teri Turner
- Department of Pediatrics, Texas Children’s Hospital/Baylor College of Medicine, Houston
| | - Bruce E. Herman
- Department of Pediatrics, University of Utah, Salt Lake City
| | - Robert Englander
- Department of Pediatrics, University of Minnesota Medical School, Minneapolis
| | - Joni Hemond
- Department of Pediatrics, University of Utah, Salt Lake City
| | - Valera Hudson
- Department of Pediatrics, Children’s Hospital of Georgia/Augusta University, Augusta
| | - Lauren Newhall
- Department of Pediatrics, Children’s Hospital of Georgia/Augusta University, Augusta
| | | | - Julie Baughn
- Department of Pediatrics, Mayo Medical School, Rochester, Minnesota
| | - Erin Giudice
- Department of Pediatrics, University of Maryland School of Medicine, Baltimore
| | | | - Jonathan Tolentino
- Department of Pediatrics, Stony Brook University, Stony Brook, New York
- Department of Internal Medicine, Stony Brook University, Stony Brook, New York
| | - Kimberly Gifford
- Department of Pediatrics, Dartmouth University, Lebanon, New Hampshire
| | | |
Collapse
|
35
|
Dunne D, Green M, Tetrault J, Barakat LA. Development of a Novel Competency-Based Evaluation System for HIV Primary Care Training: the HIV Entrustable Professional Activities. J Gen Intern Med 2020; 35:331-335. [PMID: 31667752 PMCID: PMC6957645 DOI: 10.1007/s11606-019-04956-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND There is an anticipated shortage of primary care providers trained to care for patients with HIV. The Yale School of Medicine developed and implemented a novel HIV training track within our Primary Care Internal Medicine Residency Program. A set of 12 Entrustable Professional Activities (EPAs) were developed to guide curriculum development and resident assessment. AIM To describe the process of implementing a novel EPA-based curriculum for the HIV Primary Care Training Track including EPA-based trainee evaluation tools. PARTICIPANTS/SETTINGS Two to three residents were enrolled annually from 2012 to 2017 (total n = 11). Training sites included the outpatient academic center HIV clinic and inpatient HIV ward. PROGRAM DESCRIPTION An expert panel developed 12 HIV-specific EPAs. These were mapped to curricular and reporting internal medicine milestones. Curricular activities and evaluation tools were developed to guide EPA progress. PROGRAM EVALUATION Graduating residents were ready for unsupervised practice in 91% of EPAs at the end of the 3-year program. DISCUSSION Development of HIV-specific training EPAs was effective for driving curricular development and resident evaluation, and served as an effective method to communicate expectations to resident participants. These HIV-specific EPAs could serve as a useful template to enhance HIV education in academic settings.
Collapse
Affiliation(s)
- Dana Dunne
- Section of Infectious Diseases, Yale University School of Medicine, New Haven, CT, USA.
| | - Michael Green
- Section of General Internal Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Jeanette Tetrault
- Section of General Internal Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Lydia Aoun Barakat
- Section of Infectious Diseases, Yale University School of Medicine, New Haven, CT, USA
| |
Collapse
|
36
|
Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. MEDICAL EDUCATION 2019; 53:766-777. [PMID: 30945329 DOI: 10.1111/medu.13879] [Citation(s) in RCA: 114] [Impact Index Per Article: 22.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Revised: 11/30/2018] [Accepted: 02/22/2019] [Indexed: 05/13/2023]
Abstract
CONTEXT The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care. METHOD Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes. RESULTS Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs). CONCLUSIONS Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.
Collapse
Affiliation(s)
- Shefaly Shorey
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | | | - Siew Tiang Lau
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | - Emily Ang
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| |
Collapse
|
37
|
O'Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. MEDICAL EDUCATION 2019; 53:234-249. [PMID: 30609093 DOI: 10.1111/medu.13792] [Citation(s) in RCA: 101] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Revised: 11/05/2018] [Accepted: 11/20/2018] [Indexed: 05/13/2023]
Abstract
PURPOSE This review aimed to synthesise some of the extant work on the use of entrustable professional activities (EPAs) for postgraduate physicians, to assess the quality of the work and provide direction for future research and practice. METHOD Systematic searches were conducted within five electronic databases (Medline, Scopus, Web of Science, PsycINFO and CINAHL) in September 2018. Reference lists, Google Scholar and Google were also searched. Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). RESULTS In total, 49 studies were included, classified as Development of EPAs (n = 37; 76% of total included), Implementation and/or assessment of EPAs (n = 10; 20%), or both (n = 2; 4%). EPAs were described for numerous specialties, including internal medicine (n = 14; 36%), paediatrics (n = 8; 21%) and psychiatry (n = 4; 10%). Of the development studies, 92% utilised more than one method to generate EPAs. The two most commonly used methods were developing initial EPAs in a working group, (n = 27; 69%) and revising through deliberation (n = 21; 54%). Development papers were of variable quality (mean QATSDD score = 20, range 6-41). Implementation and assessment studies utilised methods that included observing trainee performance (n = 6; 50%) and enrolling trainees in competency-based curricula, which included EPAs (n = 4; 33%). The methodological quality of these implementation studies varied (mean QATSDD score = 19.5, range = 6-32). CONCLUSIONS This review highlighted a need for: (i) consideration of best practice guidelines for EPA development; (ii) focus on the methodological quality of research on EPA development and of EPAs, and (iii) further work investigating the implementation of EPAs in the curriculum.
Collapse
Affiliation(s)
- Emily O'Dowd
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Sinéad Lydon
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| | - Paul O'Connor
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Caoimhe Madden
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Dara Byrne
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| |
Collapse
|
38
|
Duijn CCMA, Ten Cate O, Kremer WDJ, Bok HGJ. The Development of Entrustable Professional Activities for Competency-Based Veterinary Education in Farm Animal Health. JOURNAL OF VETERINARY MEDICAL EDUCATION 2018; 46:218-224. [PMID: 30565977 DOI: 10.3138/jvme.0617-073r] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Entrustable professional activities (EPAs) are professional tasks that can be entrusted to a student under a given level of supervision once he or she has demonstrated competence in these tasks. The EPA construct was conceived to increase transparency in objectives for clinical workplace learning and to help ensure patient safety and the quality of care. A first step in implementing EPAs in a veterinary curriculum is to identify the core EPAs of the profession. The aim of this study was to develop EPAs for farm animal health. An initial set of 36 EPAs for farm animal health was prepared by a team of six veterinarians and curriculum developers and used in a modified Delphi study. In this iterative process, the EPAs were evaluated until higher than 80% agreement was reached. Of 83 veterinarians who participated, 39 (47%) completed the Delphi procedure. After two rounds, the panel reached consensus. A small expert group further refined and reorganized the EPAs for educational purposes into seven core EPAs for farm animal health and 29 sub-EPAs. This study is an important step in optimizing competency-based training in veterinary medicine. Future steps are to implement EPAs in the curriculum and train supervisors to assess students' ability to perform EPAs with increasing levels of independence.
Collapse
|
39
|
Verma AA, Guo Y, Kwan JL, Lapointe-Shaw L, Rawal S, Tang T, Weinerman A, Razak F. Prevalence and Costs of Discharge Diagnoses in Inpatient General Internal Medicine: a Multi-center Cross-sectional Study. J Gen Intern Med 2018; 33:1899-1904. [PMID: 30054888 PMCID: PMC6206337 DOI: 10.1007/s11606-018-4591-7] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2018] [Revised: 06/06/2018] [Accepted: 07/12/2018] [Indexed: 11/25/2022]
Abstract
BACKGROUND Understanding the most common and costly conditions treated by inpatient general medical services is important for implementing quality improvement, developing health policy, conducting research, and designing medical education. OBJECTIVE To determine the prevalence and cost of conditions treated on general internal medicine (GIM) inpatient services. DESIGN Retrospective cross-sectional study involving 7 hospital sites in Toronto, Canada. PARTICIPANTS All patients discharged between April 1, 2010 and March 31, 2015 who were admitted to or discharged from an inpatient GIM service. MAIN MEASURES Hospital administrative data were used to identify diagnoses and costs associated with admissions. The primary discharge diagnosis was identified for each admission and categorized into clinically relevant and mutually exclusive categories using the Clinical Classifications Software (CCS) tool. KEY RESULTS Among 148,442 admissions, the most common primary discharge diagnoses were heart failure (5.1%), pneumonia (5.0%), urinary tract infection (4.6%), chronic obstructive pulmonary disease (4.5%), and stroke (4.4%). The prevalence of the 20 most common conditions was significantly correlated across hospitals (correlation coefficients ranging from 0.55 to 0.95, p ≤ 0.01 for all comparisons). No single condition represented more than 5.1% of all admissions or more than 7.9% of admissions at any hospital site. The costliest conditions were stroke (median cost $7122, interquartile range 5587-12,354, total cost $94,199,422, representing 6.0% of all costs) and the group of delirium, dementia, and cognitive disorders (median cost $12,831, IQR 9539-17,509, total cost $77,372,541, representing 4.9% of all costs). The 10 most common conditions accounted for only 36.2% of hospitalizations and 36.8% of total costs. The remaining hospitalizations included 223 different CCS conditions. CONCLUSIONS GIM services care for a markedly heterogeneous population but the most common conditions were similar across 7 hospitals. The diversity of conditions cared for in GIM may be challenging for healthcare delivery and quality improvement. Initiatives that cut across individual diseases to address processes of care, patient experience, and functional outcomes may be more relevant to a greater proportion of the GIM population than disease-specific efforts.
Collapse
Affiliation(s)
- Amol A Verma
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Room 714-3, 2 Queen St. East, Toronto, Ontario, M5C 3G7, Canada.
- Department of Medicine, University of Toronto, Toronto, Canada.
| | - Yishan Guo
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Room 714-3, 2 Queen St. East, Toronto, Ontario, M5C 3G7, Canada
| | - Janice L Kwan
- Department of Medicine, University of Toronto, Toronto, Canada
- Department of Medicine, Mount Sinai Hospital, Toronto, Ontario, Canada
| | | | - Shail Rawal
- Department of Medicine, University of Toronto, Toronto, Canada
- Division of General Internal Medicine, University Health Network, Toronto, Canada
| | - Terence Tang
- Department of Medicine, University of Toronto, Toronto, Canada
- Institute for Better Health, Trillium Health Partners, Toronto, Canada
| | - Adina Weinerman
- Department of Medicine, University of Toronto, Toronto, Canada
- Sunnybrook Health Sciences Centre, Toronto, Canada
| | - Fahad Razak
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Room 714-3, 2 Queen St. East, Toronto, Ontario, M5C 3G7, Canada
- Department of Medicine, University of Toronto, Toronto, Canada
- Harvard Center for Population and Development Studies, Cambridge, USA
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
| |
Collapse
|
40
|
van Loon KA. Formal Versus Informal Judgments: Faculty Experiences With Entrustment in Graduate Medical Education. J Grad Med Educ 2018; 10:537-542. [PMID: 30386479 PMCID: PMC6194887 DOI: 10.4300/jgme-d-18-00120.1] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/06/2018] [Revised: 05/26/2018] [Accepted: 06/01/2018] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Entrustment of residents has been formalized in many competency-based graduate medical education programs, but its relationship with informal decisions to entrust residents with clinical tasks is unclear. In addition, the effects of formal entrustment on training practice are still unknown. OBJECTIVE Our objective was to learn from faculty members in training programs with extensive experience in formal entrustment how formal entrustment relates to informal entrustment decisions. METHODS A questionnaire was e-mailed to all Dutch obstetrics and gynecology program directors to gather information on how faculty entrusts residents with clinical independence. We also interviewed faculty members to explore the relationship between formal entrustment and informal entrustment. Interviews were analyzed with conventional content analysis. RESULTS Of 92 programs, 54 program directors completed the questionnaire (59% response rate). Results showed that formal entrustment was seen as valuable for generating formative feedback and giving insight into residents' progress in technical competencies. Interviewed faculty members (n = 12) used both formal and informal entrustment to determine the level of resident independence. Faculty reported they tended to favor informal entrustment because it can be reconsidered. In contrast, formal entrustment was reported to feel like a fixed state. CONCLUSIONS In a graduate medical education program where formal entrustment has been used for more than a decade, faculty used a combination of formal and informal entrustment. Informal entrustment is key in deciding if a resident can work independently. Faculty members reported being unsure how to optimally use formal entrustment in practice next to their informal decisions.
Collapse
|
41
|
Young JQ, Hasser C, Hung EK, Kusz M, O'Sullivan PS, Stewart C, Weiss A, Williams N. Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:1048-1054. [PMID: 29166349 DOI: 10.1097/acm.0000000000002058] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
PURPOSE To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties. METHOD A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0. RESULTS Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2. CONCLUSIONS The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.
Collapse
Affiliation(s)
- John Q Young
- J.Q. Young is professor, Department of Psychiatry, Zucker School of Medicine at Hofstra/Northwell, New York, New York. C. Hasser is assistant professor, Department of Psychiatry, UCSF School of Medicine, San Francisco, California. E.K. Hung is associate professor, Department of Psychiatry, UCSF School of Medicine, San Francisco, California. M. Kusz is research assistant, Department of Psychiatry, Hofstra Northwell School of Medicine, New York, New York. P.S. O'Sullivan is professor, Department of Medicine and Surgery, UCSF School of Medicine, San Francisco, California. C. Stewart is assistant professor, Department of Psychiatry, Georgetown School of Medicine, Washington, DC. A. Weiss is associate professor, Department of Psychiatry and Behavioral Sciences, Albert Einstein School of Medicine, New York, New York. N. Williams is professor, Department of Psychiatry, University of Iowa Carver College of Medicine, Iowa City, Iowa
| | | | | | | | | | | | | | | |
Collapse
|
42
|
Horak H, Englander R, Barratt D, Kraakevik J, Soni M, Tiryaki E. Entrustable professional activities. Neurology 2018; 90:326-332. [DOI: 10.1212/wnl.0000000000004947] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2017] [Accepted: 11/21/2017] [Indexed: 11/15/2022] Open
Abstract
Medical education is currently undergoing a paradigm shift from process-based to competency-based education, focused on measuring the desired competence of a physician. In an attempt to improve the assessment framework used for medical education, the concept of entrustable professional activities (EPAs) has gained traction. EPAs are defined as professional activities that can be entrusted to an individual in a clinical context. The Association of American Medical Colleges (AAMC) defined a set of 13 such EPAs to define the core of what all students should be able to do on day 1 of residency, regardless of specialty choice. The AAMC is currently piloting these EPAs with 10 medical schools to determine if EPAs can be used as a way to observe, measure, and entrust medical students with core clinical activities by the end of the clinical immersion experiences of the third year. The specialty of pediatrics is piloting the use of specialty-specific EPAs at 5 medical schools to assess readiness for transitions from medical school into pediatric residency training and practice. To date, no neurology-specific EPAs have been published for use in neurology clerkships or neurology residencies. This article introduces the concept of EPAs in the context of competency-based medical education and describes how EPAs might be relevant and applicable in neurologic education across the continuum. The Undergraduate Education Subcommittee of the American Academy of Neurology advocates for a proactive approach to incorporating core EPAs in undergraduate medical education and to considering an EPA-based specialty-specific assessment framework for neurology.
Collapse
|
43
|
Chuang LL, Hsieh MC. A competency-based approach to critical care education. Tzu Chi Med J 2018; 30:148-151. [PMID: 30069122 PMCID: PMC6047323 DOI: 10.4103/tcmj.tcmj_84_18] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
The medical education environment is rapidly changing. Competency-based medical education (CBME) is a great advance, but operationalizing competencies for teaching and assessment is problematic. Entrustable professional activities (EPAs) can revitalize CBME by connecting competencies to practice, creating flexibility in programs. CBME requires and deepens the nature of workplace-based assessments. It is important to use EPAs to verify residents’ ability to care for critically ill patients unsupervised in simulation education.
Collapse
Affiliation(s)
- Li-Liang Chuang
- Department of Internal Medicine, Buddhist Tzu Chi General Hospital, Hualien, Taiwan
| | - Ming-Chen Hsieh
- Department of Internal Medicine, Buddhist Tzu Chi General Hospital, Hualien, Taiwan.,Department of Medical Education, Buddhist Tzu Chi General Hospital, Hualien, Taiwan.,School of Medicine, Tzu Chi University, Hualien, Taiwan
| |
Collapse
|
44
|
Berz JPB, Cheng T, Quintiliani LM. Milestones-based direct observation tools in internal medicine resident continuity clinic. BMC MEDICAL EDUCATION 2017; 17:240. [PMID: 29202742 PMCID: PMC5715641 DOI: 10.1186/s12909-017-1077-y] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/11/2016] [Accepted: 11/16/2017] [Indexed: 06/07/2023]
Abstract
BACKGROUND With milestones-based assessment, there is an increased need for tools to facilitate direct observation of clinical trainees. This study was designed to compare a Mini-CEX tool to new direct observation tools (DOTs) linked to internal medicine milestones. METHODS A web based survey was used to examine satisfaction and usefulness of DOTs compared to the Mini-CEX. Residents and preceptors were surveyed three times over 6 months with half serving as control (using mini-CEX) compared to those using the DOTs. Likert scale quantitative answers and qualitative comments were analyzed using generalized estimating equations. RESULTS Out of 94 residents and 32 faculty 81 and 90% completed the survey for at least one time point. In adjusted models, there was no significant change in resident evaluation comparing the tools on a number of questions including overall satisfaction and resident perception of receiving high quality feedback. By contrast, faculty preceptors reported increased ratings on many of the questions evaluating their use of the new tools including ability to provide higher quality feedback and overall satisfaction. A number of challenges and benefits of the new tools were identified in qualitative feedback by both residents and preceptors. CONCLUSIONS All parties recognized the value and limitations of direct observation. Overall these new office based DOTs were perceived similarly to the mini-CEX by residents while faculty reported higher satisfaction. The DOTs are a useful addition to the tool box available for the assessment of clinical skills of medical trainees, especially from the viewpoint of faculty preceptors.
Collapse
Affiliation(s)
- Jonathan P B Berz
- Boston University School of Medicine/Boston Medical Center, 801 Massachusetts Avenue, Crosstown 2, Boston, MA, 02118, USA.
| | - Teresa Cheng
- Boston University School of Medicine/Boston Medical Center, 801 Massachusetts Avenue, Crosstown 2, Boston, MA, 02118, USA
| | - Lisa M Quintiliani
- Boston University School of Medicine/Boston Medical Center, 801 Massachusetts Avenue, Crosstown 2, Boston, MA, 02118, USA
| |
Collapse
|
45
|
Choosing entrustable professional activities for neonatology: a Delphi study. J Perinatol 2017; 37:1335-1340. [PMID: 28933775 DOI: 10.1038/jp.2017.144] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/07/2017] [Revised: 06/15/2017] [Accepted: 07/05/2017] [Indexed: 11/09/2022]
Abstract
OBJECTIVE To assess candidate neonatology EPAs taken from separate but overlapping sets from two organizations. STUDY DESIGN Using a Delphi process, we asked that neonatology fellowship directors (1) assess importance and scope of 19 candidate EPAs, and (2) propose additional EPAs if necessary. In round 2, we sought clarification of first round responses and evaluated proposed additional EPAs. RESULTS Twenty program directors participated. In round 1, all EPAs were scored as important, but four were overly broad. In round 2, respondents rejected proposed subdivisions of one overly broad EPA, retaining it as originally proposed. Specification of entrustment criteria improved the scope of the three other broad EPAs. However, after specification, they were re-rated as insufficiently important and therefore rejected. Neither newly proposed EPA from round 1 was rated as sufficiently important. CONCLUSION The Delphi process yielded 13 EPAs with which to assess capability to practice clinical neonatology.
Collapse
|
46
|
|
47
|
Tunnicliffe DJ, Singh-Grewal D, Craig JC, Jesudason S, Tugwell P, Lin MW, O'Neill SG, Sumpton D, Tong A. Perspectives of Medical Specialists From Different Disciplines on the Management of Systemic Lupus Erythematosus: An Interview Study. Arthritis Care Res (Hoboken) 2017; 70:1284-1293. [PMID: 29136338 DOI: 10.1002/acr.23469] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2017] [Accepted: 11/07/2017] [Indexed: 11/10/2022]
Abstract
OBJECTIVE Systemic lupus erythematosus (SLE) is a complex autoimmune disease that can affect multiple organ systems, with specialists from many disciplines often involved, which may lead to inconsistent care. We aimed to describe the attitudes and perspectives of specialists from different medical disciplines on the management of people with SLE. METHODS Face-to-face semistructured interviews were conducted with rheumatologists (n = 16), nephrologists (n = 16), and immunologists (n = 11) providing care to adults with SLE from 19 centers across Australia in 2015. All interviews were transcribed and analyzed thematically. RESULTS Five themes were identified: uncertainties in judgments (hampered by unknown and unclear etiology, inapplicable evidence, comprehending information dispersion), reflexive responses (anchoring to specialty training, anticipating outcomes, avoiding disaster, empathy for the vulnerable), overarching duty to patients (achieving patient priorities, maximizing adherence, controlling the disease, providing legitimate information, having adequate and relevant expertise), safeguarding professional opportunities (diversifying clinical skills, protecting colleagues' interests), and optimizing access to treatment (capitalizing on multidisciplinary care, acquiring breakthrough therapies). CONCLUSION Specialists strive to deliver evidence-informed patient-centered care, but recognize that they are anchored by their training. To overcome uncertainties in clinical management due to lack of high-quality evidence and specialty silo structures, specialists translated evidence from other disease settings and collaborated with other specialists in routine care. Developing robust evidence, tools to support evidence-informed decisions, and multidisciplinary shared-care pathways may improve the management of people with this complex disease.
Collapse
Affiliation(s)
- David J Tunnicliffe
- Sydney School of Public Health, University of Sydney, and Children's Hospital at Westmead, Sydney, Australia
| | - Davinder Singh-Grewal
- Sydney Medical School, Sydney Children's Hospital Network, and University of New South Wales, Sydney, Australia
| | - Jonathan C Craig
- Sydney School of Public Health, University of Sydney, and Children's Hospital at Westmead, Sydney, Australia
| | - Shilpanjali Jesudason
- Central and Northern Adelaide Renal and Transplantation Service, Royal Adelaide Hospital, Adelaide, Australia
| | | | - Ming-Wei Lin
- Sydney Medical School, University of Sydney, and Westmead Hospital, Sydney, Australia
| | - Sean G O'Neill
- University of New South Wales, Liverpool Hospital, and Ingham Institute for Applied Medical Research, Sydney, Australia
| | - Daniel Sumpton
- Sydney School of Public Health, University of Sydney, Children's Hospital at Westmead, Liverpool Hospital, Sydney, Australia
| | - Allison Tong
- Sydney School of Public Health, University of Sydney, and Children's Hospital at Westmead, Sydney, Australia
| |
Collapse
|
48
|
ten Cate O. Competency-Based Postgraduate Medical Education: Past, Present and Future. GMS JOURNAL FOR MEDICAL EDUCATION 2017; 34:Doc69. [PMID: 29226237 PMCID: PMC5704607 DOI: 10.3205/zma001146] [Citation(s) in RCA: 61] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Subscribe] [Scholar Register] [Received: 11/06/2016] [Revised: 02/06/2017] [Accepted: 02/15/2017] [Indexed: 05/12/2023]
Abstract
Since the turn of the twenty-first century, competency-based medical education (CBME) has become a dominant approach to postgraduate medical education in many countries. CBME has a history dating back half a century and is rooted in general educational approaches such as outcome-based education and mastery learning. Despite controversies around the terminology and the CBME approach, important national medical regulatory bodies in Canada, the United States, and other countries have embraced CBME. CBME can be characterized as having two distinct features: a focus on specific domains of competence, and a relative independence of time in training, making it an individualized approach that is particularly applicable in workplace training. It is not the length of training that determines a person's readiness for unsupervised practice, but the attained competence or competencies. This shift in focus makes CBME different from traditional training. In this contribution, definitions of CBME and related concepts are detailed.
Collapse
Affiliation(s)
- Olle ten Cate
- University Medical Center Utrecht, Center for Research and Development of Education, Utrecht, The Netherlands
| |
Collapse
|
49
|
Elnicki DM, Aiyer MK, Cannarozzi ML, Carbo A, Chelminski PR, Chheda SG, Chudgar SM, Harrell HE, Hood LC, Horn M, Johl K, Kane GC, McNeill DB, Muntz MD, Pereira AG, Stewart E, Tarantino H, Vu TR. An Entrustable Professional Activity (EPA)-Based Framework to Prepare Fourth-Year Medical Students for Internal Medicine Careers. J Gen Intern Med 2017. [PMID: 28634908 PMCID: PMC5653547 DOI: 10.1007/s11606-017-4089-8] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
The purpose of the fourth year of medical school remains controversial. Competing demands during this transitional phase cause confusion for students and educators. In 2014, the Association of American Medical Colleges (AAMC) released 13 Core Entrustable Professional Activities for Entering Residency (CEPAERs). A committee comprising members of the Clerkship Directors in Internal Medicine and the Association of Program Directors in Internal Medicine applied these principles to preparing students for internal medicine residencies. The authors propose a curricular framework based on five CEPAERs that were felt to be most relevant to residency preparation, informed by prior stakeholder surveys. The critical areas outlined include entering orders, forming and answering clinical questions, conducting patient care handovers, collaborating interprofessionally, and recognizing patients requiring urgent care and initiating that care. For each CEPAER, the authors offer suggestions about instruction and assessment of competency. The fourth year of medical school can be rewarding for students, while adequately preparing them to begin residency, by addressing important elements defined in the core entrustable activities. Thus prepared, new residents can function safely and competently in supervised postgraduate settings.
Collapse
Affiliation(s)
- D Michael Elnicki
- University of Pittsburgh School of Medicine, Pittsburgh, PA, USA. .,University of Pittsburgh, Pittsburgh, PA, USA.
| | - Meenakshy K Aiyer
- University of Illinois College of Medicine at Peoria, Peoria, IL, USA
| | | | - Alexander Carbo
- Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Paul R Chelminski
- University of North Carolina School of Medicine, Chapel Hill, NC, USA
| | - Shobhina G Chheda
- University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | | | | | - L Chad Hood
- University of Central Florida College of Medicine, Orlando, FL, USA
| | - Michelle Horn
- University of Mississippi School of Medicine, Jackson, MS, USA
| | - Karnjit Johl
- University of California-Davis School of Medicine, Sacramento, CA, USA
| | - Gregory C Kane
- Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA, USA
| | | | | | - Anne G Pereira
- University of Minnesota Medical School, Minneapolis, MN, USA
| | - Emily Stewart
- Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA, USA
| | | | - T Robert Vu
- Indiana University School of Medicine, Charlotte, NC, USA
| |
Collapse
|
50
|
Taylor DR, Park YS, Egan R, Chan MK, Karpinski J, Touchie C, Snell LS, Tekian A. EQual, a Novel Rubric to Evaluate Entrustable Professional Activities for Quality and Structure. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2017; 92:S110-S117. [PMID: 29065031 DOI: 10.1097/acm.0000000000001908] [Citation(s) in RCA: 59] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
PURPOSE Entrustable professional activities (EPAs) have become a cornerstone of assessment in competency-based medical education (CBME). Increasingly, EPAs are being adopted that do not conform to EPA standards. This study aimed to develop and validate a scoring rubric to evaluate EPAs for alignment with their purpose, and to identify substandard EPAs. METHOD The EQual rubric was developed and revised by a team of education scholars with expertise in EPAs. It was then applied by four residency program directors/CBME leads (PDs) and four nonclinician support staff to 31 stage-specific EPAs developed for internal medicine in the Royal College of Physicians and Surgeons of Canada's Competency by Design framework. Results were analyzed using a generalizability study to evaluate overall reliability, with the EPAs as the object of measurement. Item-level analysis was performed to determine reliability and discrimination value for each item. Scores from the PDs were also compared with decisions about revisions made independently by the education scholars group. RESULTS The EQual rubric demonstrated high reliability in the G-study with a phi-coefficient of 0.84 when applied by the PDs, and moderate reliability when applied by the support staff at 0.67. Item-level analysis identified three items that performed poorly with low item discrimination and low interrater reliability indices. Scores from support staff only moderately correlated with PDs. Using the preestablished cut score, PDs identified 9 of 10 EPAs deemed to require major revision. CONCLUSIONS EQual rubric scores reliably measured alignment of EPAs with literature-described standards. Further, its application accurately identified EPAs requiring major revisions.
Collapse
Affiliation(s)
- David R Taylor
- D.R. Taylor is associate professor, Department of Medicine, Queen's University School of Medicine, Kingston, Ontario, Canada. Y.S. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois. R. Egan is assistant professor and director, Office of Health Science Education, Queen's University School of Medicine, Kingston, Ontario, Canada. M.-K. Chan is associate professor, Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Manitoba, Canada, and clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. J. Karpinski is associate professor, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada, and associate director, Specialties Unit, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. C. Touchie is associate professor, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada, and chief medical education advisor, Medical Council of Canada, Ottawa, Ontario, Canada. L.S. Snell is professor, Department of Medicine, McGill University, Montreal, Quebec, Canada, and senior clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. A. Tekian is professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois
| | | | | | | | | | | | | | | |
Collapse
|