1
|
Monti M, Pittet V, Frick S, Gachoud D. A multi-method approach to drafting candidate entrustable professional activities for a general internal medicine residency programme. Swiss Med Wkly 2024; 154:3592. [PMID: 38801750 DOI: 10.57187/s.3592] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/29/2024] Open
Abstract
INTRODUCTION Entrustable professional activities (EPAs) are units of concrete daily clinical tasks that trainee physicians should be able to handle with increasing autonomy during their postgraduate training. EPAs are gaining international recognition as an essential component of competency-based medical training programmes. The process of developing EPAs for a nationwide training programme is complex and requires an in-depth understanding of EPAs as a concept and good knowledge of appropriate development processes. This article provides a detailed description of the methodology and results of a multi-step approach for developing a list of candidate EPAs for Switzerland's postgraduate training programme in general internal medicine (GIM). METHODS We took a multi-step approach including a systematic review of international literature, four national focus groups, a national consensus process using a RAND appropriateness method, and a quality check of the selected candidate EPAs using EQual criteria. RESULTS These steps generated a final list of 247 candidate EPAs in general internal medicine that were submitted for the national consensus process. After two rounds of rating, experts agreed on the appropriateness for general internal medicine postgraduate training of 225 candidate EPAs. Twenty-two were deemed inappropriate, and disagreement persisted only for two EPAs. DISCUSSION This multi-step programme is one of the few describing in detail the process of developing a list of EPAs and providing evidence of validity at each step. The clinical breadth of our candidate EPAs, together with the detailed description of our methodology, could serve as a useful starting point from which medical education specialists or clinicians could develop or revise applicable lists of EPAs, particularly for postgraduate training programmes in either general internal medicine or family medicine.
Collapse
Affiliation(s)
- Matteo Monti
- Division of Internal Medicine, Lausanne University Hospital, Lausanne, Switzerland
- Medical Education Unit, School of Medicine, Faculty of Biology and Medicine, University of Lausanne, Lausanne, Switzerland
| | - Valérie Pittet
- Department of Epidemiology and Health Systems, Center for Primary Care and Public Health, University of Lausanne, Lausanne, Switzerland
| | - Sonia Frick
- Postgraduate Education Committee, Swiss Society of General Internal Medicine, Bern, Switzerland
- Swiss Institute for Postgraduate and Continuing Education in Medicine, Bern, Switzerland
| | - David Gachoud
- Division of Internal Medicine, Lausanne University Hospital, Lausanne, Switzerland
- Medical Education Unit, School of Medicine, Faculty of Biology and Medicine, University of Lausanne, Lausanne, Switzerland
| |
Collapse
|
2
|
Caretta‐Weyer HA, Sebok‐Syer SS, Morris AM, Schnapp BH, Fant AL, Scott KR, Pirotte M, Gisondi MA, Yarris LM. Better together: A multistakeholder approach to developing specialty-wide entrustable professional activities in emergency medicine. AEM EDUCATION AND TRAINING 2024; 8:e10974. [PMID: 38532740 PMCID: PMC10962124 DOI: 10.1002/aet2.10974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Revised: 02/28/2024] [Accepted: 03/04/2024] [Indexed: 03/28/2024]
Abstract
Purpose Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care. Methods In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback. Results Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally. Conclusions To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs.
Collapse
Affiliation(s)
- Holly A. Caretta‐Weyer
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Stefanie S. Sebok‐Syer
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Amanda M. Morris
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Benjamin H. Schnapp
- Berbee Walsh Department of Emergency MedicineUniversity of WisconsinMadisonWisconsinUSA
| | - Abra L. Fant
- Department of Emergency Medicine, Northwestern University Feinberg School of MedicineMcGaw Medical Center at Northwestern UniversityChicagoIllinoisUSA
| | - Kevin R. Scott
- Department of Emergency MedicineUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaPennsylvaniaUSA
| | - Matthew Pirotte
- Department of Emergency MedicineVanderbilt University Medical CenterNashvilleTennesseeUSA
| | - Michael A. Gisondi
- Department of Emergency MedicineStanford University School of MedicinePalo AltoCaliforniaUSA
| | - Lalena M. Yarris
- Department of Emergency MedicineOregon Health & Science UniversityPortlandOregonUSA
| |
Collapse
|
3
|
Amare EM, Siyoum MT, Abubeker FA, Tufa TH, Hailemeskel AT. Designing the Future of Medical Education: The EPA Framework as a Catalyst to Inform Family Planning and Reproductive Health Fellowship Training Program in Ethiopia Medical Education: An Exploratory Sequential Mixed Method Study. ADVANCES IN MEDICAL EDUCATION AND PRACTICE 2024; 15:189-200. [PMID: 38505496 PMCID: PMC10949272 DOI: 10.2147/amep.s438315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 09/29/2023] [Accepted: 03/02/2024] [Indexed: 03/21/2024]
Abstract
Introduction Entrustable Professional Activities (EPAs) are tasks or responsibilities within a specific field that can be given to a learner once they are competent to perform them independently. EPAs are being used in various specialty programs and serving as valuable tool to inform educational program. However, due to disparities in professional practice between different contexts, the automatic transfer of a set of core EPAs is not feasible. Hence, our study aims to develop an EPA framework to inform the Family Planning and Reproductive Health Fellowship Program in the local context of Ethiopia. Methods We employed an exploratory mixed-method design, which involved the collection of qualitative data using the Nominal Group Technique and quantitative data through a nationwide survey in all residency training institutions across the country. Qualitative data analysis involved several steps, including compiling a list of tasks, removing duplicate tasks, reviewing EPAs using criteria and an equal rubric tool. For quantitative data analysis, descriptive statistics, validity index analysis, and intra-class correlation coefficients, were used. Results Seven senior panelists were able to propose a total of 57 EPAs, with 17 remaining after qualitative data analysis. The panelist evaluated the relevance of each EPA in the second phase. As a result, 17 EPAs received a content validity index of >0.83, indicating satisfactory relevance. In the national survey, experts reached a high level of final agreement regarding the relevance and representativeness of all 17 EPAs (ICC = 0.815, 95% CI [0.0.756,0.865], p.0001). Conclusion The final set of 17 end-of-training EPAs is valid, acceptable and representative of the discipline, and they can be used as a framework to inform Family planning and Reproductive Health Fellowship Program in Ethiopian medical education once these core EPA statements are described in sufficient detail. This can contribute to raise the quality of training and hence the quality of patient care.
Collapse
Affiliation(s)
| | - Mekdim Tadesse Siyoum
- Department of Surgery, St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia
| | - Ferid Abbas Abubeker
- Department of Obstetrics & Gynecology, St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia
| | - Tesfaye Hurissa Tufa
- Department of Obstetrics & Gynecology, St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia
| | - Azeb Tamrat Hailemeskel
- Educational Development Center, St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia
| |
Collapse
|
4
|
Yan F, Yang X, Zhang L, Cheng H, Bai L, Yang F. Establishing entrustable professional activities for psychiatry residents in China. BMC MEDICAL EDUCATION 2023; 23:623. [PMID: 37658351 PMCID: PMC10474625 DOI: 10.1186/s12909-023-04583-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 08/11/2023] [Indexed: 09/03/2023]
Abstract
PURPOSE The authors established entrustable professional activities for psychiatry residents in China. METHODS The authors conducted a literature research and two expert consultation rounds following the Delphi method in 2022 to screen and optimize entrustable professional activities for psychiatry residents. RESULTS The effective questionnaire recovery rate in the two consultation rounds was 100% (44/44). The expert authority coefficients of the first and second consultation rounds were 0.861 and 0.881, respectively. The Kendall harmony coefficients of the first and second expert consultation rounds were 0.279 (χ2 = 405.43, P < .001) and 0.389 (χ2 = 3456.83, P < .001), respectively. The arithmetic means of the various indicators' evaluation results in the two consultation rounds ranged between 3.61 and 4.93, and the full score rates were between 13.6% and 93.2%. The authors established 17 entrustable professional activities for psychiatry residents and their contents with phase-based modularization and formulated the entrustable level of each at various stages. CONCLUSIONS Combined with standardized psychiatry training characteristics, the authors preliminarily established phase-specific and modular entrustable professional activities for psychiatry residents. The formulated entrustable professional activities are suitable for the practice and clinical environment of standardized psychiatry training in China. The devised system has good observability and measurability and provides a simple and feasible competency evaluation method for standardized psychiatry resident training.
Collapse
Affiliation(s)
- Feng Yan
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Xu Yang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Ligang Zhang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Huaqin Cheng
- Institute of Medical Education &National center for Health Professions Education Development, PeKing University, Beijing, 100083, China
| | - Luyuan Bai
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China
| | - Fude Yang
- Beijing Huilongguan Hospital, Peking University Huilongguan Clinical Medical School, Beijing, China.
| |
Collapse
|
5
|
Ganzhorn A, Schulte-Uentrop L, Küllmei J, Zöllner C, Moll-Khosrawi P. National consensus on entrustable professional activities for competency-based training in anaesthesiology. PLoS One 2023; 18:e0288197. [PMID: 37432949 DOI: 10.1371/journal.pone.0288197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 06/21/2023] [Indexed: 07/13/2023] Open
Abstract
Entrustable Professional Activities (EPA) are specialty specific tasks or responsibilities, combining the clinical workplace and the long-demanded competency-based medical education. The first step to transform time-based into EPA-based training is to reach consensus on core EPAs that describe sufficiently the workplace. We aimed to present a nationally validated EPA-based curriculum for postgraduate training in anaesthesiology. Using a predefined and validated list of EPAs, we applied a Delphi consensus approach, involving all German chair directors of anaesthesiology. We then conducted a subsequent qualitative analysis. Thirty-four chair directors participated in the Delphi survey (77% response) and twenty-five completed all the questions (56% overall response). Reflected by the intra-class-correlation, the consensus on the importance (ICC: 0.781, 95% CI [0.671, 0.868]) and the year of entrustment (ICC: 0.973, 95% CI [0.959, 0.984]) of each EPA reached high levels of agreement among the chair directors. The comparison of data assessed in the preceding validation and present study showed excellent and good levels of agreement (ICC entrustment: 0.955, 95% CI [0.902, 0.978]; ICC importance: 0.671, 95% CI [-0.204, 0.888]). The adaptation process, based on the qualitative analysis, resulted in a final set of 34 EPAs. We present an elaborate, fully described and nationally validated EPA-based curriculum, reflecting a broad consensus among different stakeholders of anaesthesiology. We hereby provide a further step towards competency-based postgraduate anaesthesiology training.
Collapse
Affiliation(s)
- Alexander Ganzhorn
- Department of Anaesthesiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Leonie Schulte-Uentrop
- Department of Anaesthesiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Josephine Küllmei
- Department of Anaesthesiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Christian Zöllner
- Department of Anaesthesiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| | - Parisa Moll-Khosrawi
- Department of Anaesthesiology, University Medical Centre Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
6
|
Pinilla S, Lerch S, Nendaz M, Huwendiek S, Klöppel S. [Graduate medical education in old age psychiatry: a scoping literature review]. FORTSCHRITTE DER NEUROLOGIE-PSYCHIATRIE 2023. [PMID: 37327816 DOI: 10.1055/a-2053-8274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
BACKGROUND Gerontopsychiatry will become increasingly relevant as a discipline for primary care based on current demographic forecasts, destigmatization of mental illness and specific diagnostic and therapeutic developments. Hence, high quality graduate medical training in old age psychiatry is needed. Objectives The goal of this review was to summarize the literature on medical education relevant to residency training in old age psychiatry and to contrast the findings with international developments in competency-based medical education. MATERIALS AND METHODS The authors used the scoping review method of Arksey and O'Malley. RESULTS The initial search yielded 913 hits. After full text screening, 20 original articles were used for data extraction. The study content was summarized under three categories: trainee recruitment, length and structure of graduate training curricula in old age psychiatry, and learning goals and competencies in old age psychiatry training. Surveys and expert consensus were mostly used as study methods. High-quality clinical training experience with gerontopsychiatric patients and supervision of residents was an important factor for stimulating interest in old age psychiatry. Few studies provided evidence for educational benefit of digital learning and teaching formats and simulation training in old age psychiatry. Overall, there were no studies in old age psychiatry with explicit reference to concepts of competency-based graduate medical education. CONCLUSIONS Clinical rotations and mentoring foster interest of clinical residents in the discipline of old age psychiatry. Systematically introducing clinical rotations in old age psychiatry in general psychiatry residency programs seems necessary to enable residents to gain relevant knowledge and skills. Educational research in old age psychiatry considering patient outcomes seems to be a meaningful next step.
Collapse
Affiliation(s)
- Severin Pinilla
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern, Bern, Switzerland
- Abteilung für Assessment und Evaluation, Institut für Medizinische Lehre (IML), Universität Bern, Bern, Switzerland
| | - Seraina Lerch
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern, Bern, Switzerland
- Abteilung für Assessment und Evaluation, Institut für Medizinische Lehre (IML), Universität Bern, Bern, Switzerland
- Institut für Medizinische Psychologie Universitätsklinikum Heidelberg, Heidelberg, Germany
| | - Mathieu Nendaz
- Universitätsspital Genf (HUG), Universität Genf , Genf, Switzerland
| | - Sören Huwendiek
- Abteilung für Assessment und Evaluation, Institut für Medizinische Lehre (IML), Universität Bern, Bern, Switzerland
| | - Stefan Klöppel
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern, Bern, Switzerland
| |
Collapse
|
7
|
Stephan A, Cheung G, van der Vleuten C. Entrustable Professional Activities and Learning: The Postgraduate Trainee Perspective. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2023; 47:134-142. [PMID: 36224504 PMCID: PMC10060374 DOI: 10.1007/s40596-022-01712-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Accepted: 09/16/2022] [Indexed: 05/25/2023]
Abstract
OBJECTIVE Entrustable professional activities (EPAs) are used as clinical activities in postgraduate psychiatry training in Australasia. This study aimed to explore psychiatry trainees' perceptions of the impact of EPAs on their motivation and learning. METHODS A constructivist grounded theory approach was used to conceptualize the impact of EPAs on trainees' motivation and learning. A purposive sample of trainees was recruited from across New Zealand. Semi-structured individual interviews were used for data collection and continued until theoretical saturation was reached. RESULTS The impact of EPAs on learning was mediated by the trainee's appraisals of subjective control, value, and the costs of engaging with EPAs. When appraisals were positive, EPAs encouraged a focus on particular learning needs and structured learning with the supervisor. However, when appraisals were negative, EPAs encouraged a superficial approach to learning. Trainee appraisals and their subsequent impact on motivation and learning were most affected by EPA granularity, alignment of EPAs with clinical practice, and the supervisor's conscientiousness in their approach to EPAs. CONCLUSIONS To stimulate learning, EPAs must be valued by both trainees and supervisors as constituting a coherent work-based curriculum that encompasses the key fellowship competencies. If EPAs are to be effective as clinical tasks for learning, ongoing faculty development must be the leading priority.
Collapse
Affiliation(s)
- Alice Stephan
- Mental Health and Addictions Service, Waikato District Health Board, Hamilton, New Zealand
| | - Gary Cheung
- School of Medicine, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand.
| | - Cees van der Vleuten
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, University of Maastricht, Maastricht, Netherlands
| |
Collapse
|
8
|
Amare EM, Siebeck M, Yigzaw T, Fischer MR, Tadesse M, Berndt M. Differences in perceptions of capability, autonomy, and expectations between residents and surgical team members in executing EPAs in Ethiopian medical education. Heliyon 2023; 9:e14316. [PMID: 36942250 PMCID: PMC10023974 DOI: 10.1016/j.heliyon.2023.e14316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 02/24/2023] [Accepted: 03/01/2023] [Indexed: 03/11/2023] Open
Abstract
Background Entrustable Professional Activities (EPAs) are units of professional practice that are defined as tasks or responsibilities that are entrusted to an unsupervised execution by a trainee. In 2021, a framework of 29 EPAs was developed for surgical residency training programs in Ethiopia, with the goal of residents being able to perform independently by the time they graduate. However, studies show that surgical residents lack confidence and are unable to execute EPAs autonomously upon graduation, and concerns have been raised about graduate competencies in EPA execution. The goal of this research is to assess how surgical team members judge/perceive residents' performance in executing these EPAs autonomously at the time of graduation and how residents rate their own capability and autonomy in executing EPAs in order to systematically introduce and implement EPAs in Ethiopian medical education. Methods A survey was conducted in the Departments of Surgery at four residency training institutions in Ethiopia. All eligible surgical team members and final-year general surgery residents were invited to participate. Surgical team members were asked to rate the observed performance of a group of graduating surgical residents in each of the 29 EPAs, and residents were asked to rate their own capability in executing EPAs. The analysis focused on variations in performance ratings between surgical team members and residents, as well as across surgical team members. Results A total of 125 surgical team members and 49 residents participated in this study. Residents rate their competence in performing these EPAs higher than surgical team members, mean 4.2 (SD = 0.63) vs. 3.7 (SD = 0.9). A statistically significant difference in perceptions of capability, autonomy, and expectations in executing EPAs was observed between the two groups of study (p = 0.03, CI: 0.51-0.95), as well as within surgical team members (p < 0.001). Conclusions Differences in perceptions of capability, autonomy, and expectations between residents and surgical team members, as well as within faculty members, were seen in executing EPAs. There were concerns about graduate surgical residents' competence to execute EPAs autonomously at the time of graduation. Surgical team members perceived that a set of graduating surgical residents are not yet safe to perform these EPAs independently (without supervision) and still requires distant supervision.
Collapse
Affiliation(s)
- Equlinet Misganaw Amare
- CIH LMU, Center for International Health, University Hospital, LMU Munich, Germany
- Corresponding author. Wollo Sefer (near Mina Building), P. O. Box 2881, Code, 1250, Addis Ababa, Ethiopia.
| | - Matthias Siebeck
- Institute of Medical Education, University Hospital, LMU Munich, Munich, Germany
| | - Tegbar Yigzaw
- Jhpiego – Ethiopia (an Affiliate of Johns Hopkins University), Addis Ababa, Ethiopia
| | - Martin R. Fischer
- Institute of Medical Education, University Hospital, LMU Munich, Munich, Germany
| | - Mekdim Tadesse
- St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia
| | - Markus Berndt
- Institute of Medical Education, University Hospital, LMU Munich, Munich, Germany
| |
Collapse
|
9
|
Hennus MP, van Dam M, Gauthier S, Taylor DR, ten Cate O. The logic behind entrustable professional activity frameworks: A scoping review of the literature. MEDICAL EDUCATION 2022; 56:881-891. [PMID: 35388517 PMCID: PMC9542438 DOI: 10.1111/medu.14806] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 03/21/2022] [Accepted: 03/29/2022] [Indexed: 05/31/2023]
Abstract
INTRODUCTION Entrustable professional activities (EPAs), discrete profession-specific tasks requiring integration of multiple competencies, are increasingly used to help define and inform curricula of specialty training programmes. Although guidelines exist to help guide the developmental process, deciding what logic to use to draft a preliminary EPA framework poses a crucial but often difficult first step. The logic of an EPA framework can be defined as the perspective used by its developers to break down the practice of a profession into units of professional work. This study aimed to map dominant logics and their rationales across postgraduate medical education and fellowship programmes. METHODS A scoping review using systematic searches within five electronic databases (Medline, Embase, Google Scholar, Scopus and Web of Science) was performed. Dominant logics of included papers were identified using inductive coding and iterative analysis. RESULTS In total, 42 studies were included. Most studies were conducted in the United States (n = 22; 52%), Canada (n = 6; 14%) and the Netherlands (n = 4; 10%). Across the reported range of specialties, family medicine (n = 4; 10%), internal medicine (n = 4; 10%), paediatrics (n = 3; 7%) and psychiatry (n = 3; 7%) were the most common. Three dominant logics could be identified, namely, 'service provision', 'procedures' and/or 'disease or patient categories'. The majority of papers (n = 37; 88%) used two or more logics when developing EPA frameworks (median = 3, range = 1-4). Disease or patient groups and service provision were the most common logics used (39% and 37%, respectively). CONCLUSIONS Most programmes used a combination of logics when trying to capture the essential tasks of a profession in EPAs. For each of the three dominant logics, the authors arrived at a definition and identified benefits, limitations and examples. These findings may potentially inform best practice guidelines for EPA development.
Collapse
Affiliation(s)
- Marije P. Hennus
- Department of Pediatric Intensive CareUniversity Medical Center UtrechtUtrechtThe Netherlands
| | - Marjel van Dam
- Intensive Care CenterUniversity Medical Center UtrechtUtrechtThe Netherlands
| | - Stephen Gauthier
- Department of MedicineQueen's University School of MedicineKingstonOntarioCanada
| | - David R. Taylor
- Department of MedicineQueen's University School of MedicineKingstonOntarioCanada
| | - Olle ten Cate
- Center for Research and Development of EducationUniversity Medical Center UtrechtUtrechtThe Netherlands
| |
Collapse
|
10
|
Lockman K, Lowry MF, DiScala S, Lovell AG, Uritsky TJ, Kematick BS, Schmidt M, Wetshtein AM, Scullion B, Herndon CM, Atayee RS. Development of Entrustable Professional Activities for Specialist Hospice and Palliative Care Pharmacists. J Pain Symptom Manage 2022; 64:37-48. [PMID: 35304228 DOI: 10.1016/j.jpainsymman.2022.03.004] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Revised: 03/03/2022] [Accepted: 03/08/2022] [Indexed: 12/30/2022]
Abstract
CONTEXT Entrustable professional activities (EPAs) translate competencies into explicit, practical terms that clearly state the expected roles and responsibilities of clinicians who have achieved proficiency and expertise in a field. EPAs are defined for Hospice and Palliative Medicine physicians but not for other members of Hospice and Palliative Care (HAPC) interprofessional teams, including pharmacists. OBJECTIVES The objective of this study was to develop EPAs for HAPC pharmacists. METHODS An 11-member workgroup of HAPC pharmacists was convened to develop candidate EPAs using nominal group and modified-Delphi methods. Content validity index was used as a measure of consensus, defined a priori at ≥ 60%. Vetting occurred via intra- and interprofessional stakeholder reactor groups and a national survey of HAPC pharmacists. RESULTS Following an iterative process of workgroup and stakeholder consensus-building, 15 HAPC pharmacist EPAs were developed. Among the workgroup, all 15 EPAs reached ≥ 70% consensus, indicating appropriate internal validity. In a national survey of 185 HAPC pharmacists with a 20% response rate, 13 EPAs were rated by most respondents as "essential" and 2 were rated by most respondents as "important but not essential." Respondents indicated the 15 EPA set represented the core professional activities of HAPC pharmacists well (median rating of 5 on a Likert-like scale, IQR 1). CONCLUSION Fifteen consensus EPAs describe essential activities of HAPC pharmacists in direct patient care, leadership, education, and scholarship. These EPAs will further guide pharmacist training programs, HAPC services seeking to incorporate a specialized pharmacist on the team, and currently practicing HAPC pharmacists.
Collapse
Affiliation(s)
- Kashelle Lockman
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA.
| | - Maria F Lowry
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Sandra DiScala
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Amanda G Lovell
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Tanya J Uritsky
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Benjamin S Kematick
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Michelle Schmidt
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Andrea M Wetshtein
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Bridget Scullion
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Christopher M Herndon
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| | - Rabia S Atayee
- University of Iowa College of Pharmacy (K.L.), Iowa City, Iowa, USA; University of Pittsburgh School of Pharmacy (M.F.L.), Pittsburgh, Pennsylvania, USA; West Palm Beach (WPB) Veterans Affairs Medical Center (S.D.), West Palm Beach, Florida, USA; Optum Hospice Pharmacy Services (A.G.L.), Westerville, Ohio, USA; Department of Pharmacy (T.J.U.), Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Dana-Farber Cancer Institute (B.S.K.), Boston, Massachusetts, USA; University of Iowa College of Pharmacy (M.S.), Iowa City, Iowa, USA; Cleveland Clinic Fairview Hospital (A.M.W.), Cleveland, Ohio, USA; Dana-Farber Cancer Institute (B.S.), Boston, Massachusetts, USA; School of Pharmacy (C.M.H.), Southern Illinois University Edwardsville, Edwardsville, Illinois, USA; San Diego Skaggs School of Pharmacy and Pharmaceutical Sciences (R.S.A.), University of California, San Diego, California, USA; San Diego Health, Palliative Care Program (R.S.A.), University of California, San Diego, California, USA
| |
Collapse
|
11
|
Burkhart CS, Dell-Kuster S, Touchie C. Who can do this procedure? Using entrustable professional activities to determine curriculum and entrustment in anesthesiology - An international survey. MEDICAL TEACHER 2022; 44:672-678. [PMID: 35021934 DOI: 10.1080/0142159x.2021.2020231] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
INTRODUCTION As competency-based curricula get increasing attention in postgraduate medical education, Entrustable Professional Activities (EPAs) are gaining in popularity. The aim of this survey was to determine the use of EPAs in anesthesiology training programs across Europe and North America. METHODS A survey was developed and distributed to anesthesiology residency training program directors in Switzerland, Germany, Austria, Netherlands, USA and Canada. A convergent design mixed-methods approach was used to analyze both quantitative and qualitative data. RESULTS The survey response rate was 38% (108 of 284). Seven percent of respondents used EPAs for making entrustment decisions. Fifty-three percent of institutions have not implemented any specific system to make such decisions. The majority of respondents agree that EPAs should become an integral part of the training of residents in anesthesiology as they are universal and easy to use. CONCLUSION Although recommended by several national societies, EPAs are used in few anesthesiology training programs. Over half of responding programs have no specific system for making entrustment decisions. Although several countries are adopting or planning to adopt EPAs and national societies are recommending the use of EPAs as a framework in their competency-based programs, few are yet using these to make "competence" decisions.
Collapse
Affiliation(s)
| | - Salome Dell-Kuster
- Department of Anesthesiology, University Hospital Basel, Basel, Switzerland
- Institute for Clinical Epidemiology and Biostatistics, University of Basel, Basel, Switzerland
| | - Claire Touchie
- Department of Medicine, University of Ottawa, Ottawa, Canada
| |
Collapse
|
12
|
Fefergrad M, Mulsant BH. Psychotherapy Training in a Competency-Based Medical Education Psychiatry Residency: A Proposal for a Practical and Socially Responsible Model. CANADIAN JOURNAL OF PSYCHIATRY. REVUE CANADIENNE DE PSYCHIATRIE 2022; 67:423-427. [PMID: 34935519 PMCID: PMC9149527 DOI: 10.1177/07067437211063096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Mark Fefergrad
- Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
- Sunnybrook Health Science Centre, Toronto, Ontario, Canada
| | - Benoit H. Mulsant
- Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
- Centre for Addiction and Mental Health, Toronto, Ontario, Canada
| |
Collapse
|
13
|
Yun Z, Jing L, Junfei C, Wenjing Z, Jinxiang W, Tong Y, Aijun Z. Entrustable Professional Activities for Chinese Standardized Residency Training in Pediatric Intensive Care Medicine. Front Pediatr 2022; 10:919481. [PMID: 35859946 PMCID: PMC9289143 DOI: 10.3389/fped.2022.919481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Accepted: 06/10/2022] [Indexed: 11/15/2022] Open
Abstract
BACKGROUND Entrustable professional activities (EPAs) were first introduced by Olle ten Cate in 2005. Since then, hundreds of applications in medical research have been reported worldwide. However, few studies discuss the use of EPAs for residency training in pediatric intensive care medicine. We conducted a pilot study of EPA for pediatric intensive care medicine to evaluate the use of EPAs in this subspecialty. MATERIALS AND METHODS A cross-sectional study was implemented in pediatric intensive care medicine standardized residency training at the Qilu Hospital of Shandong University. An electronic survey assessing EPA performance using eight scales composed of 15 categories were distributed among residents and directors. RESULTS A total of 217 director-assessment and 44 residents' self-assessment questionnaires were collected, both demonstrating a rising trend in scores across postgraduate years. There were significant differences in PGY1-vs.-PGY2 and PGY1-vs.-PGY3 director-assessment scores, while there were no differences in PGY2-vs.-PGY3 scores. PGY had a significant effect on the score of each EPA, while position significantly affected the scores of all EPAs except for EPA1 (Admit a patient) and EPA2 (Select and interpret auxiliary examinations). Gender only significantly affected the scores of EPA6 (Report a case), EPA12 (Perform health education), and EPA13 (Inform bad news). CONCLUSION This study indicates that EPA assessments have a certain discriminating capability among different PGYs in Chinese standardized residency training in pediatric intensive care medicine. Postgraduate year, gender, and resident position affected EPA scores to a certain extent. Given the inconsistency between resident-assessed and director-assessed scores, an improved feedback program is needed in the future.
Collapse
Affiliation(s)
- Zhang Yun
- Department of Pediatrics, Qilu Hospital of Shandong University, Jinan, China
| | - Liu Jing
- Department of Biostatistics, School of Public Health, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Chen Junfei
- Department of Pediatric Surgery, Qilu Hospital of Shandong University, Jinan, China
| | - Zhang Wenjing
- Department of Obstetrics and Gynecology, Qilu Hospital of Shandong University, Jinan, China
| | - Wu Jinxiang
- Department of Pulmonary and Critical Care Medicine, Qilu Hospital of Shandong University, Jinan, China
| | - Yue Tong
- Medical Training Office, Qilu Hospital of Shandong University, Jinan, China
| | - Zhang Aijun
- Department of Pediatrics, Qilu Hospital of Shandong University, Jinan, China
| |
Collapse
|
14
|
Young JQ, Thakker K, John M, Friedman K, Sugarman R, van Merriënboer JJG, Sewell JL, O'Sullivan PS. Exploring the relationship between emotion and cognitive load types during patient handovers. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2021; 26:1463-1489. [PMID: 34037906 DOI: 10.1007/s10459-021-10053-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Accepted: 05/05/2021] [Indexed: 06/12/2023]
Abstract
Cognitive Load Theory has emerged as an important approach to improving instruction in the health professions workplace, including patient handovers. At the same time, there is growing recognition that emotion influences learning through numerous cognitive processes including motivation, attention, working memory, and long-term memory. This study explores how emotion influences the cognitive load experienced by trainees performing patient handovers. From January to March 2019, 693 (38.7%) of 1807 residents and fellows from a 24-hospital health system in New York city completed a survey after performing a handover. Participants rated their emotional state and cognitive load. The survey included questions about features of the learner, task, and instructional environment. The authors used factor analysis to identify the core dimensions of emotion. Regression analyses explored the relationship between the emotion factors and cognitive load types. Two emotion dimensions were identified representing invigoration and tranquility. In regression analyses, higher levels of invigoration, tranquility, and their interaction were independently associated with lower intrinsic load and extraneous load. The interaction of invigoration and tranquility predicted lower germane load. The addition of the emotion variables to multivariate models including other predictors of cognitive load types significantly increased the amount of variance explained. The study provides a model for measuring emotions in workplace learning. Because emotion appears to have a significant influence on cognitive load types, instructional designers should consider strategies that help trainees regulate emotion in order to reduce cognitive load and improve learning and performance.
Collapse
Affiliation(s)
- John Q Young
- Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | - Krima Thakker
- Division of Education and Training, Zucker Hillside Hospital at Northwell Health, 75-59 263rd Street, Glen Oaks, NY, 10543, USA
| | - Majnu John
- Division of Research, Zucker Hillside Hospital at Northwell Health, Glen Oaks, NY, USA
| | - Karen Friedman
- Department of Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwel, Hempstead, NY, USA
| | | | - Jeroen J G van Merriënboer
- School of Health Professions Education, Faculty of Health, Medicine, and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Justin L Sewell
- Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| | - Patricia S O'Sullivan
- Department of Medicine & Office of Research and Development in Medical Education, University of California at San Francisco School of Medicine, San Francisco, CA, USA
| |
Collapse
|
15
|
Lindeman B, Brasel K, Minter RM, Buyske J, Grambau M, Sarosi G. A Phased Approach: The General Surgery Experience Adopting Entrustable Professional Activities in the United States. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:S9-S13. [PMID: 34183596 DOI: 10.1097/acm.0000000000004107] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Entrustable professional activities (EPAs) have been increasingly used as an assessment framework to formally capture the myriad ad hoc entrustment decisions that occur on a daily basis in clinical settings with learners present. Following the definition of Core EPAs for Entering Residency by the Association of American Medical Colleges in 2014, the American Board of Surgery (ABS) began to explore the utility of EPAs as a framework to support competency-based resident education within general surgery in 2016. As the complement of EPAs drafted for a specialty serve to define the core tasks of a professional within that discipline, initial efforts to define the entire scope of general surgery were fraught with difficulty as no commonly accepted definition of a general surgeon currently exists. Opting to prioritize a pilot of the EPA conceptual framework within surgical training rather than defining the entirety of the specialty, ABS leaders identified 5 EPAs that represent a core of general surgery with which to begin. This article details the process of choosing the initial set of EPAs and provides a roadmap for other disciplines interested in testing the feasibility of this assessment framework while garnering buy-in among the community of educators. Future steps, including implementation of the existing 5 EPAs beyond the initial pilot sites and drafting and implementation of the additional complement of EPAs, are also described.
Collapse
Affiliation(s)
- Brenessa Lindeman
- B. Lindeman is assistant professor, Department of Surgery, University of Alabama at Birmingham, Birmingham, Alabama
| | - Karen Brasel
- K. Brasel is professor, Department of Surgery, Oregon Health and Science University, Portland, Oregon
| | - Rebecca M Minter
- R.M. Minter is professor, Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Jo Buyske
- J. Buyske is president and chief executive officer, American Board of Surgery, Philadelphia, Pennsylvania
| | - Marni Grambau
- M. Grambau is director of examination development, American Board of Surgery, Philadelphia, Pennsylvania
| | - George Sarosi
- G. Sarosi is professor, Department of Surgery, University of Florida, Gainesville, Florida
| |
Collapse
|
16
|
Young JQ, Frank JR, Holmboe ES. Advancing Workplace-Based Assessment in Psychiatric Education: Key Design and Implementation Issues. Psychiatr Clin North Am 2021; 44:317-332. [PMID: 34049652 DOI: 10.1016/j.psc.2021.03.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
With the adoption of competency-based medical education, assessment has shifted from traditional classroom domains of knows and knows how to the workplace domain of doing. This workplace-based assessment has 2 purposes; assessment of learning (summative feedback) and the assessment for learning (formative feedback). What the trainee does becomes the basis for identifying growth edges and determining readiness for advancement and ultimately independent practice. High-quality workplace-based assessment programs require thoughtful choices about the framework of assessment, the tools themselves, the platforms used, and the contexts in which the assessments take place, with an emphasis on direct observation.
Collapse
Affiliation(s)
- John Q Young
- Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and, Zucker Hillside Hospital at Northwell Health, 75-59 263rd Street, Kaufman Building, Glen Oaks, NY 11004, USA.
| | - Jason R Frank
- Department of Emergency Medicine, University of Ottawa, Royal College of Physicians and Surgeons of Canada, 774 Echo Drive, Ottawa, Ontario K15 5NB, Canada
| | - Eric S Holmboe
- Accreditation Council for Graduate Medical Education, ACGME, 401 North Michigan Avenue, Chicago, IL 60611, USA
| |
Collapse
|
17
|
Hung EK, Jibson M, Sadhu J, Stewart C, Walker A, Wichser L, Young JQ. Wresting with Implementation: a Step-By-Step Guide to Implementing Entrustable Professional Activities (EPAs) in Psychiatry Residency Programs. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2021; 45:210-216. [PMID: 33078330 DOI: 10.1007/s40596-020-01341-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2020] [Accepted: 10/05/2020] [Indexed: 06/11/2023]
Affiliation(s)
- Erick K Hung
- University of California, San Francisco, School of Medicine, San Francisco, CA, USA.
| | - Michael Jibson
- University of Michigan School of Medicine, Ann Arbor, MI, USA
| | - Julie Sadhu
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Colin Stewart
- Georgetown University Medical Center and School of Medicine, Washington, DC, USA
| | - Ashley Walker
- University of Oklahoma School of Medicine, Oklahoma City, OK, USA
| | - Lora Wichser
- University of Minnesota Medical School, Minneapolis, MN, USA
| | - John Q Young
- Donald and Barbara Zucker School of Medicine and the Zucker Hillside Hospital, Hempstead, NY, USA
| |
Collapse
|
18
|
Development of entrustable professional activities for paediatric intensive care fellows: A national modified Delphi study. PLoS One 2021; 16:e0248565. [PMID: 33735195 PMCID: PMC7971696 DOI: 10.1371/journal.pone.0248565] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 03/01/2021] [Indexed: 11/24/2022] Open
Abstract
Entrustable professional activities (EPAs), as a focus of learner assessment, are supported by validity evidence. An EPA is a unit of professional practice requiring proficiency in multiple competencies simultaneously, that can be entrusted to a sufficiently competent learner. Taken collectively, a set of EPAs define and inform the curriculum of a specialty training. The goal of this study was to develop a set of EPAs for Dutch PICU fellows. A multistage methodology was employed incorporating sequential input from task force members, a medical education expert, PICU fellowship program directors, and PICU physicians and fellows via a modified three-round Delphi study. In the first modified Delphi round, experts rated indispensability and clarity of preliminary EPAs. In the subsequent rounds, aggregated scores for each EPA and group comments were provided. In round two, respondents rated indispensability and clarity of revised EPAs. Round three was used to gain explicit confirmation of suitability to implement these EPAs. Based on median ratings and content validity index (CVI) analysis for indispensability in the first two rounds, all nine preliminary EPAs covered activities that were deemed essential to the clinical practice of PICU physicians. Based on median ratings and CVI analysis for clarity however, four EPAs needed revision. With an agreement percentage of 93–100% for all individual EPAs as well as the set as a whole, a high degree of consensus among experts was reached in the third round. The resulting nine PICU EPAs provide a succinct overview of the core tasks of Dutch PICU physicians. These EPAs were created as an essential first step towards developing an assessment system for PICU fellows, grounded in core professional activities. The robust methodology used, may have broad applicability for other (sub)specialty training programs aiming to develop specialty specific EPAs.
Collapse
|
19
|
Young JQ, Thakker K, John M, Friedman K, Sugarman R, Sewell JL, O'Sullivan PS. Features of the learner, task, and instructional environment that predict cognitive load types during patient handoffs: Implications for instruction. APPLIED COGNITIVE PSYCHOLOGY 2021. [DOI: 10.1002/acp.3803] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- John Q. Young
- Department of Psychiatry Donald and Barbara Zucker School of Medicine at Hofstra/Northwell Hempstead New York USA
- Department of Psychiatry Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Krima Thakker
- Department of Psychiatry Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Majnu John
- Division of Research Zucker Hillside Hospital at Northwell Health Glen Oaks New York USA
| | - Karen Friedman
- Department of Medicine Donald and Barbara Zucker School of Medicine at Hofstra/Northwell Hempstead New York USA
| | - Rebekah Sugarman
- School of Medicine University of Michigan Ann Arbor Michigan USA
| | - Justin L. Sewell
- Department of Medicine School of Medicine, University of California, San Francisco San Francisco California USA
| | - Patricia S. O'Sullivan
- Department of Medicine School of Medicine, University of California, San Francisco San Francisco California USA
| |
Collapse
|
20
|
Pinilla S, Matthes O, Gehret A, Huwendiek S, Lenouvel E, Lindert S, Marty AP, Nick B, Nissen C, Schneeberger A, Stocker L, Wallies M, Weidt S, Weiss-Breckwoldt A, Klöppel S. [Entrustable Professional Activities in Graduate Medical Education in Psychiatry: A Promising Concept]. PRAXIS 2021; 110:30-36. [PMID: 33406929 DOI: 10.1024/1661-8157/a003601] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Entrustable Professional Activities in Graduate Medical Education in Psychiatry: A Promising Concept Abstract. Entrustable Professional Activities (EPAs) are competency-based learning goals derived from observable clinical activities. In undergraduate medical education, they have now been adopted throughout Switzerland as part of the so-called PROFILES catalog (Principal Relevant Objectives and Framework for Integrated Learning and Education in Switzerland). The nine core EPAs to be mastered in undergraduate medical education can serve as a basis for introducing EPAs in graduate medical education as well. We will discuss this approach in the context of graduate medical education in psychiatry and psychotherapy from the perspective of different training contexts and a pilot example. In this position paper, we describe a promising opportunity to improve graduate medical training through the implementation of EPAs, both in terms of the quality of training and thus of patient care, as well as in terms of the attractiveness of the specialty for future residents.
Collapse
Affiliation(s)
- Severin Pinilla
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern
- Institut für Medizinische Lehre (IML), Abteilung für Assessment und Evaluation, Universität Bern
- Geteilte Erstautorenschaft
| | - Oliver Matthes
- Klinik für Konsiliarpsychiatrie und Psychosomatik, Universitätsspital Zürich
- Geteilte Erstautorenschaft
| | | | - Sören Huwendiek
- Institut für Medizinische Lehre (IML), Abteilung für Assessment und Evaluation, Universität Bern
| | - Eric Lenouvel
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern
| | - Susanne Lindert
- Clienia Schlössli, Privatklinik und Lehrkrankenhaus der Universität Zürich, Oetwil am See/Zürich
| | | | - Beat Nick
- Psychiatrische Dienste der Spitäler AG, Solothurn, Psychiatrische Dienste der Spitäler AG Solothurn, Präsident der Prüfungskommission und Mitglied der Kommission Weiter- und Fortbildung (SKWF), Schweizerische Gesellschaft für Psychiatrie und Psychotherapie (SGPP)
- Ressort Qualität der Ständigen Kommission Weiter- und Fortbildung (SKWF), Schweizerische Gesellschaft für Psychiatrie und Psychotherapie (SGPP)
| | - Christoph Nissen
- Universitätsklinik für Psychiatrie und Psychotherapie, Universität Bern
| | | | - Lea Stocker
- Akuttagesklinik, Psychiatrische Universitätsklinik Zürich
| | - Michael Wallies
- Privatklinik für Psychiatrie und Psychotherapie, Clienia Littenheid
| | | | | | - Stefan Klöppel
- Universitätsklinik für Alterspsychiatrie und Psychotherapie, Universität Bern
- Ressort Qualität der Ständigen Kommission Weiter- und Fortbildung (SKWF), Schweizerische Gesellschaft für Psychiatrie und Psychotherapie (SGPP)
| |
Collapse
|
21
|
Taylor D, Park YS, Smith C, Cate OT, Tekian A. Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice. TEACHING AND LEARNING IN MEDICINE 2021; 33:89-97. [PMID: 32634323 DOI: 10.1080/10401334.2020.1784740] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Issue: Entrustable Professional Activities (EPAs) describe the core tasks health professionals must be competent performing prior to promotion and/or moving into unsupervised practice. When used for learner assessment, they serve as gateways to increased responsibility and autonomy. It follows that identifying and describing EPAs is a high-stakes form of work analysis aiming to describe the core work of a profession. However, hasty creation and adoption of EPAs without rigorous attention to content threatens the quality of judgments subsequently made from using EPA-based assessment tools. There is a clear need for approaches to identify validity evidence for EPAs themselves prior to their deployment in workplace-based assessment. Evidence: For EPAs to realize their potential in health professions education, they must first be constructed to reflect accurately the work of that profession or specialty. If the EPAs fail to do so, they cannot predict a graduate's readiness for or future performance in professional practice. Evaluating the methods used for identification, description, and adoption of EPAs through a construct validity lens helps give leaders and stakeholders of EPA development confidence that the EPAs constructed are, in fact, an accurate representation of the profession's work. Implications: Application of a construct validity lens to EPA development impacts all five commonly followed steps in EPA development: selection of experts; identification of candidate EPAs; iterative revisions; evaluation of proposed EPAs; and formal adoption of EPAs into curricula. It allows curricular developers to avoid pitfalls, bias, and common mistakes. Further, construct validity evidence for EPA development provides assurance that the EPAs adopted are appropriate for use in workplace-based assessment and entrustment decision-making.
Collapse
Affiliation(s)
- David Taylor
- Department of Medicine, Queen's University, Kingston, Canada
| | - Yoon Soo Park
- Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois, USA
| | | | - Olle Ten Cate
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Ara Tekian
- Center for Research and Development of Education, University Medical Center Utrecht, Utrecht, The Netherlands
| |
Collapse
|
22
|
Young JQ, Sugarman R, Schwartz J, O'Sullivan PS. Faculty and Resident Engagement With a Workplace-Based Assessment Tool: Use of Implementation Science to Explore Enablers and Barriers. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1937-1944. [PMID: 32568853 DOI: 10.1097/acm.0000000000003543] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Implementation of workplace-based assessment programs has encountered significant challenges. Faculty and residents alike often have a negative view of these programs as "tick-box" or "jump through the hoops" exercises. A number of recommendations have been made to address these challenges. To understand the experience with a workplace-based assessment tool that follows many of these recommendations, the authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with the tool. METHOD The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance during a psychiatric medication management visit. From August 2017 to February 2018, the P-SCO was implemented in the outpatient continuity clinics for second- and third-year residents at Zucker Hillside Hospital/Northwell Health. In February and March 2019, the authors conducted semistructured interviews of participating faculty and residents. Interview guides based on the CFIR were used to capture the enablers and barriers to engagement. Interview transcripts were independently coded. Codes were then organized into themes relevant to the domains of the CFIR. RESULTS Ten faculty and 10 residents were interviewed. Overall, participants had a positive experience with the P-SCO. Enabling factors for faculty and residents included the ongoing training, design features of the P-SCO, predisposing beliefs, dedicated faculty time, and the perception that the P-SCO improved verbal feedback quality. Barriers for faculty included checklist length and discomfort with feedback that threatens identity, and barriers for residents included faculty variability in timeliness and quality of feedback and minimal review of the feedback after initial receipt. CONCLUSIONS This study demonstrates that the negative experience of faculty and residents with workplace-based assessment tools shown in prior studies can be overcome, at least in part, when specific implementation strategies are pursued. The findings provide guidance for future research and implementation efforts.
Collapse
Affiliation(s)
- John Q Young
- J.Q. Young is professor of psychiatry and vice chair for education, Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, and Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Rebekah Sugarman
- R. Sugarman is a research assistant, Department of Psychiatry, Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Jessica Schwartz
- J. Schwartz is a resident, Department of Psychiatry, Zucker Hillside Hospital at Northwell Health, Glen Oaks, New York
| | - Patricia S O'Sullivan
- P.S. O'Sullivan is professor, Department of Medicine, and director of research and development in medical education, University of California, San Francisco, School of Medicine, San Francisco, California
| |
Collapse
|
23
|
Young JQ, McClure M. Fast, Easy, and Good: Assessing Entrustable Professional Activities in Psychiatry Residents With a Mobile App. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1546-1549. [PMID: 32271227 DOI: 10.1097/acm.0000000000003390] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PROBLEM Entrustable professional activities (EPAs) can be used to operationalize competency-based medical education. Mobile apps can efficiently capture feedback based on direct observation. To leverage the benefits of both, the authors developed an assessment tool that combines EPAs with mobile technology. APPROACH The authors designed an app to collect EPA data based on direct observation using human-technology interface guidelines. Data collected in the app included: name of resident, the 13 end-of-training EPAs for psychiatry, entrustment ratings, and corrective narrative feedback. The app was implemented in an outpatient continuity clinic for second-year psychiatry residents over a 10-month period between September 2017 and June 2018. Ten faculty-resident dyads piloted the app. To assess the feasibility, utility, and validity of this intervention, the authors examined 3 outcomes: (1) utilization (mean time to complete each assessment; percentage of dyads who completed 10 assessments), (2) quality of the comments (proportion of comments that were behaviorally specific and actionable), and (3) correlation between entrustment level and resident experience (defined as days elapsed since the beginning of the experience). OUTCOMES A total of 99 assessments were completed during the pilot. Mean time to complete an assessment was 76 seconds (standard deviation = 50 seconds, median = 67 seconds). Only 6 of the 10 dyads completed at least 10 assessments. Of all comments, 95% (94) were behaviorally specific and actionable and 91% (90) were corrective. Entrustment scores correlated moderately with resident experience (r = 0.43, P < .001). NEXT STEPS The authors' EPA mobile app was efficient, generated high-quality feedback, and produced entrustment scores that improved as the residents gained experience. Challenges included uneven adoption. Looking forward, the authors plan to examine the enablers and barriers to adoption from an implementation science perspective.
Collapse
Affiliation(s)
- John Q Young
- J.Q. Young is professor, Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York
| | - Matthew McClure
- M. McClure is chief resident, Department of Psychiatry, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York
| |
Collapse
|
24
|
Young JQ, Sugarman R, Schwartz J, McClure M, O'Sullivan PS. A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement. PERSPECTIVES ON MEDICAL EDUCATION 2020; 9:210-219. [PMID: 32504446 PMCID: PMC7459074 DOI: 10.1007/s40037-020-00587-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2023]
Abstract
INTRODUCTION Mobile apps that utilize the framework of entrustable professional activities (EPAs) to capture and deliver feedback are being implemented. If EPA apps are to be successfully incorporated into programmatic assessment, a better understanding of how they are experienced by the end-users will be necessary. The authors conducted a qualitative study using the Consolidated Framework for Implementation Research (CFIR) to identify enablers and barriers to engagement with an EPA app. METHODS Structured interviews of faculty and residents were conducted with an interview guide based on the CFIR. Transcripts were independently coded by two study authors using directed content analysis. Differences were resolved via consensus. The study team then organized codes into themes relevant to the domains of the CFIR. RESULTS Eight faculty and 10 residents chose to participate in the study. Both faculty and residents found the app easy to use and effective in facilitating feedback immediately after the observed patient encounter. Faculty appreciated how the EPA app forced brief, distilled feedback. Both faculty and residents expressed positive attitudes and perceived the app as aligned with the department's philosophy. Barriers to engagement included faculty not understanding the EPA framework and scale, competing clinical demands, residents preferring more detailed feedback and both faculty and residents noting that the app's feedback should be complemented by a tool that generates more systematic, nuanced, and comprehensive feedback. Residents rarely if ever returned to the feedback after initial receipt. DISCUSSION This study identified key enablers and barriers to engagement with the EPA app. The findings provide guidance for future research and implementation efforts focused on the use of mobile platforms to capture direct observation feedback.
Collapse
Affiliation(s)
- John Q Young
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA.
| | - Rebekah Sugarman
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Jessica Schwartz
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Matthew McClure
- Donald and Barbara Zucker School of Medicine at Hofstra/Northwell and the Zucker Hillside Hospital at Northwell Health, Hempstead, NY, USA
| | - Patricia S O'Sullivan
- Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, USA
| |
Collapse
|
25
|
Competency-based resident education: The United States perspective. Surgery 2020; 167:777-781. [DOI: 10.1016/j.surg.2019.05.059] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2019] [Revised: 05/15/2019] [Accepted: 05/21/2019] [Indexed: 11/20/2022]
|
26
|
Abstract
In North America, the psychiatric workforce while relatively large, remains insufficient for the population needs. Specialist training opportunities in psychiatry continues to grow, reflected in both increased recruitment and competitiveness. Furthermore, the iterative process of curriculum development and standardisation of training encourages greater educational consistency. There is an ongoing growth in subspecialty training opportunities, however access varies across centres and curriculum gaps remain. The diverse demographic characteristics and requirements of populations also play a role in educational needs, such as youth mental health, tele-psychiatry, cultural, rural and addictions psychiatry. Advances in science such as genetics, and the use of technologies and digital media also invite innovative approaches to knowledge acquisition. Overall, training needs to be matched by sufficient numbers of experienced and skilful trainers, mentorship and leadership in psychiatry with awareness of physician wellbeing and the risks of burnout. In the USA and Canada, the growing prospect of fellowships in leadership and administration lay important foundations for the growth of psychiatry, driven by high quality education for the mental health professionals and leaders of tomorrow.
Collapse
Affiliation(s)
| | - Shadé Miller
- Psychiatry Resident, Department of Psychiatry, Queen's University, Kingston, Canada
| | - Shiva K Shivakumar
- Department of Psychiatry, Health Sciences North, Northern Ontario School of Medicine, Sudbury, Canada
| |
Collapse
|
27
|
Pinilla S, Lenouvel E, Strik W, Klöppel S, Nissen C, Huwendiek S. Entrustable Professional Activities in Psychiatry: A Systematic Review. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2020; 44:37-45. [PMID: 31732885 DOI: 10.1007/s40596-019-01142-7] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/26/2019] [Accepted: 10/31/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Entrustable professional activities (EPAs) represent discrete clinical tasks that can be entrusted to trainees in psychiatry. They are increasingly being used as educational framework in several countries. However, the empirical evidence available has not been synthesized in the field of psychiatry. Therefore, the authors conducted a systematic review in order to summarize and evaluate the available evidence in the field of EPAs in undergraduate and graduate medical education in psychiatry. METHODS The authors searched PubMed, Cochrane Library, ERIC, Embase, PsycINFO, all Ovid journals, Scopus, Web of Science, MedEdPORTAL, and the archives of Academic Psychiatry for articles reporting quantitative and qualitative research as well as educational case reports on EPAs in undergraduate and graduate psychiatry education published since 2005. All included articles were assessed for content (development, implementation, and assessment of EPAs) and quality using the Quality Assessment Tool for Studies with Diverse Designs. RESULTS The authors screened 2807 records and included a total of 20 articles in the final data extraction. Most studies were expert consensus reports (n = 6, 30%) and predominantly conducted in English-speaking countries (n = 17, 85%). Papers reported mainly EPA development and/or EPA implementation studies (n = 14, 70%), whereas EPA assessment studies were less frequent (n = 6, 30%). Publications per year showed an increasing trend both in quantity (from 1 in 2011 to 7 in 2018) and quality (from a QATSDD score of 27 in 2011 to an average score of 39 in 2018). The main focus of the articles was the development of individual EPAs for different levels of training for psychiatry or on curricular frameworks based on EPAs in psychiatry (n = 10, 50%). The lack of empirical controlled studies does currently not allow for meta-analyses of educational outcomes. CONCLUSIONS The concept of EPA-based curricula seems to become increasingly present, a focus in the specialty of psychiatry both in UME and GME. The lack of empirical research in this context is an important limitation for educational practice recommendations. Currently there is only preliminary but promising data available for using EPAs with regard to educational outcomes. EPAs seem to be effectively used from a curriculum design perspective for UME and GME in psychiatry.
Collapse
Affiliation(s)
- Severin Pinilla
- University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland.
| | - Eric Lenouvel
- University Hospital of Old Age Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
| | - Werner Strik
- University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
| | - Stefan Klöppel
- University Hospital of Old Age Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
| | - Christoph Nissen
- University Hospital of Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland
| | - Sören Huwendiek
- Institute for Medical Education, University of Bern, Bern, Switzerland
| |
Collapse
|
28
|
Schumacher DJ, West DC, Schwartz A, Li ST, Millstein L, Griego EC, Turner T, Herman BE, Englander R, Hemond J, Hudson V, Newhall L, McNeal Trice K, Baughn J, Giudice E, Famiglietti H, Tolentino J, Gifford K, Carraccio C. Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities. JAMA Netw Open 2020; 3:e1919316. [PMID: 31940042 PMCID: PMC6991321 DOI: 10.1001/jamanetworkopen.2019.19316] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
IMPORTANCE Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. OBJECTIVE To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. DESIGN, SETTING, AND PARTICIPANTS Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. INTERVENTIONS Longitudinal, prospective assessment using EPAs. MAIN OUTCOMES AND MEASURES Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. RESULTS Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). CONCLUSIONS AND RELEVANCE This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.
Collapse
Affiliation(s)
- Daniel J. Schumacher
- Department of Pediatrics, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
- University of Cincinnati College of Medicine, Cincinnati, Ohio
| | - Daniel C. West
- Department of Pediatrics, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
- Department of Pediatrics, University of Pennsylvania, Philadelphia
| | - Alan Schwartz
- Department of Medical Education, University of Illinois at Chicago
- Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network, McLean, Virginia
- Department of Pediatrics, University of Illinois at Chicago
| | - Su-Ting Li
- Department of Pediatrics at the University of California Davis Health, Sacramento
| | - Leah Millstein
- Department of Pediatrics, University of Maryland School of Medicine, Baltimore
| | - Elena C. Griego
- Department of Pediatrics, Seattle Children’s Hospital, Seattle, Washington
| | - Teri Turner
- Department of Pediatrics, Texas Children’s Hospital/Baylor College of Medicine, Houston
| | - Bruce E. Herman
- Department of Pediatrics, University of Utah, Salt Lake City
| | - Robert Englander
- Department of Pediatrics, University of Minnesota Medical School, Minneapolis
| | - Joni Hemond
- Department of Pediatrics, University of Utah, Salt Lake City
| | - Valera Hudson
- Department of Pediatrics, Children’s Hospital of Georgia/Augusta University, Augusta
| | - Lauren Newhall
- Department of Pediatrics, Children’s Hospital of Georgia/Augusta University, Augusta
| | | | - Julie Baughn
- Department of Pediatrics, Mayo Medical School, Rochester, Minnesota
| | - Erin Giudice
- Department of Pediatrics, University of Maryland School of Medicine, Baltimore
| | | | - Jonathan Tolentino
- Department of Pediatrics, Stony Brook University, Stony Brook, New York
- Department of Internal Medicine, Stony Brook University, Stony Brook, New York
| | - Kimberly Gifford
- Department of Pediatrics, Dartmouth University, Lebanon, New Hampshire
| | | |
Collapse
|
29
|
Young JQ. Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation. J Grad Med Educ 2019; 11:570-579. [PMID: 31636828 PMCID: PMC6795331 DOI: 10.4300/jgme-d-19-00207.1] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2019] [Revised: 07/07/2019] [Accepted: 08/05/2019] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND While prior research has focused on the validity of quantitative ratings generated by direct observation tools, much less is known about the written comments. OBJECTIVE This study examines the quality of written comments and their relationship with checklist scores generated by a direct observation tool, the Psychopharmacotherapy-Structured Clinical Observation (P-SCO). METHODS From 2008 to 2012, faculty in a postgraduate year 3 psychiatry outpatient clinic completed 601 P-SCOs. Twenty-five percent were randomly selected from each year; the sample included 8 faculty and 57 residents. To assess quality, comments were coded for valence (reinforcing or corrective), behavioral specificity, and content. To assess the relationship between comments and scores, the authors calculated the correlation between comment and checklist score valence and examined the degree to which comments and checklist scores addressed the same content. RESULTS Ninety-one percent of the comments were behaviorally specific. Sixty percent were reinforcing, and 40% were corrective. Eight themes were identified, including 2 constructs not adequately represented by the checklist. Comment and checklist score valence was moderately correlated (Spearman's rho = 0.57, P < .001). Sixty-seven percent of high and low checklist scores were associated with a comment of the same valence and content. Only 50% of overall comments were associated with a checklist score of the same valence and content. CONCLUSIONS A direct observation tool such as the P-SCO can generate high-quality written comments. Narrative comments both explain checklist scores and convey unique content. Thematic coding of comments can improve the content validity of a checklist.
Collapse
|
30
|
Shorey S, Lau TC, Lau ST, Ang E. Entrustable professional activities in health care education: a scoping review. MEDICAL EDUCATION 2019; 53:766-777. [PMID: 30945329 DOI: 10.1111/medu.13879] [Citation(s) in RCA: 114] [Impact Index Per Article: 22.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Revised: 11/30/2018] [Accepted: 02/22/2019] [Indexed: 05/13/2023]
Abstract
CONTEXT The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care. METHOD Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes. RESULTS Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs). CONCLUSIONS Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.
Collapse
Affiliation(s)
- Shefaly Shorey
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | | | - Siew Tiang Lau
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| | - Emily Ang
- Alice Lee Centre for Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- National University Health System, Singapore, Singapore
- Yong Loo Lin School of Medicine, Singapore, Singapore
| |
Collapse
|
31
|
O'Dowd E, Lydon S, O'Connor P, Madden C, Byrne D. A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011-2018. MEDICAL EDUCATION 2019; 53:234-249. [PMID: 30609093 DOI: 10.1111/medu.13792] [Citation(s) in RCA: 101] [Impact Index Per Article: 20.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Revised: 11/05/2018] [Accepted: 11/20/2018] [Indexed: 05/13/2023]
Abstract
PURPOSE This review aimed to synthesise some of the extant work on the use of entrustable professional activities (EPAs) for postgraduate physicians, to assess the quality of the work and provide direction for future research and practice. METHOD Systematic searches were conducted within five electronic databases (Medline, Scopus, Web of Science, PsycINFO and CINAHL) in September 2018. Reference lists, Google Scholar and Google were also searched. Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). RESULTS In total, 49 studies were included, classified as Development of EPAs (n = 37; 76% of total included), Implementation and/or assessment of EPAs (n = 10; 20%), or both (n = 2; 4%). EPAs were described for numerous specialties, including internal medicine (n = 14; 36%), paediatrics (n = 8; 21%) and psychiatry (n = 4; 10%). Of the development studies, 92% utilised more than one method to generate EPAs. The two most commonly used methods were developing initial EPAs in a working group, (n = 27; 69%) and revising through deliberation (n = 21; 54%). Development papers were of variable quality (mean QATSDD score = 20, range 6-41). Implementation and assessment studies utilised methods that included observing trainee performance (n = 6; 50%) and enrolling trainees in competency-based curricula, which included EPAs (n = 4; 33%). The methodological quality of these implementation studies varied (mean QATSDD score = 19.5, range = 6-32). CONCLUSIONS This review highlighted a need for: (i) consideration of best practice guidelines for EPA development; (ii) focus on the methodological quality of research on EPA development and of EPAs, and (iii) further work investigating the implementation of EPAs in the curriculum.
Collapse
Affiliation(s)
- Emily O'Dowd
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Sinéad Lydon
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| | - Paul O'Connor
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Caoimhe Madden
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- Discipline of General Practice, School of Medicine, National University of Ireland, Galway, Ireland
| | - Dara Byrne
- Irish Centre for Applied Patient Safety and Simulation, National University of Ireland, Galway, Ireland
- School of Medicine, National University of Ireland, Galway, Ireland
| |
Collapse
|
32
|
Young JQ, Irby DM, Kusz M, O'Sullivan PS. Performance Assessment of Pharmacotherapy: Results from a Content Validity Survey of the Psychopharmacotherapy-Structured Clinical Observation (P-SCO) Tool. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2018; 42:765-772. [PMID: 29380145 DOI: 10.1007/s40596-017-0876-0] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/21/2017] [Accepted: 12/22/2017] [Indexed: 06/07/2023]
Abstract
OBJECTIVE The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) tool is designed to assess performance of a medication management visit and to enhance feedback. Prior research indicated that the P-SCO was feasible to implement in a resident clinic and generated behaviorally specific, high-quality feedback. This research also highlighted problems with some of the instrument's items. This study seeks to improve the items. METHODS The authors initially revised the P-SCO items based on the problems identified by a prior study. Next, these items were iteratively modified by experts in clinical pharmacotherapy and educational assessment. Forty-five items emerged. Finally, faculty attending an annual department education retreat rated each item on its relevance (4-point scale) and provided comments on how the item might be revised. For final inclusion, an item must have met a quantitative threshold (i.e., content validity index equal to or greater than 0.8 and the lower end of the asymmetric confidence interval equal to or greater than 3.0) and received comments that were supportive. RESULTS Forty-one of the 45 items had strong quantitative support. However, the comments endorsed lumping a number of items in order to decrease overlap between items and to shorten the instrument. This process resulted in the further elimination of 15 items. CONCLUSIONS The revised 26-item P-SCO builds upon prior evidence of feasibility and utility and now possesses additional evidence of content validity. The use of the tool should enhance feedback and improve the capacity of educational programs to assess performance.
Collapse
Affiliation(s)
- John Q Young
- Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | | | - Martin Kusz
- Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA
| | | |
Collapse
|
33
|
Young JQ, Rasul R, O'Sullivan PS. Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis. ACADEMIC PSYCHIATRY : THE JOURNAL OF THE AMERICAN ASSOCIATION OF DIRECTORS OF PSYCHIATRIC RESIDENCY TRAINING AND THE ASSOCIATION FOR ACADEMIC PSYCHIATRY 2018; 42:759-764. [PMID: 29951950 DOI: 10.1007/s40596-018-0928-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/28/2017] [Accepted: 04/18/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance of a medication visit. This study examines two dimensions of validity for the P-SCO: internal structure and how scores correlate with another variable associated with competence (experience). METHODS The faculty completed 601 P-SCOs over 4 years. Multilevel exploratory factor analysis was performed with minimum thresholds for eigenvalue (≥ 1.0) and proportion of variance explained (≥ 5.0%). Internal reliability was assessed with Cronbach alpha. To examine how scores changed with experience, mean ratings (1-4 scale) were calculated for each factor by quarter of the academic year. Separate linear mixed models were also performed. RESULTS The analysis yielded three factors that explained 50% of the variance and demonstrated high internal reliability: affective tasks (alpha = 0.90), cognitive tasks (alpha = 0.84), and hard tasks (alpha = 0.74). Items within "hard tasks" were assessment of substance use, violence risk, and adherence, and inquiry about interactions with other providers. Monitoring adverse effects did not load on the hard task factor but also had overall low mean ratings. Compared to the first quarter, fourth quarter scores for affective tasks (b = 0.54, p < 0.01) and hard tasks (b = 0.46, p = 0.02) were significantly improved while cognitive tasks had a non-significant increase. For the hard tasks, the proportion of residents with a low mean rating improved but was still over 30% during the fourth quarter. CONCLUSIONS The results provide evidence for the validity of the P-SCO with respect to its internal structure and how scores correlate with experience. Curricular implications are explored, especially for the tasks that were hard to learn.
Collapse
Affiliation(s)
- John Q Young
- Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA.
| | | | | |
Collapse
|
34
|
Sklar DP. Supporting Our Residents: A Time for Vision and Voice. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:955-958. [PMID: 29944537 DOI: 10.1097/acm.0000000000002242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
|