Precision Medical Education

Evaluation of a Direct Observation, Coaching and Assessment Model for the Internal Medicine Clerkship

BACKGROUND

With a trend towards competency-based assessment in medical education, there is a need for increased direct observation, feedback and coaching of medical students during clinical rotations.

APPROACH

To increase observation and provide more coaching and feedback, we designed a model in which a faculty coach met with students longitudinally during the internal medicine clerkship. The first session included an observed history and physical (H&P), and the coach and student identified skill areas to focus on in remaining sessions. All sessions included a debrief with feedback.

EVALUATION

Students received a survey to rate the amount and quality of observation and feedback received, and we used ordinal logistic regression models to assess the intervention. We conducted thematic analysis to assess what students found most useful. Students in the intervention group reported more direct observation performing the H&P (OR = 9.17, 95% CI [1.86, 70.05], p = 0.01) and found the personalized feedback and increased opportunities to discuss clinical reasoning valuable.

IMPLICATIONS

With a growing need for longitudinal observation of clinical skills to allow for competency-based assessments, at a time in which there is often insufficient continuity between students and supervising physicians, this model helps address needs for increased direct observation, coaching and feedback on skill development over time.

Beyond thinking fast and slow: Implications of a transtheoretical model of clinical reasoning and error on teaching, assessment, and research

From dual process to a family of theories known collectively as situativity, both micro and macro theories of cognition inform our current understanding of clinical reasoning (CR) and error. CR is a complex process that occurs in a complex environment, and a nuanced, expansive, integrated model of these theories is necessary to fully understand how CR is performed in the present day and in the future. In this perspective, we present these individual theories along with figures and descriptive cases for purposes of comparison before exploring the implications of a transtheoretical model of these theories for teaching, assessment, and research in CR and error.

Towards precision well-being in medical education

Medical trainee well-being is often met with generalized solutions that overlook substantial individual variations in mental health predisposition and stress reactivity. Precision medicine leverages individual environmental, genetic, and lifestyle factors to tailor preventive and therapeutic interventions. In addition, an exclusive focus on clinical mental illness tends to disregard the importance of supporting the positive aspects of medical trainee well-being. We introduce a novel precision well-being framework for medical education that is built on a comprehensive and individualized view of mental health, combining measures from mental health and positive psychology in a unified, data-driven framework. Unsupervised machine learning techniques commonly used in precision medicine were applied to uncover patterns within multidimensional mental health data of medical students. Using data from 3,632 US medical students, clusters were formulated based on recognized metrics for depression, anxiety, and flourishing. The analysis identified three distinct clusters. Membership in the 'Healthy Flourishers' well-being phenotype was associated with no signs of anxiety or depression while simultaneously reporting high levels of flourishing. Students in the 'Getting By' cluster reported mild anxiety and depression and diminished flourishing. Membership in the 'At-Risk' cluster was associated with high anxiety and depression, languishing, and increased suicidality. Nearly half (49%) of the medical students surveyed were classified as 'Healthy Flourishers', whereas 36% were grouped into the 'Getting-By' cluster and 15% were identified as 'At-Risk'. Findings show that a substantial portion of medical students report diminished well-being during their studies, with a significant number struggling with mental health challenges. This novel precision well-being framework represents an integrated empirical model that classifies individual medical students into distinct and meaningful well-being phenotypes based on their holistic mental health. This approach has direct applicability to student support and can be used to evaluate the effectiveness of personalized intervention strategies stratified by cluster membership.

Resident clinical dashboards to support precision education in emergency medicine

Introduction

With the move toward competency-based medical education (CBME), data from the electronic health record (EHR) for informed self-improvement may be valuable as a part of programmatic assessment. Personalized dashboards are one way to view these clinical data. The purpose of this concept paper is to summarize the current state of clinical dashboards as they can be utilized by emergency medicine (EM) residency programs.

Methods

The author group consisted of EM physicians from multiple institutions with medical education and informatics backgrounds and was identified by querying faculty presenting on resident clinical dashboards at the 2024 Society for Academic Emergency Medicine conference. Additional authors were identified by members of the initial group. Best practice literature was referenced; if none was available, group consensus was used.

Categories of Metrics

Clinical exposures as well as efficiency, quality, documentation, and diversity metrics may be included in a resident dashboard. Resident dashboard metrics should focus on resident-sensitive measures rather than those primarily affected by attendings or systems-based factors.

Considerations for Implementation

Implementation of these dashboards requires the technical expertise to turn EHR data into actionable data, a process called EHR phenotyping. The dashboard can be housed directly in the EHR or on a separate platform. Dashboard developers should consider how their implementation plan will affect how often dashboard data will be refreshed and how to best display the data for ease of understanding.

Implications for Education & Training

Dashboards can provide objective data to residents, residency leadership and clinical competency committees as they identify areas of strength, growth areas, and set specific and actionable goals. The success of resident dashboards is reliant on resident buy-in and creating a culture of psychological safety through thoughtful implementation, coaching, and regular feedback. .

Conclusion

Personalized clinical dashboards can play a crucial role in programmatic assessment within CBME, helping EM residents focus their efforts as they advance and refine their skills during training.

Bridging Artificial Intelligence and Medical Education: Navigating the Alignment Paradox

The integration of artificial intelligence (AI) into medical education presents both unprecedented opportunities and significant challenges, epitomized by the "alignment paradox." This paradox asks: How do we ensure AI systems remain aligned with our educational goals? For instance, AI could create highly personalized learning pathways, but this might conflict with educators' intentions for structured skill development. This paper proposes a framework to address this paradox, focusing on four key principles: ethics, robustness, interpretability, and scalable oversight. We examine the current landscape of AI in medical education, highlighting its potential to enhance learning experiences, improve clinical decision making, and personalize education. We review ethical considerations, emphasize the importance of robustness across diverse healthcare settings, and present interpretability as crucial for effective human-AI collaboration. For example, AI-based feedback systems like i-SIDRA enable real-time, actionable feedback, enhancing interpretability while reducing cognitive overload. The concept of scalable oversight is introduced to maintain human control while leveraging AI's autonomy. We outline strategies for implementing this oversight, including directable behaviors and human-AI collaboration techniques. With this road map, we aim to support the medical education community in responsibly harnessing AI's power in its educational systems.

Revealing Patient Dissatisfaction With Health Care Resource Allocation in Multiple Dimensions Using Large Language Models and the International Classification of Diseases 11th Revision: Aspect-Based Sentiment Analysis

BACKGROUND

Accurately measuring the health care needs of patients with different diseases remains a public health challenge for health care management worldwide. There is a need for new computational methods to be able to assess the health care resources required by patients with different diseases to avoid wasting resources.

OBJECTIVE

This study aimed to assessing dissatisfaction with allocation of health care resources from the perspective of patients with different diseases that can help optimize resource allocation and better achieve several of the Sustainable Development Goals (SDGs), such as SDG 3 ("Good Health and Well-being"). Our goal was to show the effectiveness and practicality of large language models (LLMs) in assessing the distribution of health care resources.

METHODS

We used aspect-based sentiment analysis (ABSA), which can divide textual data into several aspects for sentiment analysis. In this study, we used Chat Generative Pretrained Transformer (ChatGPT) to perform ABSA of patient reviews based on 3 aspects (patient experience, physician skills and efficiency, and infrastructure and administration)00 in which we embedded chain-of-thought (CoT) prompting and compared the performance of Chinese and English LLMs on a Chinese dataset. Additionally, we used the International Classification of Diseases 11th Revision (ICD-11) application programming interface (API) to classify the sentiment analysis results into different disease categories.

RESULTS

We evaluated the performance of the models by comparing predicted sentiments (either positive or negative) with the labels judged by human evaluators in terms of the aforementioned 3 aspects. The results showed that ChatGPT 3.5 is superior in a combination of stability, expense, and runtime considerations compared to ChatGPT-4o and Qwen-7b. The weighted total precision of our method based on the ABSA of patient reviews was 0.907, while the average accuracy of all 3 sampling methods was 0.893. Both values suggested that the model was able to achieve our objective. Using our approach, we identified that dissatisfaction is highest for sex-related diseases and lowest for circulatory diseases and that the need for better infrastructure and administration is much higher for blood-related diseases than for other diseases in China.

CONCLUSIONS

The results prove that our method with LLMs can use patient reviews and the ICD-11 classification to assess the health care needs of patients with different diseases, which can assist with resource allocation rationally.

Ethical considerations of using learning analytics in medical education: a critical review

Learning analytics are increasingly used in medical education to analyze data and make decisions about learners' abilities. While there are many potential benefits of using learning analytics to drive improvement in medical education, there are also ethical concerns surrounding how this may affect learners and their patients. We conducted a critical review of studies that use learning analytics and big data within medical education. Using guidelines established by Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), relevant articles were identified in MEDLINE (PubMed) and SocINDEX databases from inception to April 2021. Detailed data abstraction was performed across studies to identify current uses of learning analytics and identify potential ethical concerns. Eighteen articles met the search criteria. Our analysis identified the use of learning analytics and big data in four aspects of medical education: (1) the learning process and pedagogy; (2) retrospective assessment; (3) prospective assessment; and (4) improvement of education. We identified some ethical concerns surrounding the use of learning analytics and big data, including the (1) trustworthiness of data; (2) reliability of methodology; (3) privacy, confidentiality, and management of data; and (4) labeling of learners as "problematic." Using Beauchamp and Childress's biomedical ethics as a framework, we identified potential consequences of using learning analytics for learners within the principles of beneficence, nonmaleficence, autonomy, and justice. As learning analytics becomes more widespread in medical education, examining and mitigating potential harm towards learners is imperative.

Stepping Back: How Should Pass/Fail Scoring Influence Step 1 Timing?

ABSTRACT

Although most students complete Step 1 before clerkships, some institutions delay the exam until after clerkships. The change to pass/fail grading adds additional complexity that should be considered when deciding about exam timing. Both early and late administration may affect learning outcomes, learner behavior, student well-being, and residency match success. Step 1 completion before clerkships promotes learning outcomes (e.g., integration and mastery of foundational material), may encourage students to focus on the curriculum, and may better prepare students for clinical science exams (CSEs). However, delaying the exam ensures that students maintain foundational knowledge and may encourage clinical educators to demonstrate basic science illustrations. An early Step 1 may affect learner behavior by allowing clerkship students to focus on clinical learning. The associated National Board of Medical Examiners performance report may also be used for Step 2 and CSE preparation. However, delaying Step 1 allows greater scheduling flexibility based on developmental milestones. Administration of Step 1 before clerkships removes a significant stressor from the clinical year and decompresses the residency application period. However, a delayed Step 1 reduces the pressure on students to engage in numerous extracurricular and research activities to distinguish themselves due to the pass/fail change. An early Step 1 exam may also lead to improved CSE performance, which is often linked to clerkship honors criteria, an increasingly valuable distinction for residency match success after the change to pass/fail. In contrast, delaying Step 1 is associated with higher first-time pass rates, which may be especially important for students at risk for failure. Medical educators and students should collaboratively approach the question of Step 1 timing, considering these factors within the context of the medical school program, curricular constraints and priorities, and students' individual needs and goals.

Coaching for improving clinical performance of surgeons: a scoping review

Surgical coaching has been proven to effectively enhance clinical performance. However, diverse implementation strategies present challenges when initiating new programs. Our scoping review aimed to synthesize the existing literature on surgical coaching, thereby informing the direction of future coaching initiatives. We reviewed published articles in PubMed/Medline and suppletory manuscripts from reference lists. The protocol of our review was registered (osf.io, Z3S8H). Inclusion criteria were studies that provided a detailed description of structured surgical coaching programs aimed at improving clinical performance. Excluded were studies focused on mentoring, teaching, or other forms of coaching that did not align with our specific definition of surgical coaching. We extracted and charted variables such as authors, publication year, geographic region, and others for subsequent analysis. A total of 117 studies were screened, and 11 met our inclusion criteria. Among these, five articles (45%) employed objective metrics to evaluate clinician performance. One study reported on the overall complication rates within 30 days as a measured outcome. Surgeons were the primary coachees in ten of the studies (91%), and the training of coaches was deemed necessary in seven studies (64%). The analyses revealed a preference for expert coaching models (6/11, 55%), video-based coaching (9/11, 82%), and postoperative timelines (7/11, 64%). Various coaching models were identified, including PRACTICE, GROW, and WISCONSIN. As an effective education method, surgical coaching has been conducted in many regions with varied designs. The implementation of structured surgical coaching programs offers substantial benefits for trainers, enhancing efficiency. Future research should focus on generating higher-level evidence, utilizing objective measurement tools, and integrating innovative technologies to further enhance the efficacy of surgical coaching programs.

Artificial Intelligence in Health Professions Education assessment: AMEE Guide No. 178

Health Professions Education (HPE) assessment is being increasingly impacted by Artificial Intelligence (AI), and institutions, educators, and learners are grappling with AI's ever-evolving complexities, dangers, and potential. This AMEE Guide aims to assist all HPE stakeholders by helping them navigate the assessment uncertainty before them. Although the impetus is AI, the Guide grounds its path in pedagogical theory, considers the range of human responses, and then deals with assessment types, challenges, AI roles as tutor and learner, and required competencies. It then discusses the difficult and ethical issues, before ending with considerations for faculty development and the technicalities of AI acknowledgment in assessment. Through this Guide, we aim to allay fears in the face of change and demonstrate possibilities that will allow educators and learners to harness the full potential of AI in HPE assessment.

Introducing the Next Era in Assessment

In this introduction, the guest editors of the "Next Era in Assessment" special collection frame the invited papers by envisioning a next era in assessment of medical education, based on ideas developed during a summit that convened professional and educational leaders and scholars. The authors posit that the next era of assessment will focus unambiguously on serving patients and the health of society, reflect its sociocultural context, and support learners' longitudinal growth and development. As such, assessment will be characterized as transformational, development-oriented and socially accountable. The authors introduce the papers in this special collection, which represent elements of a roadmap towards the next era in assessment by exploring several foundational considerations that will make the next era successful. These include the equally important issues of (1) focusing on accountability, trust and power in assessment, (2) addressing implementation and contextualization of assessment systems, (3) optimizing the use of technology in assessment, (4) establishing infrastructure for data sharing and data storage, (5) developing a vocabulary around emerging sources of assessment data, and (6) reconceptualizing validity around patient care and learner equity. Attending to these priority areas will help leaders create authentic assessment systems that are responsive to learners' and society's needs, while reaping the full promise of competency-based medical education (CBME) as well as emerging data science and artificial intelligence technologies.

Toward precision medical education: Characterizing individual residents' clinical experiences throughout training

BACKGROUND

Despite the central role of experiential learning in residency training, the actual clinical experiences residents participate in are not well characterized. A better understanding of the type, volume, and variation in residents' clinical experiences is essential to support precision medical education strategies.

OBJECTIVE

We sought to characterize the entirety of the clinical experiences had by individual internal medicine residents throughout their time in training.

METHOD

We evaluated the clinical experiences of medicine residents (n = 51) who completed training at NYU Grossman School of Medicine's Brooklyn campus between 2020 and 2023. Residents' inpatient and outpatient experiences were identified using notes written, orders placed, and care team sign-ins; principal ICD-10 codes for each encounter were converted into medical content categories using a previously described crosswalk tool.

RESULTS

Of 152,426 clinical encounters with available ICD-10 codes, 132,284 were mapped to medical content categories (94.5% capture). Residents' clinical experiences were particularly enriched in infectious and cardiovascular disease; most had very little exposure to allergy, dermatology, oncology, or rheumatology. Some trainees saw twice as many cases in a given content area as did others. There was little concordance between actual frequency of clinical experience and expected content frequency on the ABIM certification exam.

CONCLUSIONS

Individual residents' clinical experiences in training vary widely, both in number and in type. Characterizing these experiences paves the way for exploration of the relationships between clinical exposure and educational outcomes, and for the implementation of precision education strategies that could fill residents' experiential gaps and complement strengths with targeted educational interventions.

Proximity to Practice: The Role of Technology in the Next Era of Assessment

The integration of technology into health professions assessment has created multiple possibilities. In this paper, we focus on the challenges and opportunities of integrating technologies that are used during clinical activities or that are completed by raters after a clinical encounter. In focusing on technologies that are more proximal to practice, we identify tradeoffs with different data collection approaches. To maximize the benefits of integrating technology in workplace-based assessment, we describe the importance of using preexisting frameworks from the fields of assessment design, implementation research, and clinical artificial intelligence governance.

Assessing how paramedical faculty's professors and students at Kurdistan- Iraq Universities of Medical Sciences and Health Services perceive the quality of clinical learning environments through the application of the DREEM model

BACKGROUND

The education process in paramedical faculties is influenced by various factors affecting students' learning.The present study aimed to assess the quality of clinical learning environments in the paramedical faculty of Kurdistan-Iraq Universities of Medical Science.

METHODS

This study used a cross-sectional survey based on convenience sampling, involving 552 paramedical students and 125 professors from relevant departments within Kurdistan-Iraq's medical sciences universities. Data analysis involved descriptive and inferential statistics, including frequency, mean, standard deviation,, t-test and ANOVA, conducted using SPSS version 22.

RESULTS

The mean scores of the overall DREEM scores for students and professors were 115.07 ± 31.65 and 95.16 ± 12.67, respectively. Significant differences were observed in students' fields of study and in all areas of the attitudes questionnaire. Additionally, a significant difference was found between professors' degrees in the scientific ability subscale of attitude towards the fields, and characteristics of clinical learning environments (p < .05).

CONCLUSION

The findings suggest relative satisfaction among professors and students with the clinical education environment within the paramedical faculty.

Improving primary healthcare quality in China through training needs analysis

The diagnosis and treatment capability levels of healthcare professionals directly affect the overall quality of medical services. Enhancing these capability levels requires strengthening the professional skill training of healthcare workers, especially those in primary care settings in economically underdeveloped areas. To understand the actual needs for professional skill training among primary healthcare workers, thereby providing data support for targeted training initiatives. An online survey was conducted using convenience sampling and subsequently, snowball sampling from May 10, 2023 to January 31, 2024. The survey included 3811 healthcare workers across China, and 3617 valid questionnaires were recovered. Descriptive analysis was used to compare the professional backgrounds and training needs of healthcare workers at different medical facility levels. Univariate and multivariate logistic regression analyses were employed to explore factors influencing the training of primary healthcare workers. The survey revealed that 70.1% of respondents were female, and 94.2% were from medical facilities below the provincial level, with 43.2% from township-level or lower medical facilities. Significant differences were found in age distribution, work experience, professional titles, educational levels, and training needs among provincial/national, prefectural/county, and township levels (P < 0.05). Busy clinical work schedules were the primary barrier to training participation. Most healthcare workers (78.7%) expected more than four training sessions per year, with the optimal frequency being quarterly. The most anticipated training topics among primary care workers were latest medical guidelines, new technologies/skills, and advanced management concepts, with over 80% interest. Compared with prefectural/county-level facilities, primary care workers at grassroots facilities are more significantly impacted by lower professional titles, lower education levels, weaker medical/technical skills, and insufficient specialized funding (odds ratio > 1, P < 0.001). Training should be tailored to the needs of healthcare workers at different medical facility levels. Particularly for primary care settings, providing special funding support and training in the latest medical guidelines, new technologies/skills, and advanced management concepts are important to improve the composition of titles, education, and professional technicians.

Implementing an accelerated three-year MD curriculum at NYU Grossman School of Medicine

Over the last decade there has been tremendous growth in the development of accelerated MD pathways that allow medical students to graduate in three years. Developing an accelerated pathway program requires commitment from students and faculty with intensive re-thinking and altering of the curriculum to ensure adequate content to achieve competency in an accelerated timeline. A re-visioning of assessment and advising must follow and the application of AI and new technologies can be added to support teaching and learning. We describe the curricular revision to an accelerated pathway at NYU Grossman School of Medicine highlighting our thought process, conceptual framework, assessment methods and outcomes over the last ten years.

Precision emergency medicine

BACKGROUND

Precision health is a burgeoning scientific discipline that aims to incorporate individual variability in biological, behavioral, and social factors to develop personalized health solutions. To date, emergency medicine has not deeply engaged in the precision health movement. However, rapid advances in health technology, data science, and medical informatics offer new opportunities for emergency medicine to realize the promises of precision health.

METHODS

In this article, we conceptualize precision emergency medicine as an emerging paradigm and identify key drivers of its implementation into current and future clinical practice. We acknowledge important obstacles to the specialty-wide adoption of precision emergency medicine and offer solutions that conceive a successful path forward.

RESULTS

Precision emergency medicine is defined as the use of information and technology to deliver acute care effectively, efficiently, and authentically to individual patients and their communities. Key drivers and opportunities include leveraging human data, capitalizing on technology and digital tools, providing deliberate access to care, advancing population health, and reimagining provider education and roles. Overcoming challenges in equity, privacy, and cost is essential for success. We close with a call to action to proactively incorporate precision health into the clinical practice of emergency medicine, the training of future emergency physicians, and the research agenda of the specialty.

CONCLUSIONS

Precision emergency medicine leverages new technology and data-driven artificial intelligence to advance diagnostic testing, individualize patient care plans and therapeutics, and strategically refine the convergence of the health system and the community.

Precision surgical education

Information and data are accelerating the implementation of competency-based medical education. The adoption of precision education can contribute to this purpose. This article discusses the extent to which precision surgical education can be used in assessing the minimum reliability standards of future surgeons - given the advent of Entrustable Professional Activities - and as an option to strengthen the career trajectory of residents.

Using the electronic health record to provide audit and feedback in medical student clerkships

Objectives

This article focuses on the role of the electronic health record (EHR) to generate meaningful formative feedback for medical students in the clinical setting. Despite the scores of clinical data housed within the EHR, medical educators have only just begun to tap into this data to enhance student learning. Literature to-date has focused almost exclusively on resident education.

Materials and Methods

Development of EHR auto-logging and triggered notifications are discussed as specific use cases in providing enhanced feedback for medical students.

Results

By incorporating predictive and prescriptive analytics into the EHR, there is an opportunity to create powerful educational tools which may also support general clinical activity.

Discussion

This article explores the possibilities of EHR as an educational resource. This serves as a call to action for educators and technology developers to work together on creating health record user-centric tools, acknowledging the ongoing work done to improve student-level attribution to patients.

Conclusion

EHR analytics and tools present a novel approach to enhancing clinical clerkship education for medical students.

Embracing ChatGPT for Medical Education: Exploring Its Impact on Doctors and Medical Students

ChatGPT (OpenAI), a cutting-edge natural language processing model, holds immense promise for revolutionizing medical education. With its remarkable performance in language-related tasks, ChatGPT offers personalized and efficient learning experiences for medical students and doctors. Through training, it enhances clinical reasoning and decision-making skills, leading to improved case analysis and diagnosis. The model facilitates simulated dialogues, intelligent tutoring, and automated question-answering, enabling the practical application of medical knowledge. However, integrating ChatGPT into medical education raises ethical and legal concerns. Safeguarding patient data and adhering to data protection regulations are critical. Transparent communication with students, physicians, and patients is essential to ensure their understanding of the technology's purpose and implications, as well as the potential risks and benefits. Maintaining a balance between personalized learning and face-to-face interactions is crucial to avoid hindering critical thinking and communication skills. Despite challenges, ChatGPT offers transformative opportunities. Integrating it with problem-based learning, team-based learning, and case-based learning methodologies can further enhance medical education. With proper regulation and supervision, ChatGPT can contribute to a well-rounded learning environment, nurturing skilled and knowledgeable medical professionals ready to tackle health care challenges. By emphasizing ethical considerations and human-centric approaches, ChatGPT's potential can be fully harnessed in medical education, benefiting both students and patients alike.

An Automated System for Physician Trainee Procedure Logging via Electronic Health Records

Importance

Procedural proficiency is a core competency for graduate medical education; however, procedural reporting often relies on manual workflows that are duplicative and generate data whose validity and accuracy are difficult to assess. Failure to accurately gather these data can impede learner progression, delay procedures, and negatively impact patient safety.

Objective

To examine accuracy and procedure logging completeness of a system that extracts procedural data from an electronic health record system and uploads these data securely to an application used by many residency programs for accreditation.

Design, Setting, and Participants

This quality improvement study of all emergency medicine resident physicians at University of California, San Diego Health was performed from May 23, 2023, to June 25, 2023.

Exposures

Automated system for procedure data extraction and upload to a residency management software application.

Main Outcomes and Measures

The number of procedures captured by the automated system when running silently compared with manually logged procedures in the same timeframe, as well as accuracy of the data upload.

Results

Forty-seven residents participated in the initial silent assessment of the extraction component of the system. During a 1-year period (May 23, 2022, to May 7, 2023), 4291 procedures were manually logged by residents, compared with 7617 procedures captured by the automated system during the same period, representing a 78% increase. During assessment of the upload component of the system (May 8, 2023, to June 25, 2023), a total of 1353 procedures and patient encounters were evaluated, with the system operating with a sensitivity of 97.4%, specificity of 100%, and overall accuracy of 99.5%.

Conclusions and Relevance

In this quality improvement study of emergency medicine resident physicians, an automated system demonstrated that reliance on self-reported procedure logging resulted in significant procedural underreporting compared with the use of data obtained at the point of performance. Additionally, this system afforded a degree of reliability and validity heretofore absent from the usual after-the-fact procedure logging workflows while using a novel application programming interface-based approach. To our knowledge, this system constitutes the first generalizable implementation of an automated solution to a problem that has existed in graduate medical education for decades.

Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence

The proliferation of generative artificial intelligence (AI) and its extensive potential for integration into many aspects of health care signal a transformational shift within the health care environment. In this context, medical education must evolve to ensure that medical trainees are adequately prepared to navigate the rapidly changing health care landscape. Medical education has moved toward a competency-based education paradigm, leading the Association of American Medical Colleges (AAMC) to define a set of Entrustable Professional Activities (EPAs) as its practical operational framework in undergraduate medical education. The AAMC's 13 core EPAs for entering residencies have been implemented with varying levels of success across medical schools. In this paper, we critically assess the existing core EPAs in the context of rapid AI integration in medicine. We identify EPAs that require refinement, redefinition, or comprehensive change to align with the emerging trends in health care. Moreover, this perspective proposes a set of "emerging" EPAs, informed by the changing landscape and capabilities presented by generative AI technologies. We provide a practical evaluation of the EPAs, alongside actionable recommendations on how medical education, viewed through the lens of the AAMC EPAs, can adapt and remain relevant amid rapid technological advancements. By leveraging the transformative potential of AI, we can reshape medical education to align with an AI-integrated future of medicine. This approach will help equip future health care professionals with technological competence and adaptive skills to meet the dynamic and evolving demands in health care.