Educating the next generation of radiologists: a comparative report of ChatGPT and e-learning resources

Teaching methods, facilities, and institutions in student ultrasound education (SUSE): e-learning, simulation, and ultrasound skills labs

To acquire ultrasound skills, students need access to educational resources for both theoretical and practical knowledge. Effective training depends on the availability of educational content, training opportunities, and facilities - all of which are often scarce. E-learning platforms, simulation, and ultrasound skills labs are potential solutions to complement supervised real-life bedside training on patients and improve ultrasound education. This review discusses the advantages and disadvantages of e-learning, simulation, and ultrasound skills labs in the specific context of student education. E-learning platforms and teaching videos support students by offering flexible, accessible learning, allowing them to engage with material at their own pace. These digital resources complement practical lessons by providing essential theoretical knowledge that can be applied during hands-on sessions. Simulation creates a controlled environment for skill development and enhances patient safety, especially during interventional procedures. However, simulation equipment's high cost and technical complexity strain budgets and require specialized staff and training. Simulators often fail to replicate real-life variability, limiting skill transfer to patient care. The establishment of ultrasound skills labs offers a solid, long-term opportunity for skill retention but requires sufficient and sustainable funding. In conclusion, e-learning, simulation, and ultrasound skills labs can be valuable components of student ultrasound education if used deliberately. They should be included in a blended medical curriculum incorporating real-world clinical experiences to ensure effective transfer of learning to clinical practice.

Large language models in methodological quality evaluation of radiomics research based on METRICS: ChatGPT vs NotebookLM vs radiologist

OBJECTIVES

This study aimed to evaluate the effectiveness of large language models (LLM) in assessing the methodological quality of radiomics research, using METhodological RadiomICs Score (METRICS) tool.

METHODS

This study included open access radiomic research articles published in 2024 across various journals and a preprint repository, all under the Creative Commons Attribution License. Each study was independently evaluated using METRICS by two LLMs, ChatGPT-4 and NotebookLM, and a consensus assessment performed by two radiologists with expertise in radiomics research.

RESULTS

A total of 48 open access articles were included in this study. ChatGPT-4, NotebookLM, and human readers achieved median scores of 79.5 %, 61.6 %, and 69.0 %, respectively, with a statistically significant difference across these evaluations (p < 0.05). Pairwise comparisons indicated no statistically significant difference for NotebookLM vs human experts (p > 0.05), in contrast to other pairs (p < 0.05). Intraclass correlation coefficient (ICC) for ChatGPT-4 and human experts was 0.563 (95 % CI: 0.050---0.795), corresponding to poor to good agreement. The ICC for ChatGPT-4 and NotebookLM and for human experts and NotebookLM were 0.391 (95 % CI: -0.031---0.665) and 0.555 (95 % CI: 0.326---0.723), respectively, indicating poor to moderate agreement. LLMs completed the tasks in a significantly shorter time (p < 0.05). In item-wise reliability analysis, ChatGPT-4 generally demonstrated higher consistency than NotebookLM.

CONCLUSION

LLMs hold promise for automatically evaluating the quality of radiomics research using METRICS, a new tool that is relatively more complex yet comprehensive compared to its counterparts. However, substantial improvements are needed for full alignment with human experts.

Foundation Models in Radiology: What, How, Why, and Why Not

Recent advances in artificial intelligence have witnessed the emergence of large-scale deep learning models capable of interpreting and generating both textual and imaging data. Such models, typically referred to as foundation models (FMs), are trained on extensive corpora of unlabeled data and demonstrate high performance across various tasks. FMs have recently received extensive attention from academic, industry, and regulatory bodies. Given the potentially transformative impact that FMs can have on the field of radiology, radiologists must be aware of potential pathways to train these radiology-specific FMs, including understanding both the benefits and challenges. Thus, this review aims to explain the fundamental concepts and terms of FMs in radiology, with a specific focus on the requirements of training data, model training paradigms, model capabilities, and evaluation strategies. Overall, the goal of this review is to unify technical advances and clinical needs for safe and responsible training of FMs in radiology to ultimately benefit patients, providers, and radiologists.

Generative artificial intelligence in graduate medical education

Generative artificial intelligence (GenAI) is rapidly transforming various sectors, including healthcare and education. This paper explores the potential opportunities and risks of GenAI in graduate medical education (GME). We review the existing literature and provide commentary on how GenAI could impact GME, including five key areas of opportunity: electronic health record (EHR) workload reduction, clinical simulation, individualized education, research and analytics support, and clinical decision support. We then discuss significant risks, including inaccuracy and overreliance on AI-generated content, challenges to authenticity and academic integrity, potential biases in AI outputs, and privacy concerns. As GenAI technology matures, it will likely come to have an important role in the future of GME, but its integration should be guided by a thorough understanding of both its benefits and limitations.

ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review

ChatGPT's role in creating multiple-choice questions (MCQs) is growing but the validity of these artificial-intelligence-generated questions is unclear. This literature review was conducted to address the urgent need for understanding the application of ChatGPT in generating MCQs for medical education. Following the database search and screening of 1920 studies, we found 23 relevant studies. We extracted the prompts for MCQ generation and assessed the validity evidence of MCQs. The findings showed that prompts varied, including referencing specific exam styles and adopting specific personas, which align with recommended prompt engineering tactics. The validity evidence covered various domains, showing mixed accuracy rates, with some studies indicating comparable quality to human-written questions, and others highlighting differences in difficulty and discrimination levels, alongside a significant reduction in question creation time. Despite its efficiency, we highlight the necessity of careful review and suggest a need for further research to optimize the use of ChatGPT in question generation. Main messages  Ensure high-quality outputs by utilizing well-designed prompts; medical educators should prioritize the use of detailed, clear ChatGPT prompts when generating MCQs. Avoid using ChatGPT-generated MCQs directly in examinations without thorough review to prevent inaccuracies and ensure relevance. Leverage ChatGPT's potential to streamline the test development process, enhancing efficiency without compromising quality.

Expert assessment of ChatGPT's ability to generate illness scripts: an evaluative study

BACKGROUND

An illness script is a specific script format geared to represent patient-oriented clinical knowledge organized around enabling conditions, faults (i.e., pathophysiological process), and consequences. Generative artificial intelligence (AI) stands out as an educational aid in continuing medical education. The effortless creation of a typical illness script by generative AI could help the comprehension of key features of diseases and increase diagnostic accuracy. No systematic summary of specific examples of illness scripts has been reported since illness scripts are unique to each physician.

OBJECTIVE

This study investigated whether generative AI can generate illness scripts.

METHODS

We utilized ChatGPT-4, a generative AI, to create illness scripts for 184 diseases based on the diseases and conditions integral to the National Model Core Curriculum in Japan for undergraduate medical education (2022 revised edition) and primary care specialist training in Japan. Three physicians applied a three-tier grading scale: "A" denotes that the content of each disease's illness script proves sufficient for training medical students, "B" denotes that it is partially lacking but acceptable, and "C" denotes that it is deficient in multiple respects.

RESULTS

By leveraging ChatGPT-4, we successfully generated each component of the illness script for 184 diseases without any omission. The illness scripts received "A," "B," and "C" ratings of 56.0% (103/184), 28.3% (52/184), and 15.8% (29/184), respectively.

CONCLUSION

Useful illness scripts were seamlessly and instantaneously created using ChatGPT-4 by employing prompts appropriate for medical students. The technology-driven illness script is a valuable tool for introducing medical students to key features of diseases.

Harnessing ChatGPT dialogues to address claustrophobia in MRI - A radiographers' education perspective

INTRODUCTION

The healthcare sector invests significantly in communication skills training, but not always with satisfactory results. Recently, generative Large Language Models, have shown promising results in medical education. This study aims to use ChatGPT to simulate radiographer-patient conversations about the critical moment of claustrophobia management during MRI, exploring how Artificial Intelligence can improve radiographers' communication skills.

METHODS

This study exploits specifically designed prompts on ChatGPT-3.5 and ChatGPT-4 to generate simulated conversations between virtual claustrophobic patients and six radiographers with varying levels of work experience focusing on their differences in model size and language generation capabilities. Success rates and responses were analysed. The methods of radiographers in convincing virtual patients to undergo MRI despite claustrophobia were also evaluated.

RESULTS

A total of 60 simulations were conducted, achieving a success rate of 96.7% (58/60). ChatGPT-3.5 exhibited errors in 40% (12/30) of the simulations, while ChatGPT-4 showed no errors. In terms of radiographers' communication during the simulations, out of 164 responses, 70.2% (115/164) were categorized as "Supportive Instructions," followed by "Music Therapy" at 18.3% (30/164). Experts mainly used "Supportive Instructions" (82.2%, 51/62) and "Breathing Techniques" (9.7%, 6/62). Intermediate participants favoured "Music Therapy" (26%, 13/50), while Beginner participants frequently utilized "Mild Sedation" (15.4%, 8/52).

CONCLUSION

The simulation of clinical scenarios via ChatGPT proves valuable in assessing and testing radiographers' communication skills, especially in managing claustrophobic patients during MRI. This pilot study highlights the potential of ChatGPT in preclinical training, recognizing different training needs at different levels of professional experience.

IMPLICATIONS FOR PRACTICE

This study is relevant in radiography practice, where AI is increasingly widespread, as it explores a new way to improve the training of radiographers.