Heterogeneity and predictors of the effects of AI assistance on radiologists

Micro-Nano Convergence-Driven Radiotheranostic Revolution in Hepatocellular Carcinoma

Radiotherapy, as an important means of treating hepatocellular carcinoma (HCC), has shown unique therapeutic advantages, especially in patients who are unable to undergo surgery or transplantation. It mainly includes external radiotherapy, transarterial radioembolization and intratumoral radioactive particle implantation. However, under the influence of factors such as the hypoxic characteristics of the liver tumor microenvironment and the radioresistance of tumor cells, the effect of radiotherapy may be unstable and may cause side effects, affecting the quality of life of patients. In recent years, with the development of nanotechnology, drug delivery systems based on micro-nanomaterials have provided new solutions for improving the effect of radiotherapy for HCC. Despite this, the application of micro-nano drug delivery systems in the treatment of HCC still faces some challenges, mainly including the in vivo safety and in vivo metabolism of micro-nano materials. This article reviews the latest progress of micro-nano materials in the treatment of HCC, especially their application in radiosensitization and their clinical translation potential. This article systematically analyzes the role of micro-nanomaterials in external or internal radiotherapy sensitization and radioimmunotherapy and explores the advantages of micro-nanomaterials in improving the treatment effect of HCC.

Role and Potential of Artificial Intelligence in Biomarker Discovery and Development of Treatment Strategies for Amyotrophic Lateral Sclerosis

Neurodegenerative diseases, including amyotrophic lateral sclerosis (ALS), present significant challenges owing to their complex pathologies and a lack of curative treatments. Early detection and reliable biomarkers are critical but remain elusive. Artificial intelligence (AI) has emerged as a transformative tool, enabling advancements in biomarker discovery, diagnostic accuracy, and therapeutic development. From optimizing clinical-trial designs to leveraging omics and neuroimaging data, AI facilitates understanding of disease and treatment innovation. Notably, technologies such as AlphaFold and deep learning models have revolutionized proteomics and neuroimaging, offering unprecedented insights into ALS pathophysiology. This review highlights the intersection of AI and ALS, exploring the current state of progress and future therapeutic prospects.

Are we sleepwalking into a fully automated medical imaging service?

INTRODUCTION

Artificial intelligence (AI) is already embedded in medical imaging services, but now that the National Institute for Health and Care Excellence (NICE) has released position statements looking favourably on AI use in healthcare, its use will embed even further.

DISCUSSION

AI has brought many positives to medical imaging services and is far superior at making calculations using vast amounts of data. It can therefore help improve the speed and accuracy of diagnosis and treatment plans for many patients, but at what cost to the radiography profession? Surveys have shown that the majority of the workforce welcome AI, but admit that they don't fully understand the principles behind it. AI developers are keen to improve patient output, and many are unconcerned about the possible negative effects on staff morale and expertise. As computers remove the autonomy and competency that radiographers have previously held with pride, staff may find that they become de-skilled and de-motivated, and it may eventually subsume the traditional role of the radiographer altogether. The profession needs to be aware of these potential impacts and prepare accordingly.

CONCLUSION

Higher education plays an important role in preparing radiographers of the future for the changing landscape of medical imaging and should include more engineering and data science modules in the curriculum to prevent radiographers from becoming irrelevant.

A promptable CT foundation model for solid tumor evaluation

Carcinogenesis is inherently complex, resulting in heterogeneous tumors with variable outcomes and frequent metastatic potential. Conventional longitudinal evaluation methods like RECIST 1.1 remain labor-intensive and prone to measurement errors, while existing AI solutions face critical limitations due to tumor heterogeneity, insufficient annotations, and lack of user interaction. We developed ONCOPILOT, an interactive CT-based foundation model dedicated to 3D tumor segmentation, significantly refining RECIST 1.1 evaluations with active radiologist engagement. Trained on more than 8000 CT scans, ONCOPILOT employs intuitive visual prompts, including point-click, bounding boxes, and edit-points. It attains segmentation accuracy that matches or exceeds state-of-the-art methods, provides radiologist-level precision for RECIST 1.1 measurements, reduces inter-observer variability, and enhances workflow efficiency. Integrating clinical expertise with interactive AI capabilities, ONCOPILOT facilitates widespread access to advanced biomarkers, notably volumetric tumor analyses, thereby supporting improved clinical decision-making, patient stratification, and accelerating advancements in oncology research.

Generative Artificial Intelligence in Clinical Medicine and Impact on Gastroenterology

The pace of artificial intelligence (AI) integration into health care has accelerated with rapid advances in generative AI (genAI). Gastroenterology and hepatology in particular will be transformed due to the multimodal workflows that integrate endoscopic video, radiologic imaging, tabular data, and unstructured note text. GenAI will impact the entire spectrum of clinical experience, from administrative tasks, diagnostic guidance, and treatment recommendations. Unlike traditional machine learning approaches, genAI is more flexible, with one platform able to be used across multiple tasks. Initial evidence suggests benefits in lower-level administrative tasks, such as clinical documentation, medical billing, and scheduling; and information tasks, such as patient education and summarization of the medical literature. No evidence exists for genAI solutions for more complex tasks relevant to clinical care, such as clinical reasoning for diagnostic and treatment decisions that may affect patient outcomes. Challenges of output reliability, data privacy, and useful integration remain; potential solutions include robust validation, regulatory oversight, and "human-AI teaming" strategies to ensure safe, effective deployment. We remain optimistic in the potential of genAI to augment clinical expertise due to the adaptability of genAI to handle multiple data modalities to obtain and focus relevant information flows and the human-friendly interfaces that facilitate ease of use. We believe that the potential of genAI for dynamic human-algorithmic interactions may allow for a degree of clinician-directed customization to enhance human presence.

A concept-based interpretable model for the diagnosis of choroid neoplasias using multimodal data

Diagnosing rare diseases remains a critical challenge in clinical practice, often requiring specialist expertise. Despite the promising potential of machine learning, the scarcity of data on rare diseases and the need for interpretable, reliable artificial intelligence (AI) models complicates development. This study introduces a multimodal concept-based interpretable model tailored to distinguish uveal melanoma (0.4-0.6 per million in Asians) from hemangioma and metastatic carcinoma following the clinical practice. We collected a comprehensive dataset on Asians to date on choroid neoplasm imaging with radiological reports, encompassing over 750 patients from 2013 to 2019. Our model integrates domain expert insights from radiological reports and differentiates between three types of choroidal tumors, achieving an F1 score of 0.91. This performance not only matches senior ophthalmologists but also improves the diagnostic accuracy of less experienced clinicians by 42%. The results underscore the potential of interpretable AI to enhance rare disease diagnosis and pave the way for future advancements in medical AI.

Artificial Intelligence Models Accuracy for Odontogenic Keratocyst Detection From Panoramic View Radiographs: A Systematic Review and Meta-Analysis

Background and Aims

Odontogenic keratocyst (OKC) is a radiolucent jaw lesion often mistaken for similar conditions like ameloblastomas on panoramic radiographs. Accurate diagnosis is vital for effective management, but manual image interpretation can be inconsistent. While deep learning algorithms in AI have shown promise in improving diagnostic accuracy for OKCs, their performance across studies is still unclear. This systematic review and meta-analysis aimed to evaluate the diagnostic accuracy of AI models in detecting OKC from panoramic radiographs.

Methods

A systematic search was performed across 5 databases. Studies were included if they examined the PICO question of whether AI models (I) could improve the diagnostic accuracy (O) of OKC in panoramic radiographs (P) compared to reference standards (C). Key performance metrics including sensitivity, specificity, accuracy, and area under the curve (AUC) were extracted and pooled using random-effects models. Meta-regression and subgroup analyses were conducted to identify sources of heterogeneity. Publication bias was evaluated through funnel plots and Egger's test.

Results

Eight studies were included in the meta-analysis. The pooled sensitivity across all studies was 83.66% (95% CI:73.75%-93.57%) and specificity was 82.89% (95% CI:70.31%-95.47%). YOLO-based models demonstrated superior diagnostic performance with a sensitivity of 96.4% and specificity of 96.0%, compared to other architectures. Meta-regression analysis indicated that model architecture was a significant predictor of diagnostic performance, accounting for a significant portion of the observed heterogeneity. However, the analysis also revealed publication bias and high variability across studies (Egger's test, p = 0.042).

Conclusion

AI models, particularly YOLO-based architectures, can improve the diagnostic accuracy of OKCs in panoramic radiographs. While AI shows strong capabilities in simple cases, it should complement, not replace, human expertise, especially in complex situations.

Using deep learning to enhance reporting efficiency and accuracy in degenerative cervical spine MRI

BACKGROUND CONTEXT

Cervical spine MRI is essential for evaluating degenerative cervical spondylosis (DCS) but is time-consuming to report and subject to interobserver variability. The integration of artificial intelligence in medical imaging offers potential solutions to enhance productivity and diagnostic consistency.

PURPOSE

To assess whether a transformer-based deep learning model (DLM) can improve the efficiency and accuracy of radiologists in reporting DCS MRIs.

STUDY DESIGN/SETTING

Retrospective study using external DCS MRIs from December 2015 to August 2018.

PATIENT SAMPLE

The test dataset comprised 50 preoperative DCS MRIs (2,555 images) from 50 patients (mean age = 60 years ± SD 14; 13 women [26%]), excluding cases with instrumentation.

OUTCOME MEASURES

Primary outcomes were interpretation time and interobserver agreement (Gwet's kappa) among radiologists grading spinal canal and neural foramina stenosis with and without DLM-assistance.

METHODS

A transformer-based DLM was used to classify spinal canal (grades 0/1/2/3) and neural foramina (grades 0/1/2) stenosis at each disc level. Two experienced musculoskeletal radiologists (both with 12-years-of-experience) provided reference standard labels in consensus. Ten radiologists (0-7 years of experience) graded DCS MRIs with and without DLM-assistance, with a 1-month washout period between sessions to minimize recall bias. Interpretation time and interobserver agreement were assessed.

RESULTS

DLM-assistance significantly improved interpretation time by 69 to 308 s (p<.001), reducing mean time from 159-490 s (SD 27-649) to 90-182 s (SD 42-218). Radiology residents experienced the largest time savings. DLM-assistance improved interobserver agreement across all stenosis gradings compared to baseline. For dichotomous spinal canal grading, residents had the largest improvement in agreement (κ = 0.63 to 0.77, p<.001). Conversely, for dichotomous neural foramina grading, musculoskeletal radiologists had the largest improvement (κ=0.60 to 0.72, p<.001). Notably, independent DLM performance alone was equivalent or superior to all readers.

CONCLUSIONS

The integration of a deep learning model into the radiological assessment of DCS MRI improved radiologists' interpretation time and interobserver agreement, regardless of experience level.

The artificial intelligence revolution in gastric cancer management: clinical applications

Nowadays, gastric cancer has become a significant issue in the global cancer burden, and its impact cannot be ignored. The rapid development of artificial intelligence technology is attempting to address this situation, aiming to change the clinical management landscape of gastric cancer fundamentally. In this transformative change, machine learning and deep learning, as two core technologies, play a pivotal role, bringing unprecedented innovations and breakthroughs in the diagnosis, treatment, and prognosis evaluation of gastric cancer. This article comprehensively reviews the latest research status and application of artificial intelligence algorithms in gastric cancer, covering multiple dimensions such as image recognition, pathological analysis, personalized treatment, and prognosis risk assessment. These applications not only significantly improve the sensitivity of gastric cancer risk monitoring, the accuracy of diagnosis, and the precision of survival prognosis but also provide robust data support and a scientific basis for clinical decision-making. The integration of artificial intelligence, from optimizing the diagnosis process and enhancing diagnostic efficiency to promoting the practice of precision medicine, demonstrates its promising prospects for reshaping the treatment model of gastric cancer. Although most of the current AI-based models have not been widely used in clinical practice, with the continuous deepening and expansion of precision medicine, we have reason to believe that a new era of AI-driven gastric cancer care is approaching.

Examining human-AI interaction in real-world healthcare beyond the laboratory

Artificial Intelligence (AI) is revolutionizing healthcare, but its true impact depends on seamless human interaction. While most research focuses on technical metrics, we lack frameworks to measure the compatibility or synergy of real-world human-AI interactions in healthcare settings. We propose a multimodal toolkit combining ecological momentary assessment, quantitative observations, and baseline measurements to optimize AI implementation.

Artificial Intelligence in Relation to Accurate Information and Tasks in Gynecologic Oncology and Clinical Medicine-Dunning-Kruger Effects and Ultracrepidarianism

Publications on the application of artificial intelligence (AI) to many situations, including those in clinical medicine, created in 2023-2024 are reviewed here. Because of the short time frame covered, here, it is not possible to conduct exhaustive analysis as would be the case in meta-analyses or systematic reviews. Consequently, this literature review presents an examination of narrative AI's application in relation to contemporary topics related to clinical medicine. The landscape of the findings reviewed here span 254 papers published in 2024 topically reporting on AI in medicine, of which 83 articles are considered in the present review because they contain evidence-based findings. In particular, the types of cases considered deal with AI accuracy in initial differential diagnoses, cancer treatment recommendations, board-style exams, and performance in various clinical tasks, including clinical imaging. Importantly, summaries of the validation techniques used to evaluate AI findings are presented. This review focuses on AIs that have a clinical relevancy evidenced by application and evaluation in clinical publications. This relevancy speaks to both what has been promised and what has been delivered by various AI systems. Readers will be able to understand when generative AI may be expressing views without having the necessary information (ultracrepidarianism) or is responding as if the generative AI had expert knowledge when it does not. A lack of awareness that AIs may deliver inadequate or confabulated information can result in incorrect medical decisions and inappropriate clinical applications (Dunning-Kruger effect). As a result, in certain cases, a generative AI system might underperform and provide results which greatly overestimate any medical or clinical validity.

On the practical, ethical, and legal necessity of clinical Artificial Intelligence explainability: an examination of key arguments

The necessity for explainability of artificial intelligence technologies in medical applications has been widely discussed and heavily debated within the literature. This paper comprises a systematized review of the arguments supporting and opposing this purported necessity. Both sides of the debate within the literature are quoted to synthesize discourse on common recurring themes and subsequently critically analyze and respond to it. While the use of autonomous black box algorithms is compellingly discouraged, the same cannot be said for the whole of medical artificial intelligence technologies that lack explainability. We contribute novel comparisons of unexplainable clinical artificial intelligence tools, diagnosis of idiopathy, and diagnoses by exclusion, to analyze implications on patient autonomy and informed consent. Applying a novel approach using comparisons with clinical practice guidelines, we contest the claim that lack of explainability compromises clinician due diligence and undermines epistemological responsibility. We find it problematic that many arguments in favour of the practical, ethical, or legal necessity of clinical artificial intelligence explainability conflate the use of unexplainable AI with automated decision making, or equate the use of clinical artificial intelligence with the exclusive use of clinical artificial intelligence.

Let us talk about mistakes

Unfortunately, errors and mistakes are part of life. Errors and mistakes can harm patients and incur unplanned costs. Errors may arise from various sources, which may be classified as systematic, latent, or active. Intrinsic and extrinsic factors also contribute to incorrect decisions. In addition to cognitive biases, our personality, socialization, personal chronobiology, and way of thinking (heuristic versus analytical) are influencing factors. Factors such as overload from private situations, long commuting times, and the complex environment of information technology must also be considered. The objective of this paper is to define and classify errors and mistakes in radiology, to discuss the influencing factors, and to present strategies for prevention. Hierarchical responsibilities and team "well-being" are also discussed.

Artificial intelligence for direct-to-physician reporting of ambulatory electrocardiography

Developments in ambulatory electrocardiogram (ECG) technology have led to vast amounts of ECG data that currently need to be interpreted by human technicians. Here we tested an artificial intelligence (AI) algorithm for direct-to-physician reporting of ambulatory ECGs. Beat-by-beat annotation of 14,606 individual ambulatory ECG recordings (mean duration = 14 ± 10 days) was performed by certified ECG technicians (n = 167) and an ensemble AI model, called DeepRhythmAI. To compare the performance of the AI model and the technicians, a random sample of 5,235 rhythm events identified by the AI model or by technicians, of which 2,236 events were identified as critical arrhythmias, was selected for annotation by one of 17 cardiologist consensus panels. The mean sensitivity of the AI model for the identification of critical arrhythmias was 98.6% (95% confidence interval (CI) = 97.7-99.4), as compared to 80.3% (95% CI = 77.3-83.3%) for the technicians. False-negative findings were observed in 3.2/1,000 patients for the AI model versus 44.3/1,000 patients for the technicians. Accordingly, the relative risk of a missed diagnosis was 14.1 (95% CI = 10.4-19.0) times higher for the technicians. However, a higher false-positive event rate was observed for the AI model (12 (interquartile range (IQR) = 6-74)/1,000 patient days) as compared to the technicians (5 (IQR = 2-153)/1,000 patient days). We conclude that the DeepRhythmAI model has excellent negative predictive value for critical arrhythmias, substantially reducing false-negative findings, but at a modest cost of increased false-positive findings. AI-only analysis to facilitate direct-to-physician reporting could potentially reduce costs and improve access to care and outcomes in patients who need ambulatory ECG monitoring.

An Assessment of Deep Learning's Impact on General Dentists' Ability to Detect Alveolar Bone Loss in 2D Intraoral Radiographs

Background/Objective: Deep learning (DL) technology has shown potential in enhancing diagnostic accuracy in dentomaxillofacial radiology, particularly for detecting carious lesions, apical lesions, and periodontal bone loss. However, its effect on general dentists' ability to detect radiographic bone loss (RBL) in clinical practice remains unclear. This study investigates the impact of the Denti.AI DL technology on general dentists' ability to identify bone loss in intraoral radiographs, addressing this gap in the literature. Methods: Ten dentists from the university's dental clinics independently assessed 26 intraoral radiographs (periapical and bitewing) for bone loss using a Likert scale probability index with and without DL assistance. The participants viewed images on identical monitors with controlled lighting. This study generated 3940 data points for analysis. The statistical analyses included receiver operating characteristic (ROC) curves, area under the curve (AUC), and ANOVA tests. Results: Most dentists showed minor improvement in detecting bone loss on periapical radiographs when using DL. For bitewing radiographs, only a few dentists showed minor improvement. Overall, the difference in diagnostic accuracy between evaluations with and without DL was minimal (0.008). The differences in AUC for periapical and bitewing radiographs were 0.031 and -0.009, respectively, and were not statistically significant. Conclusions: This study found no statistically significant improvement in experienced dentists' diagnostic accuracy for detecting bone loss in intraoral radiographs when using Denti.AI deep learning technology.

CORE-MD clinical risk score for regulatory evaluation of artificial intelligence-based medical device software

The European CORE-MD consortium (Coordinating Research and Evidence for Medical Devices) proposes a score for medical devices incorporating artificial intelligence or machine learning algorithms. Its domains are summarised as valid clinical association, technical performance, and clinical performance. High scores indicate that extensive clinical investigations should be undertaken before regulatory approval, whereas lower scores indicate devices for which less pre-market clinical evaluation may be balanced by more post-market evidence.

Upskilling or deskilling? Measurable role of an AI-supported training for radiology residents: a lesson from the pandemic

OBJECTIVES

This article aims to evaluate the use and effects of an artificial intelligence system supporting a critical diagnostic task during radiology resident training, addressing a research gap in this field.

MATERIALS AND METHODS

We involved eight residents evaluating 150 CXRs in three scenarios: no AI, on-demand AI, and integrated-AI. The considered task was the assessment of a multi-regional severity score of lung compromise in patients affected by COVID-19. The chosen artificial intelligence tool, fully integrated in the RIS/PACS, demonstrated superior performance in scoring compared to the average radiologist. Using quantitative metrics and questionnaires, we measured the 'upskilling' effects of using AI support and residents' resilience to 'deskilling,' i.e., their ability to overcome AI errors.

RESULTS

Residents required AI in 70% of cases when left free to choose. AI support significantly reduced severity score errors and increased inter-rater agreement by 22%. Residents were resilient to AI errors above an acceptability threshold. Questionnaires indicated high tool usefulness, reliability, and explainability, with a preference for collaborative AI scenarios.

CONCLUSION

With this work, we gathered quantitative and qualitative evidence of the beneficial use of a high-performance AI tool that is well integrated into the diagnostic workflow as a training aid for radiology residents.

CRITICAL RELEVANCE STATEMENT

Balancing educational benefits and deskilling risks is essential to exploit AI systems as effective learning tools in radiology residency programs. Our work highlights metrics for evaluating these aspects.

KEY POINTS

Insights into AI tools' effects in radiology resident training are lacking. Metrics were defined to observe residents using an AI tool in different settings. This approach is advisable for evaluating AI tools in radiology training.

Clinical validation of explainable AI for fetal growth scans through multi-level, cross-institutional prospective end-user evaluation

We aimed to develop and evaluate Explainable Artificial Intelligence (XAI) for fetal ultrasound using actionable concepts as feedback to end-users, using a prospective cross-center, multi-level approach. We developed, implemented, and tested a deep-learning model for fetal growth scans using both retrospective and prospective data. We used a modified Progressive Concept Bottleneck Model with pre-established clinical concepts as explanations (feedback on image optimization and presence of anatomical landmarks) as well as segmentations (outlining anatomical landmarks). The model was evaluated prospectively by assessing the following: the model's ability to assess standard plane quality, the correctness of explanations, the clinical usefulness of explanations, and the model's ability to discriminate between different levels of expertise among clinicians. We used 9352 annotated images for model development and 100 videos for prospective evaluation. Overall classification accuracy was 96.3%. The model's performance in assessing standard plane quality was on par with that of clinicians. Agreement between model segmentations and explanations provided by expert clinicians was found in 83.3% and 74.2% of cases, respectively. A panel of clinicians evaluated segmentations as useful in 72.4% of cases and explanations as useful in 75.0% of cases. Finally, the model reliably discriminated between the performances of clinicians with different levels of experience (p- values < 0.01 for all measures) Our study has successfully developed an Explainable AI model for real-time feedback to clinicians performing fetal growth scans. This work contributes to the existing literature by addressing the gap in the clinical validation of Explainable AI models within fetal medicine, emphasizing the importance of multi-level, cross-institutional, and prospective evaluation with clinician end-users. The prospective clinical validation uncovered challenges and opportunities that could not have been anticipated if we had only focused on retrospective development and validation, such as leveraging AI to gauge operator competence in fetal ultrasound.

Current state and promise of user-centered design to harness explainable AI in clinical decision-support systems for patients with CNS tumors

In neuro-oncology, MR imaging is crucial for obtaining detailed brain images to identify neoplasms, plan treatment, guide surgical intervention, and monitor the tumor's response. Recent AI advances in neuroimaging have promising applications in neuro-oncology, including guiding clinical decisions and improving patient management. However, the lack of clarity on how AI arrives at predictions has hindered its clinical translation. Explainable AI (XAI) methods aim to improve trustworthiness and informativeness, but their success depends on considering end-users' (clinicians') specific context and preferences. User-Centered Design (UCD) prioritizes user needs in an iterative design process, involving users throughout, providing an opportunity to design XAI systems tailored to clinical neuro-oncology. This review focuses on the intersection of MR imaging interpretation for neuro-oncology patient management, explainable AI for clinical decision support, and user-centered design. We provide a resource that organizes the necessary concepts, including design and evaluation, clinical translation, user experience and efficiency enhancement, and AI for improved clinical outcomes in neuro-oncology patient management. We discuss the importance of multi-disciplinary skills and user-centered design in creating successful neuro-oncology AI systems. We also discuss how explainable AI tools, embedded in a human-centered decision-making process and different from fully automated solutions, can potentially enhance clinician performance. Following UCD principles to build trust, minimize errors and bias, and create adaptable software has the promise of meeting the needs and expectations of healthcare professionals.

Controversies in Artificial Intelligence in Neurosurgery

Artificial intelligence (AI) has evolved from science fiction to a technology infiltrating everyday life. In neurosurgery, clinicians and researchers are exploring ways to implement this powerful tool to improve the safety and efficiency of the perioperative process. Current applications include preoperative diagnosis, intraoperative detection and recommendations, and technical skills assessment and feedback. Although the potential benefits are evident, AI integration into neurosurgical workflows requires discussions around ethical regulations, cybersecurity, privacy concerns, and data and algorithm ownership.

Generalizability, robustness, and correction bias of segmentations of thoracic organs at risk in CT images

OBJECTIVE

This study aims to assess and compare two state-of-the-art deep learning approaches for segmenting four thoracic organs at risk (OAR)-the esophagus, trachea, heart, and aorta-in CT images in the context of radiotherapy planning.

MATERIALS AND METHODS

We compare a multi-organ segmentation approach and the fusion of multiple single-organ models, each dedicated to one OAR. All were trained using nnU-Net with the default parameters and the full-resolution configuration. We evaluate their robustness with adversarial perturbations, and their generalizability on external datasets, and explore potential biases introduced by expert corrections compared to fully manual delineations.

RESULTS

The two approaches show excellent performance with an average Dice score of 0.928 for the multi-class setting and 0.930 when fusing the four single-organ models. The evaluation of external datasets and common procedural adversarial noise demonstrates the good generalizability of these models. In addition, expert corrections of both models show significant bias to the original automated segmentation. The average Dice score between the two corrections is 0.93, ranging from 0.88 for the trachea to 0.98 for the heart.

CONCLUSION

Both approaches demonstrate excellent performance and generalizability in segmenting four thoracic OARs, potentially improving efficiency in radiotherapy planning. However, the multi-organ setting proves advantageous for its efficiency, requiring less training time and fewer resources, making it a preferable choice for this task. Moreover, corrections of AI segmentation by clinicians may lead to biases in the results of AI approaches. A test set, manually annotated, should be used to assess the performance of such methods.

KEY POINTS

Question While manual delineation of thoracic organs at risk is labor-intensive, prone to errors, and time-consuming, evaluation of AI models performing this task lacks robustness. Findings The deep-learning model using the nnU-Net framework showed excellent performance, generalizability, and robustness in segmenting thoracic organs in CT, enhancing radiotherapy planning efficiency. Clinical relevance Automatic segmentation of thoracic organs at risk can save clinicians time without compromising the quality of the delineations, and extensive evaluation across diverse settings demonstrates the potential of integrating such models into clinical practice.

Artificial Intelligence Efficacy as a Function of Trainee Interpreter Proficiency: Lessons from a Randomized Controlled Trial

BACKGROUND AND PURPOSE

Recently, artificial intelligence tools have been deployed with increasing speed in educational and clinical settings. However, the use of artificial intelligence by trainees across different levels of experience has not been well-studied. This study investigates the impact of artificial intelligence assistance on the diagnostic accuracy for intracranial hemorrhage and large-vessel occlusion by medical students and resident trainees.

MATERIALS AND METHODS

This prospective study was conducted between March 2023 and October 2023. Medical students and resident trainees were asked to identify intracranial hemorrhage and large-vessel occlusion in 100 noncontrast head CTs and 100 head CTAs, respectively. One group received diagnostic aid simulating artificial intelligence for intracranial hemorrhage only (n = 26); the other, for large-vessel occlusion only (n = 28). Primary outcomes included accuracy, sensitivity, and specificity for intracranial hemorrhage/large-vessel occlusion detection without and with aid. Study interpretation time was a secondary outcome. Individual responses were pooled and analyzed with the t test; differences in continuous variables were assessed with ANOVA.

RESULTS

Forty-eight participants completed the study, generating 10,779 intracranial hemorrhage or large-vessel occlusion interpretations. With diagnostic aid, medical student accuracy improved 11.0 points (P < .001) and resident trainee accuracy showed no significant change. Intracranial hemorrhage interpretation time increased with diagnostic aid for both groups (P < .001), while large-vessel occlusion interpretation time decreased for medical students (P < .001). Despite worse performance in the detection of the smallest-versus-largest hemorrhages at baseline, medical students were not more likely to accept a true-positive artificial intelligence result for these more difficult tasks. Both groups were considerably less accurate when disagreeing with the artificial intelligence or when supplied with an incorrect artificial intelligence result.

CONCLUSIONS

This study demonstrated greater improvement in diagnostic accuracy with artificial intelligence for medical students compared with resident trainees. However, medical students were less likely than resident trainees to overrule incorrect artificial intelligence interpretations and were less accurate, even with diagnostic aid, than the artificial intelligence was by itself.

Toward Foundation Models in Radiology? Quantitative Assessment of GPT-4V's Multimodal and Multianatomic Region Capabilities

Background Large language models have already demonstrated potential in medical text processing. GPT-4V, a large vision-language model from OpenAI, has shown potential for medical imaging, yet a quantitative analysis is lacking. Purpose To quantitatively assess the performance of GPT-4V in interpreting radiologic images using unseen data. Materials and Methods This retrospective study included single representative abnormal and healthy control images from neuroradiology, cardiothoracic radiology, and musculoskeletal radiology (CT, MRI, radiography) to generate reports using GPT-4V via the application programming interface from February to March 2024. The factual correctness of free-text reports and the performance in detecting abnormalities in binary classification tasks were assessed using accuracy, sensitivity, and specificity. The binary classification performance was compared with that of a first-year nonradiologist in training and four board-certified radiologists. Results A total of 515 images in 470 patients (median age, 61 years [IQR, 44-71 years]; 267 male) were included, of which 345 images were abnormal. GPT-4V correctly identified the imaging modality and anatomic region in 100% (515 of 515) and 99.2% (511 of 515) of images, respectively. Diagnostic accuracy in free-text reports was between 0% (0 of 33 images) for pneumothorax (CT and radiography) and 90% (45 of 50 images) for brain tumor (MRI). In binary classification tasks, GPT-4V showed sensitivities between 56% (14 of 25 images) for ischemic stroke and 100% (25 of 25 images) for brain hemorrhage and specificities between 8% (two of 25 images) for brain hemorrhage and 52% (13 of 25 images) for pneumothorax, compared with a pooled sensitivity of 97.2% (1103 of 1135 images) and pooled specificity of 97.2% (1084 of 1115 images) for the human readers across all tasks. The model exhibited a clear tendency to overdiagnose abnormalities, with 86.5% (147 of 170 images) and 67.7% (151 of 223 images) false-positive rates for the free-text and binary classification tasks, respectively. Conclusion GPT-4V, in its earliest version, recognized medical image content and reliably determined the modality and anatomic region from single images. However, GPT-4V failed to detect, classify, or rule out abnormalities in image interpretation. © RSNA, 2024 Supplemental material is available for this article.

Applications of Artificial Intelligence and Machine Learning in Spine MRI

Diagnostic imaging, particularly MRI, plays a key role in the evaluation of many spine pathologies. Recent progress in artificial intelligence and its subset, machine learning, has led to many applications within spine MRI, which we sought to examine in this review. A literature search of the major databases (PubMed, MEDLINE, Web of Science, ClinicalTrials.gov) was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The search yielded 1226 results, of which 50 studies were selected for inclusion. Key data from these studies were extracted. Studies were categorized thematically into the following: Image Acquisition and Processing, Segmentation, Diagnosis and Treatment Planning, and Patient Selection and Prognostication. Gaps in the literature and the proposed areas of future research are discussed. Current research demonstrates the ability of artificial intelligence to improve various aspects of this field, from image acquisition to analysis and clinical care. We also acknowledge the limitations of current technology. Future work will require collaborative efforts in order to fully exploit new technologies while addressing the practical challenges of generalizability and implementation. In particular, the use of foundation models and large-language models in spine MRI is a promising area, warranting further research. Studies assessing model performance in real-world clinical settings will also help uncover unintended consequences and maximize the benefits for patient care.

Interobserver Agreement and Performance of Concurrent AI Assistance for Radiographic Evaluation of Knee Osteoarthritis

Background Due to conflicting findings in the literature, there are concerns about a lack of objectivity in grading knee osteoarthritis (KOA) on radiographs. Purpose To examine how artificial intelligence (AI) assistance affects the performance and interobserver agreement of radiologists and orthopedists of various experience levels when evaluating KOA on radiographs according to the established Kellgren-Lawrence (KL) grading system. Materials and Methods In this retrospective observer performance study, consecutive standing knee radiographs from patients with suspected KOA were collected from three participating European centers between April 2019 and May 2022. Each center recruited four readers across radiology and orthopedic surgery at in-training and board-certified experience levels. KL grading (KL-0 = no KOA, KL-4 = severe KOA) on the frontal view was assessed by readers with and without assistance from a commercial AI tool. The majority vote of three musculoskeletal radiology consultants established the reference standard. The ordinal receiver operating characteristic method was used to estimate grading performance. Light kappa was used to estimate interrater agreement, and bootstrapped t statistics were used to compare groups. Results Seventy-five studies were included from each center, totaling 225 studies (mean patient age, 55 years ± 15 [SD]; 113 female patients). The KL grades were KL-0, 24.0% (n = 54); KL-1, 28.0% (n = 63); KL-2, 21.8% (n = 49); KL-3, 18.7% (n = 42); and KL-4, 7.6% (n = 17). Eleven readers completed their readings. Three of the six junior readers showed higher KL grading performance with versus without AI assistance (area under the receiver operating characteristic curve, 0.81 ± 0.017 [SEM] vs 0.88 ± 0.011 [P < .001]; 0.76 ± 0.018 vs 0.86 ± 0.013 [P < .001]; and 0.89 ± 0.011 vs 0.91 ± 0.009 [P = .008]). Interobserver agreement for KL grading among all readers was higher with versus without AI assistance (κ = 0.77 ± 0.018 [SEM] vs 0.85 ± 0.013; P < .001). Board-certified radiologists achieved almost perfect agreement for KL grading when assisted by AI (κ = 0.90 ± 0.01), which was higher than that achieved by the reference readers independently (κ = 0.84 ± 0.017; P = .01). Conclusion AI assistance increased junior readers' radiographic KOA grading performance and increased interobserver agreement for osteoarthritis grading across all readers and experience levels. Published under a CC BY 4.0 license. Supplemental material is available for this article.

The use of automated and AI-driven algorithms for the detection of hippocampal sclerosis and focal cortical dysplasia

In drug-resistant epilepsy, magnetic resonance imaging (MRI) plays a central role in detecting lesions as it offers unmatched spatial resolution and whole-brain coverage. In addition, the last decade has witnessed continued developments in MRI-based computer-aided machine-learning techniques for improved diagnosis and prognosis. In this review, we focus on automated algorithms for the detection of hippocampal sclerosis and focal cortical dysplasia, particularly in cases deemed as MRI negative, with an emphasis on studies with histologically validated data. In addition, we discuss imaging-derived prognostic markers, including response to anti-seizure medication, post-surgical seizure outcome, and cognitive reserves. We also highlight the advantages and limitations of these approaches and discuss future directions toward person-centered care.