Creating high-quality radiology reports in foreign languages through multilingual structured reporting

Enhancing Oncological Surveillance Through Large Language Model-Assisted Analysis: A Comparative Study of GPT-4 and Gemini in Evaluating Oncological Issues From Serial Abdominal CT Scan Reports

RATIONALE AND OBJECTIVES

We aimed to compare the capabilities of two leading large language models (LLMs), GPT-4 and Gemini, in analyzing serial radiology reports, to highlight oncological issues that require further clinical attention.

MATERIALS AND METHODS

This study included 205 patients, each with two consecutive radiological reports. We designed a prompt comprising a three-step task to analyze report findings using LLMs. To establish a ground truth, two radiologists reached a consensus on a six-level categorization, comprising tumor findings (categorized as improved, stable, or aggravated), "benign", "no tumor description," and "other malignancy." The performance of GPT-4 and Gemini was then compared based on their ability to match corresponding findings between two radiological reports and accurately reflect these categories.

RESULTS

In terms of accuracy in matching findings between serial reports, the proportion of correctly matched findings was significantly higher for GPT-4 (96.2%) than for Gemini (91.7%) (P < 0.01). For oncological issue identification, the precision for tumor-related finding determinations, recall, and F1-scores were 0.68 and 0.63 (P = 0.006), 0.91 and 0.80 (P < 0.001), and 0.78 and 0.70 for GPT-4 and Gemini, respectively. GPT-4 was more accurate than Gemini in determining the correct tumor status for tumor-related findings (P < 0.001).

CONCLUSION

This study demonstrated the potential of LLM-assisted analysis of serial radiology reports in enhancing oncological surveillance, using a carefully engineered prompt. GPT-4 showed superior performance compared to Gemini in matching corresponding findings, identifying tumor-related findings, and accurately determining tumor status.

Large Language Model Ability to Translate CT and MRI Free-Text Radiology Reports Into Multiple Languages

Background High-quality translations of radiology reports are essential for optimal patient care. Because of limited availability of human translators with medical expertise, large language models (LLMs) are a promising solution, but their ability to translate radiology reports remains largely unexplored. Purpose To evaluate the accuracy and quality of various LLMs in translating radiology reports across high-resource languages (English, Italian, French, German, and Chinese) and low-resource languages (Swedish, Turkish, Russian, Greek, and Thai). Materials and Methods A dataset of 100 synthetic free-text radiology reports from CT and MRI scans was translated by 18 radiologists between January 14 and May 2, 2024, into nine target languages. Ten LLMs, including GPT-4 (OpenAI), Llama 3 (Meta), and Mixtral models (Mistral AI), were used for automated translation. Translation accuracy and quality were assessed with use of BiLingual Evaluation Understudy (BLEU) score, translation error rate (TER), and CHaRacter-level F-score (chrF++) metrics. Statistical significance was evaluated with use of paired t tests with Holm-Bonferroni corrections. Radiologists also conducted a qualitative evaluation of translations with use of a standardized questionnaire. Results GPT-4 demonstrated the best overall translation quality, particularly from English to German (BLEU score: 35.0 ± 16.3 [SD]; TER: 61.7 ± 21.2; chrF++: 70.6 ± 9.4), to Greek (BLEU: 32.6 ± 10.1; TER: 52.4 ± 10.6; chrF++: 62.8 ± 6.4), to Thai (BLEU: 53.2 ± 7.3; TER: 74.3 ± 5.2; chrF++: 48.4 ± 6.6), and to Turkish (BLEU: 35.5 ± 6.6; TER: 52.7 ± 7.4; chrF++: 70.7 ± 3.7). GPT-3.5 showed highest accuracy in translations from English to French, and Qwen1.5 excelled in English-to-Chinese translations, whereas Mixtral 8x22B performed best in Italian-to-English translations. The qualitative evaluation revealed that LLMs excelled in clarity, readability, and consistency with the original meaning but showed moderate medical terminology accuracy. Conclusion LLMs showed high accuracy and quality for translating radiology reports, although results varied by model and language pair. © RSNA, 2024 Supplemental material is available for this article.

DXA Reporting Updates: 2023 Official Positions of the International Society for Clinical Densitometry

INTRODUCTION

Professional guidance and standards assist radiologic interpreters in generating high quality reports. Initially DXA reporting Official Positions were provided by the ISCD in 2003; however, as the field has progressed, some of the current recommendations require revision and updating. This manuscript details the research approach and provides updated DXA reporting guidance.

METHODS

Key Questions were proposed by ISCD established protocols and approved by the Position Development Conference Steering Committee. Literature related to each question was accumulated by searching PubMed, and existing guidelines from other organizations were extracted from websites. Modifications and additions to the ISCD Official Positions were determined by an expert panel after reviewing the Task Force proposals and position papers.

RESULTS

Since most DXA is now performed in radiology departments, an approach was endorsed that better aligns with standard radiologic reports. To achieve this, reporting elements were divided into required minimum or optional. Collectively, required components comprise a standard diagnostic report and are considered the minimum necessary to generate an acceptable report. Additional elements were retained and categorized as optional. These optional components were considered relevant but tailored to a consultative, clinically oriented report. Although this information is beneficial, not all interpreters have access to sufficient clinical information, or may not have the clinical expertise to expand beyond a diagnostic report. Consequently, these are not required for an acceptable report.

CONCLUSION

These updated ISCD positions conform with the DXA field's evolution over the past 20 years. Specifically, a basic diagnostic report better aligns with radiology standards, and additional elements (which are valued by treating clinicians) remain acceptable but are optional and not required. Additionally, reporting guidance for newer elements such as fracture risk assessment are incorporated. It is our expectation that these updated Official Positions will improve compliance with required standards and generate high quality DXA reports that are valuable to the recipient clinician and contribute to best patient care.

Structured reporting of computed tomography in the polytrauma patient assessment: a Delphi consensus proposal

OBJECTIVES

To develop a structured reporting (SR) template for whole-body CT examinations of polytrauma patients, based on the consensus of a panel of emergency radiology experts from the Italian Society of Medical and Interventional Radiology.

METHODS

A multi-round Delphi method was used to quantify inter-panelist agreement for all SR sections. Internal consistency for each section and quality analysis in terms of average inter-item correlation were evaluated by means of the Cronbach's alpha (Cα) correlation coefficient.

RESULTS

The final SR form included 118 items (6 in the "Patient Clinical Data" section, 4 in the "Clinical Evaluation" section, 9 in the "Imaging Protocol" section, and 99 in the "Report" section). The experts' overall mean score and sum of scores were 4.77 (range 1-5) and 257.56 (range 206-270) in the first Delphi round, and 4.96 (range 4-5) and 208.44 (range 200-210) in the second round, respectively. In the second Delphi round, the experts' overall mean score was higher than in the first round, and standard deviation was lower (3.11 in the second round vs 19.71 in the first round), reflecting a higher expert agreement in the second round. Moreover, Cα was higher in the second round than in the first round (0.97 vs 0.87).

CONCLUSIONS

Our SR template for whole-body CT examinations of polytrauma patients is based on a strong agreement among panel experts in emergency radiology and could improve communication between radiologists and the trauma team.

Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability

Purpose Ultrasound examinations are often criticized for having higher examiner dependency compared to other imaging techniques. Compared to free-text reporting, structured reporting (SR) of head and neck sonography (HNS) achieves superior time efficiency as well as report quality. However, there are no findings concerning the influence of SR on the interrater reliability (IRR) of HNS. Materials and Methods Typical pathologies (n=4) in HNS were documented by video/images by two certified head and neck ultrasound instructors. Consequently, structured reports of these videos/images were created by n=9 senior physicians at departments of otolaryngology or maxillofacial surgery with DEGUM instructors on staff. Reports (n=36) were evaluated regarding overall completeness and IRR. Additionally, user satisfaction was assessed by a visual analog scale (VAS). Results SR yielded very high report completeness (91.8%) in all four cases with a substantial IRR (Fleiss' κ 0.73). Interrater agreement was high at 87.2% with very good user satisfaction (VAS 8.6). Conclusion SR has the potential to ensure high-quality examination reports with substantial comparability and very high user satisfaction. Furthermore, big data collection and analysis are facilitated by SR. Therefore, process quality, workflow, and scientific output are potentially enhanced by SR.

Structured Reporting in Radiological Settings: Pitfalls and Perspectives

Objective: The aim of this manuscript is to give an overview of structured reporting in radiological settings. Materials and Method: This article is a narrative review on structured reporting in radiological settings. Particularly, limitations and future perspectives are analyzed. RESULTS: The radiological report is a communication tool for the referring physician and the patients. It was conceived as a free text report (FTR) to allow radiologists to have their own individuality in the description of the radiological findings. However, this form could suffer from content, style, and presentation discrepancies, with a probability of transferring incorrect radiological data. Quality, datafication/quantification, and accessibility represent the three main goals in moving from FTRs to structured reports (SRs). In fact, the quality is related to standardization, which aims to improve communication and clarification. Moreover, a “structured” checklist, which allows all the fundamental items for a particular radiological study to be reported and permits the connection of the radiological data with clinical features, allowing a personalized medicine. With regard to accessibility, since radiological reports can be considered a source of research data, SR allows data mining to obtain new biomarkers and to help the development of new application domains, especially in the field of radiomics. Conclusions: Structured reporting could eliminate radiologist individuality, allowing a standardized approach.

Structured reporting of x-ray mammography in the first diagnosis of breast cancer: a Delphi consensus proposal

Background

Radiology is an essential tool in the management of a patient. The aim of this manuscript was to build structured report (SR) Mammography based in Breast Cancer.

Methods

A working team of 16 experts (group A) was composed to create a SR for Mammography Breast Cancer. A further working group of 4 experts (group B), blinded to the activities of the group A, was composed to assess the quality and clinical usefulness of the SR final draft. Modified Delphi process was used to assess level of agreement for all report sections. Cronbach’s alpha (Cα) correlation coefficient was used to assess internal consistency and to measure quality analysis according to the average inter-item correlation.

Results

The final SR version was built by including n = 2 items in Personal Data, n = 4 items in Setting, n = 2 items in Comparison with previous breast examination, n = 19 items in Anamnesis and clinical context; n = 10 items in Technique; n = 1 item in Radiation dose; n = 5 items Parenchymal pattern; n = 28 items in Description of the finding; n = 12 items in Diagnostic categories and Report and n = 1 item in Conclusions. The overall mean score of the experts and the sum of score for structured report were 4.9 and 807 in the second round. The Cronbach’s alpha (Cα) correlation coefficient was 0.82 in the second round. About the quality evaluation, the overall mean score of the experts was 3.3. The Cronbach’s alpha (Cα) correlation coefficient was 0.90.

Conclusions

Structured reporting improves the quality, clarity and reproducibility of reports across departments, cities, countries and internationally and will assist patient management and improve breast health care and facilitate research.

[Structured reporting in radiology : German and European radiology societies' point of view]

Zahlreiche Publikationen belegen den herausragenden Wert einer strukturierten Befundung sowohl für die Kommunikation mit zuweisenden klinischen Kollegen als auch für die Weiterverwendung der Befunddaten in anderen Kontexten. Obwohl das Thema bereits seit vielen Jahren in der Radiologie bekannt ist, hat sich die strukturierte Befundung noch nicht flächendeckend in der klinischen Routine etablieren können. Alle größeren radiologischen Fachgesellschaften haben sich klar für die strukturierte Befundung ausgesprochen und verfolgen etliche Initiativen auf diesem Gebiet. Dazu zählt der Aufbau frei zugänglicher Sammlungen von Befundvorlagen und die Qualitätssicherung derselben sowie die Pflege und Entwicklung standardisierter Begriffslexika. Im vorliegenden Artikel werden insbesondere die Aktivitäten der Deutschen Röntgengesellschaft und der European Society of Radiology dargestellt sowie ein kurzer Überblick über Vor- und Nachteile und verfügbare Ressourcen gegeben.

A Hybrid Reporting Platform for Extended RadLex Coding Combining Structured Reporting Templates and Natural Language Processing

Structured reporting is a favorable and sustainable form of reporting in radiology. Among its advantages are better presentation, clearer nomenclature, and higher quality. By using MRRT-compliant templates, the content of the categorized items (e.g., select fields) can be automatically stored in a database, which allows further research and quality analytics based on established ontologies like RadLex® linked to the items. Additionally, it is relevant to provide free-text input for descriptions of findings and impressions in complex imaging studies or for the information included with the clinical referral. So far, however, this unstructured content cannot be categorized. We developed a solution to analyze and code these free-text parts of the templates in our MRRT-compliant reporting platform, using natural language processing (NLP) with RadLex® terms in addition to the already categorized items. The established hybrid reporting concept is working successfully. The NLP tool provides RadLex® codes with modifiers (affirmed, speculated, negated). Radiologists can confirm or reject codes provided by NLP before finalizing the structured report. Furthermore, users can suggest RadLex® codes from free text that is not correctly coded with NLP or can suggest to change the modifier. Analyzing free-text fields took 1.23 s on average. Hybrid reporting enables coding of free-text information in our MRRT-compliant templates and thus increases the amount of categorized data that can be stored in the database. This enhances the possibilities for further analyses, such as correlating clinical information with radiological findings or storing high-quality structured information for machine-learning approaches.

Quantitative evaluation of chronically obstructed kidneys from noncontrast computed tomography based on deep learning

OBJECTIVE

To quantitatively report renal parenchymal volume (RPV), renal sinus volume (RSV), and renal parenchymal density (RPD) for chronically obstructed kidneys from noncontrast computed tomography (NCCT).

METHODS

This retrospective study was approved by the institutional review board of our hospital with a waiver of informed consent. We retrospectively collected 304 consecutive NCCT scans of urinary obstruction and constructed two datasets: one with 167 patient scans for parenchyma and sinus segmentation (segmentation dataset) and the other containing 137 scans from different patients diagnosed with chronic urinary obstruction (CUO dataset) and paired with split glomerular filtration rate (sGFR). A cascaded three-dimensional (3D) U-Net model was developed and validated for parenchyma and sinus segmentation. The RPV, RSV, and RPD of the CUO dataset were calculated by the model with manual editing. A multivariate analysis was performed to show the association between all parameters and the sGFR.

RESULTS

In the test dataset, the Dice values for parenchyma and sinus segmentation were 0.95 ± 0.04 and 0.90 ± 0.05, respectively. Compared with those of nonobstructed kidneys, the RSV and RPD of obstructed kidneys increased, but RPV and sGFR decreased (P < .001). For chronically obstructed kidneys, age (r = -0.292, P < .001), RPV (r = 0.849, P < .001), RSV (r = -0.331, P < .001), and RPD (r = -0.296, P < .001) were significantly correlated with sGFR. The fitted regression model was sGFR = 10.873-0.111 Age + 0.211 RPV - 0.022 RSV (r² = 0.712).

CONCLUSIONS

NCCT combined with deep learning has the potential to be a single radiological procedure for morphological and functional evaluation of chronically obstructed kidneys.

Artificial Intelligence Applications for Workflow, Process Optimization and Predictive Analytics

There is great potential for artificial intelligence (AI) applications, especially machine learning and natural language processing, in medical imaging. Much attention has been garnered by the image analysis tasks for diagnostic decision support and precision medicine, but there are many other potential applications of AI in radiology and have potential to enhance all levels of the radiology workflow and practice, including workflow optimization and support for interpretation tasks, quality and safety, and operational efficiency. This article reviews the important potential applications of informatics and AI related to process improvement and operations in the radiology department.

Emergent Comprehensive Imaging of the Major Trauma Patient: A New Paradigm for Improved Clinical Decision-Making

Modern advances in the medical imaging layered onto sophisticated trauma resuscitation strategies in highly organized regionalized trauma systems have created a paradigm shift in the management of severely injured patients. Although immediate exploratory surgery to identify and control life-threatening injuries still has its place, accelerated image acquisition and interpretation procedures now make it rare for trauma surgeons in major centers to venture into damage control surgery unaided by computed tomography (CT) or other imaging, particularly in cases of blunt trauma. Indeed, because of the high incidence of clinically occult injuries associated with major mechanism trauma, and even lower energy trauma in frail or elderly patients, CT imaging has become as invaluable as physical examination, if not more so, in critical decision-making in support of optimal outcomes. In particular, whole-body computed tomography (WBCT) completed promptly after initial assessment of a major trauma provides a quick, comprehensive survey of injuries that enables better surgical planning, obviates the need for multiple subsequent studies, and permits specialized reconstructions when needed. For those at risk for problematic occult injury after modest trauma, WBCT facilitates safer discharge planning and simplified follow-up. Through standardized guidelines, streamlined protocols, synoptic reporting, accessible web-based platforms, and active collaboration with clinicians, radiologists dedicated to trauma and emergency imaging enable clearer understanding of complex injuries in high-risk patients which leads to superior clinical decision-making. Whereas dated dogma has long warned that the CT scanner is the last place to take a challenging trauma patient, modern practice suggests that, more often than not, early comprehensive imaging can be done safely and efficiently and is in the patient's best interest. This article outlines how the role of diagnostic imaging for major trauma has evolved considerably in recent years.

Centralized expert HRCT Reading in suspected idiopathic pulmonary fibrosis: Experience from an Eurasian teleradiology program

PURPOSE

To share experience from a large, ongoing expert reading teleradiology program in Europe and Asia aiming at supporting referring centers to interpret high-resolution computed tomography (HRCT) with respect to presence of Usual Interstitial Pneumonia (UIP)-pattern in patients with suspected Idiopathic Pulmonary Fibrosis (IPF).

METHOD

We analyzed data from 01/2014 to 05/2019, including HRCTs from 239 medical centers in 12 European and Asian countries that were transmitted to our Picture Archiving and Communication System (PACS) via a secured internet connection. Structured reports were generated in consensus by a radiologist with over 20 years of experience in thoracic imaging and a pulmonologist with specific expertise in interstitial lung disease according to current guidelines on IPF. Reports were sent to referring physicians. We evaluated patient characteristics, technical issues, report turnaround times and frequency of diagnoses. We also conducted a survey to collect feedback from referring physicians.

RESULTS

HRCT image data from 703 patients were transmitted (53.5% male). Mean age was 63.7 years (SD:17). In 35.1% of all cases diagnosis was "UIP"/"Typical UIP". The mean report turnaround time was 1.7 days (SD:2.9). Data transmission errors occurred in 7.1%. Overall satisfaction rate among referring physicians was high (8.4 out of 10; SD:3.2).

CONCLUSIONS

This Eurasian teleradiology program demonstrates the feasibility of cross-border teleradiology for the provision of state-of-the-art reporting despite heterogeneity of referring medical centers and challenges like data transmission errors and language barriers. We also point out important factors for success like the usage of structured reporting templates.