Structured report data can be used to develop deep learning algorithms: a proof of concept in ankle radiographs

Open-source Large Language Models can Generate Labels from Radiology Reports for Training Convolutional Neural Networks

RATIONALE AND OBJECTIVES

Training Convolutional Neural Networks (CNN) requires large datasets with labeled data, which can be very labor-intensive to prepare. Radiology reports contain a lot of potentially useful information for such tasks. However, they are often unstructured and cannot be directly used for training. The recent progress in large language models (LLMs) might introduce a new useful tool in interpreting radiology reports. This study aims to explore the use of the LLM to classify radiology reports and generate labels. These labels will be utilized then to train a CNN to detect ankle fractures to evaluate the effectiveness of using automatically generated labels.

MATERIALS AND METHODS

We used the open-weight LLM Mixtral-8×7B-Instruct-v0.1 to classify radiology reports of ankle x-ray images. The generated labels were used to train a CNN to recognize ankle fractures. The model's accuracy, sensitivity, specificity, and area under the receiver operating characteristics curve were used for evaluation.

RESULTS

Using common prompt engineering techniques, a prompt was found that reached an accuracy of 92% on a test dataset. By parsing all radiology reports using the LLM, a training dataset of 15,896 images and labels was created. Using this dataset, a CNN was trained, which achieved an accuracy of 89.5% and an area under the receiver operating characteristic curve of 0.926 on a test dataset.

CONCLUSION

Our classification model based on labels generated with a large language model achieved high accuracy. This performance is comparable to models trained with manually labeled data, demonstrating the potential of language models in automating the labeling process.

SUMMARY

Large language models can be used to reliably detect pathologies in radiology reports.

KEY RESULTS

In this study, 7561 radiological reports of ankle X-ray images were automatically classified as describing an ankle fracture or not using a large language model. Using a dataset of 250 reports, the language model showed a classification accuracy of 92%. The generated labels were used to train an image classifier to detect ankle fractures on X-ray images. 15,896 images were used for training. The resulting model achieved an accuracy of 89.5% on a test dataset.

A systematic review of natural language processing applications in Trauma & Orthopaedics

Aims

Prevalence of artificial intelligence (AI) algorithms within the Trauma & Orthopaedics (T&O) literature has greatly increased over the last ten years. One increasingly explored aspect of AI is the automated interpretation of free-text data often prevalent in electronic medical records (known as natural language processing (NLP)). We set out to review the current evidence for applications of NLP methodology in T&O, including assessment of study design and reporting.

Methods

MEDLINE, Allied and Complementary Medicine (AMED), Excerpta Medica Database (EMBASE), and Cochrane Central Register of Controlled Trials (CENTRAL) were screened for studies pertaining to NLP in T&O from database inception to 31 December 2023. An additional grey literature search was performed. NLP quality assessment followed the criteria outlined by Farrow et al in 2021 with two independent reviewers (classification as absent, incomplete, or complete). Reporting was performed according to the Synthesis-Without Meta-Analysis (SWiM) guidelines. The review protocol was registered on the Prospective Register of Systematic Reviews (PROSPERO; registration no. CRD42022291714).

Results

The final review included 31 articles (published between 2012 and 2021). The most common subspeciality areas included trauma, arthroplasty, and spine; 13% (4/31) related to online reviews/social media, 42% (13/31) to clinical notes/operation notes, 42% (13/31) to radiology reports, and 3% (1/31) to systematic review. According to the reporting criteria, 16% (5/31) were considered good quality, 74% (23/31) average quality, and 6% (2/31) poor quality. The most commonly absent reporting criteria were evaluation of missing data (26/31), sample size calculation (31/31), and external validation of the study results (29/31 papers). Code and data availability were also poorly documented in most studies.

Conclusion

Application of NLP is becoming increasingly common in T&O; however, published article quality is mixed, with few high-quality studies. There are key consistent deficiencies in published work relating to NLP which ultimately influence the potential for clinical application. Open science is an important part of research transparency that should be encouraged in NLP algorithm development and reporting.

Artificial intelligence and machine learning algorithms in diagnosis and therapy of the ankle joint

The recent advancement of computational systems provides fast information exchange and the collection of large amounts of data. Growing number of those systems allow for effective processing of huge amounts of information, utilizing advanced algorithms that are called artificial intelligence (AI). AI has been used for many years, and the number of its applications is growing in various areas. Such solutions are also being developed increasingly in medicine, including orthopedics and radiology, to support the diagnostic and therapeutic processes. Progress in this area is particularly targeted at the skeletal sites that most often require intervention, such as the hip or knee area, with modest interest in the ankle joint. The ankle is one of the most complicated human joints, and therapeutic procedures for its treatment are relatively common. One of the solutions used in the event of serious ankle joint damage is arthroplasty. This review summarizes the current state of AI applications for the diagnosis and therapy of the ankle joint, focusing on trends and achievements in ankle joint arthroplasty and contemporary orthopedic AI solutions. Ideas from other fields of medical diagnostics or orthopedic surgery that may be utilized in the diagnosis and treatment of ankle joint are also discussed.

Improving the use of LLMs in radiology through prompt engineering: from precision prompts to zero-shot learning

PURPOSE

Large language models (LLMs) such as ChatGPT have shown significant potential in radiology. Their effectiveness often depends on prompt engineering, which optimizes the interaction with the chatbot for accurate results. Here, we highlight the critical role of prompt engineering in tailoring the LLMs' responses to specific medical tasks.

MATERIALS AND METHODS

Using a clinical case, we elucidate different prompting strategies to adapt the LLM ChatGPT using GPT4 to new tasks without additional training of the base model. These approaches range from precision prompts to advanced in-context methods such as few-shot and zero-shot learning. Additionally, the significance of embeddings, which serve as a data representation technique, is discussed.

RESULTS

Prompt engineering substantially improved and focused the chatbot's output. Moreover, embedding of specialized knowledge allows for more transparent insight into the model's decision-making and thus enhances trust.

CONCLUSION

Despite certain challenges, prompt engineering plays a pivotal role in harnessing the potential of LLMs for specialized tasks in the medical domain, particularly radiology. As LLMs continue to evolve, techniques like few-shot learning, zero-shot learning, and embedding-based retrieval mechanisms will become indispensable in delivering tailored outputs.

KEY POINTS

· Large language models might impact radiological practice and decision-masking.. · However, implementation and performance are dependent on the assigned task.. · Optimization of prompting strategies can substantially improve model performance.. · Strategies for prompt engineering range from precision prompts to zero-shot learning..

CITATION FORMAT

· Russe MF, Reisert M, Bamberg F et al. Improving the use of LLMs in radiology through prompt engineering: from precision prompts to zero-shot learning . Fortschr Röntgenstr 2024; 196: 1166 - 1170.

A novel reporting workflow for automated integration of artificial intelligence results into structured radiology reports

OBJECTIVES

Artificial intelligence (AI) has tremendous potential to help radiologists in daily clinical routine. However, a seamless, standardized, and time-efficient way of integrating AI into the radiology workflow is often lacking. This constrains the full potential of this technology. To address this, we developed a new reporting pipeline that enables automated pre-population of structured reports with results provided by AI tools.

METHODS

Findings from a commercially available AI tool for chest X-ray pathology detection were sent to an IHE-MRRT-compliant structured reporting (SR) platform as DICOM SR elements and used to automatically pre-populate a chest X-ray SR template. Pre-populated AI results could be validated, altered, or deleted by radiologists accessing the SR template. We assessed the performance of this newly developed AI to SR pipeline by comparing reporting times and subjective report quality to reports created as free-text and conventional structured reports.

RESULTS

Chest X-ray reports with the new pipeline could be created in significantly less time than free-text reports and conventional structured reports (mean reporting times: 66.8 s vs. 85.6 s and 85.8 s, respectively; both p < 0.001). Reports created with the pipeline were rated significantly higher quality on a 5-point Likert scale than free-text reports (p < 0.001).

CONCLUSION

The AI to SR pipeline offers a standardized, time-efficient way to integrate AI-generated findings into the reporting workflow as parts of structured reports and has the potential to improve clinical AI integration and further increase synergy between AI and SR in the future.

CRITICAL RELEVANCE STATEMENT

With the AI-to-structured reporting pipeline, chest X-ray reports can be created in a standardized, time-efficient, and high-quality manner. The pipeline has the potential to improve AI integration into daily clinical routine, which may facilitate utilization of the benefits of AI to the fullest.

KEY POINTS

• A pipeline was developed for automated transfer of AI results into structured reports. • Pipeline chest X-ray reporting is faster than free-text or conventional structured reports. • Report quality was also rated higher for reports created with the pipeline. • The pipeline offers efficient, standardized AI integration into the clinical workflow.

European Society of Radiology (ESR), dos Santos DP, Kotter E, Mildenberger P, Martí-Bonmatí L. ESR paper on structured reporting in radiology-update 2023

Structured reporting in radiology continues to hold substantial potential to improve the quality of service provided to patients and referring physicians. Despite many physicians' preference for structured reports and various efforts by radiological societies and some vendors, structured reporting has still not been widely adopted in clinical routine.While in many countries national radiological societies have launched initiatives to further promote structured reporting, cross-institutional applications of report templates and incentives for usage of structured reporting are lacking. Various legislative measures have been taken in the USA and the European Union to promote interoperable data formats such as Fast Healthcare Interoperability Resources (FHIR) in the context of the EU Health Data Space (EHDS) which will certainly be relevant for the future of structured reporting. Lastly, recent advances in artificial intelligence and large language models may provide innovative and efficient approaches to integrate structured reporting more seamlessly into the radiologists' workflow.The ESR will remain committed to advancing structured reporting as a key component towards more value-based radiology. Practical solutions for structured reporting need to be provided by vendors. Policy makers should incentivize the usage of structured radiological reporting, especially in cross-institutional setting.Critical relevance statement Over the past years, the benefits of structured reporting in radiology have been widely discussed and agreed upon; however, implementation in clinical routine is lacking due-policy makers should incentivize the usage of structured radiological reporting, especially in cross-institutional setting.Key points1. Various national societies have established initiatives for structured reporting in radiology.2. Almost no monetary or structural incentives exist that favor structured reporting.3. A consensus on technical standards for structured reporting is still missing.4. The application of large language models may help structuring radiological reports.5. Policy makers should incentivize the usage of structured radiological reporting.

Advancements in Artificial Intelligence for Foot and Ankle Surgery: A Systematic Review

Background

There has been a rapid increase in research applying artificial intelligence (AI) to various subspecialties of orthopaedic surgery, including foot and ankle surgery. The purpose of this systematic review is to (1) characterize the topics and objectives of studies using AI in foot and ankle surgery, (2) evaluate the performance of their models, and (3) evaluate their validity (internal or external validation).

Methods

A systematic literature review was conducted using PubMed/MEDLINE and Embase databases in December 2022. All studies that used AI or its subsets machine learning (ML) and deep learning (DL) in the setting of foot and ankle surgery relevant to orthopaedic surgeons were included. Studies were evaluated for their demographics, subject area, outcomes of interest, model(s) tested, model(s)' performance, and validity (internal or external).

Results

A total of 31 studies met inclusion criteria: 14 studies investigated AI for image interpretation, 13 studies investigated AI for clinical predictions, and 4 studies were grouped as "other." Studies commonly explored AI for ankle fractures, calcaneus fractures, hallux valgus, Achilles tendon pathologies, plantar fasciitis, and sports injuries. For studies reporting the area under the receiver operating characteristic curve (AUC), AUCs ranged from 0.64 (poor) to 0.99 (excellent). Two studies (6.45%) reported external validation.

Conclusion

Applications of AI in the field of foot and ankle surgery are expanding, particularly for image interpretation and clinical predictions. Current model performances range from poor to excellent, and most studies lack external validation, demonstrating a need for further research prior to deploying AI-based clinical applications.

Level of Evidence

Level III, retrospective cohort study.

Deep learning to detect anterior cruciate ligament tear on knee MRI: multi-continental external validation

OBJECTIVES

To develop a deep-learning algorithm for anterior cruciate ligament (ACL) tear detection and to compare its accuracy using two external datasets.

METHODS

A database of 19,765 knee MRI scans (17,738 patients) issued from different manufacturers and magnetic fields was used to build a deep learning-based ACL tear detector. Fifteen percent showed partial or complete ACL rupture. Coronal and sagittal fat-suppressed proton density or T2-weighted sequences were used. A Natural Language Processing algorithm was used to automatically label reports associated with each MRI exam. We compared the accuracy of our model on two publicly available external datasets: MRNet, Bien et al, USA (PLoS Med 15:e1002699, 2018); and KneeMRI, Stajduhar et al, Croatia (Comput Methods Prog Biomed 140:151-164, 2017). Receptor operating characteristics (ROC) curves, area under the curve (AUC), sensitivity, specificity, and accuracy were used to evaluate our model.

RESULTS

Our neural networks achieved an AUC value of 0.939 for detection of ACL tears, with a sensitivity of 87% (0.875) and a specificity of 91% (0.908). After retraining our model on Bien dataset and Stajduhar dataset, our algorithm achieved AUC of 0.962 (95% CI 0.930-0.988) and 0.922 (95% CI 0.875, 0.962) respectively. Sensitivity, specificity, and accuracy were respectively 85% (95% CI 75-94%, 0.852), 89% (95% CI 82-97%, 0.894), 0.875 (95% CI 0.817-0.933) for Bien dataset, and 68% (95% CI 54-81%, 0.681), 93% (95% CI 89-97%, 0.934), and 0.870 (95% CI 0.821-0.913) for Stajduhar dataset.

CONCLUSION

Our algorithm showed high performance in the detection of ACL tears with AUC on two external datasets, demonstrating its generalizability on different manufacturers and populations. This study shows the performance of an algorithm for detecting anterior cruciate ligament tears with an external validation on populations from countries and continents different from the study population.

KEY POINTS

• An algorithm for detecting anterior cruciate ligament ruptures was built from a large dataset of nearly 20,000 MRI with AUC values of 0.939, sensitivity of 87%, and specificity of 91%. • This algorithm was tested on two external populations from different other countries: a dataset from an American population and a dataset from a Croatian population. Performance remains high on these two external validation populations (AUC of 0.962 and 0.922 respectively).

Radiological Structured Report Integrated with Quantitative Imaging Biomarkers and Qualitative Scoring Systems

The benefits of structured reporting (SR) in radiology are well-known and have been widely described. However, there are limitations that must be overcome. Radiologists may be reluctant to change the conventional way of reporting. Error rates could potentially increase if SR is used improperly. Interruption of the visual search pattern by keeping the eyes focused on the report rather than the images may increase reporting time. Templates that include unnecessary or irrelevant information may undermine the consistency of the report. Last, the lack of support for multiple languages may hamper the adaptation of the report to the target audience. This work aims to mitigate these limitations with a web-based structured reporting system based on templates. By including field validators and logical rules, the system avoids reporting mistakes and allows to automatically calculate values and radiological qualitative scores. The system can manage quantitative information from imaging biomarkers, combining this with qualitative radiological information usually present in the structured report. It manages SR templates as plugins (IHE MRRT compliant and compatible with RSNA's Radreport templates), ensures a seamless integration with PACS/RIS systems, and adapts the report to the target audience by means of natural language extracts generated in multiple languages. We describe a use case of SR template for prostate cancer including PI-RADS 2.1 scoring system and imaging biomarkers. For the time being, the system comprises 24 SR templates and provides service in 37 hospitals and healthcare institutions, endorsing the success of this contribution to mitigate some of the limitations of the SR.

Radiology reporting in rectal cancer using MRI: adherence to national template for structured reporting

BACKGROUND

In 2014, a national workshop program was initiated and a reporting template and manual for rectal cancer primary staging using magnetic resonance imaging (MRI) was introduced and made available by the national Swedish Colorectal Cancer Registry.

PURPOSE

To evaluate the effect of the national template program by identify if there was a gap between the content in Swedish MRI reports from 2016 and the national reporting template from 2014. The aim was to explore and compare differences in content in reporting practice in different hospitals in relation to the national reporting template, with focus on: (i) identifying any implementational differences in reporting styles; and (ii) evaluating if reporting completeness vary based on such implementational differences.

MATERIAL AND METHODS

A total of 250 MRI reports from 10 hospitals in four healthcare regions in Sweden were collected. Reports were analyzed using qualitative content analysis with a deductive thematic coding scheme based on the national reporting template.

RESULTS

Three different implemented reporting styles were identified with variations of content coverage in relation to the template: (i) standardized and structured protocol (reporting style A); (ii) standardized semi-structured free-text (reporting style B); and (iii) regular free-text (reporting style C). The relative completeness of reporting practice of rectal cancer staging in relation to the national reporting template were 92.9% for reporting style A, 77.5% for reporting style B, and 63.9% for reporting style C.

CONCLUSION

The implementation of template-based reporting according to reporting style A is a key factor to conform to evidence-based practice for rectal cancer reporting using MRI.

[Structured reporting in radiology : German and European radiology societies' point of view]

Zahlreiche Publikationen belegen den herausragenden Wert einer strukturierten Befundung sowohl für die Kommunikation mit zuweisenden klinischen Kollegen als auch für die Weiterverwendung der Befunddaten in anderen Kontexten. Obwohl das Thema bereits seit vielen Jahren in der Radiologie bekannt ist, hat sich die strukturierte Befundung noch nicht flächendeckend in der klinischen Routine etablieren können. Alle größeren radiologischen Fachgesellschaften haben sich klar für die strukturierte Befundung ausgesprochen und verfolgen etliche Initiativen auf diesem Gebiet. Dazu zählt der Aufbau frei zugänglicher Sammlungen von Befundvorlagen und die Qualitätssicherung derselben sowie die Pflege und Entwicklung standardisierter Begriffslexika. Im vorliegenden Artikel werden insbesondere die Aktivitäten der Deutschen Röntgengesellschaft und der European Society of Radiology dargestellt sowie ein kurzer Überblick über Vor- und Nachteile und verfügbare Ressourcen gegeben.

Collection and Reporting of Patient-reported Outcome Measures in Arthroplasty Registries: Multinational Survey and Recommendations

BACKGROUND

Patient-reported outcome measures (PROMs) are validated questionnaires that are completed by patients. Arthroplasty registries vary in PROM collection and use. Current information about registry collection and use of PROMs is important to help improve methods of PROM data analysis, reporting, comparison, and use toward improving clinical practice.

QUESTIONS/PURPOSES

To characterize PROM collection and use by registries, we asked: (1) What is the current practice of PROM collection by arthroplasty registries that are current or former members of the International Society of Arthroplasty Registries, and are there sufficient similarities in PROM collection between registries to enable useful international comparisons that could inform the improvement of arthroplasty care? (2) How do registries differ in PROM administration and demographic, clinical, and comorbidity index variables collected for case-mix adjustment in data analysis and reporting? (3) What quality assurance methods are used for PROMs, and how are PROM results reported and used by registries? (4) What recommendations to arthroplasty registries may improve PROM reporting and facilitate international comparisons?

METHODS

An electronic survey was developed with questions about registry structure and collection, analysis, reporting, and use of PROM data and distributed to directors or senior administrators of 39 arthroplasty registries that were current or former members of the International Society of Arthroplasty Registries. In all, 64% (25 of 39) of registries responded and completed the survey. Missing responses from incomplete surveys were captured by contacting the registries, and up to three reminder emails were sent to nonresponding registries. Recommendations about PROM collection were drafted, revised, and approved by the International Society of Arthroplasty Registries PROMs Working Group members.

RESULTS

Of the 25 registries that completed the survey, 15 collected generic PROMs, most frequently the EuroQol-5 Dimension survey; 16 collected joint-specific PROMs, most frequently the Knee Injury and Osteoarthritis Outcome Score and Hip Disability and Osteoarthritis Outcome Score; and 11 registries collected a satisfaction item. Most registries administered PROM questionnaires within 3 months before and 1 year after surgery. All 16 registries that collected PROM data collected patient age, sex or gender, BMI, indication for the primary arthroplasty, reason for revision arthroplasty, and a comorbidity index, most often the American Society of Anesthesiologists classification. All 16 registries performed regular auditing and reporting of data quality, and most registries reported PROM results to hospitals and linked PROM data to other data sets such as hospital, medication, billing, and emergency care databases. Recommendations for transparent reporting of PROMs were grouped into four categories: demographic and clinical, survey administration, data analysis, and results.

CONCLUSION

Although registries differed in PROM collection and use, there were sufficient similarities that may enable useful data comparisons. The International Society of Arthroplasty Registries PROMs Working Group recommendations identify issues that may be important to most registries such as the need to make decisions about survey times and collection methods, as well as how to select generic and joint-specific surveys, handle missing data and attrition, report data, and ensure representativeness of the sample.

CLINICAL RELEVANCE

By collecting PROMs, registries can provide patient-centered data to surgeons, hospitals, and national entities to improve arthroplasty care.

Structured reporting of CT scans of patients with trauma leads to faster, more detailed diagnoses: An experimental study

PURPOSE

This study aimed to determine whether structured reports (SRs) reduce reporting time and/or increase the level of detail for trauma CT scans compared to free-text reports (FTRs).

METHOD

Eight radiology residents used SRs and FTRs to describe 14 whole-body CT scans of patients with polytrauma in a simulated emergency room setting. Each resident created both a brief report and a detailed report for each case using one of the two formats. We measured the time to complete the detailed reports and established a scoring system to objectively measure report completeness and the level of detail. Scoring sheets divided the CT findings into main and secondary criteria. Finally, the radiological residents completed a questionnaire on their opinions of the SRs and FTRs.

RESULTS

The detailed SRs were completed significantly faster than the detailed FTRs (mean 19 min vs. 25 min; p < 0.001). The maximum allowance of 25 min was used for 25% of SRs and 59% of FTRs. For brief reports, the SRs contained more secondary criteria than the FTRs (p = 0.001), but no significant differences were detected in main criteria. Study participants rated their own SRs as significantly more time-efficient, concise, and clearly structured compared to the FTRs. However, SRs and FTRs were rated similarly for quality, accuracy, and completeness.

CONCLUSION

We found that SRs for whole-body trauma CT add clinical value compared to FTRs because SRs reduce reporting time and increase the level of detail for trauma CT scans.

Machine learning based natural language processing of radiology reports in orthopaedic trauma

OBJECTIVES

To compare different Machine Learning (ML) Natural Language Processing (NLP) methods to classify radiology reports in orthopaedic trauma for the presence of injuries. Assessing NLP performance is a prerequisite for downstream tasks and therefore of importance from a clinical perspective (avoiding missed injuries, quality check, insight in diagnostic yield) as well as from a research perspective (identification of patient cohorts, annotation of radiographs).

METHODS

Datasets of Dutch radiology reports of injured extremities (n = 2469, 33% fractures) and chest radiographs (n = 799, 20% pneumothorax) were collected in two different hospitals and labeled by radiologists and trauma surgeons for the presence or absence of injuries. NLP classification was applied and optimized by testing different preprocessing steps and different classifiers (Rule-based, ML, and Bidirectional Encoder Representations from Transformers (BERT)). Performance was assessed by F1-score, AUC, sensitivity, specificity and accuracy.

RESULTS

The deep learning based BERT model outperforms all other classification methods which were assessed. The model achieved an F1-score of (95 ± 2)% and accuracy of (96 ± 1)% on a dataset of simple reports (n= 2469), and an F1 of (83 ± 7)% with accuracy (93 ± 2)% on a dataset of complex reports (n= 799).

CONCLUSION

BERT NLP outperforms traditional ML and rule-base classifiers when applied to Dutch radiology reports in orthopaedic trauma.

A Hybrid Reporting Platform for Extended RadLex Coding Combining Structured Reporting Templates and Natural Language Processing

Structured reporting is a favorable and sustainable form of reporting in radiology. Among its advantages are better presentation, clearer nomenclature, and higher quality. By using MRRT-compliant templates, the content of the categorized items (e.g., select fields) can be automatically stored in a database, which allows further research and quality analytics based on established ontologies like RadLex® linked to the items. Additionally, it is relevant to provide free-text input for descriptions of findings and impressions in complex imaging studies or for the information included with the clinical referral. So far, however, this unstructured content cannot be categorized. We developed a solution to analyze and code these free-text parts of the templates in our MRRT-compliant reporting platform, using natural language processing (NLP) with RadLex® terms in addition to the already categorized items. The established hybrid reporting concept is working successfully. The NLP tool provides RadLex® codes with modifiers (affirmed, speculated, negated). Radiologists can confirm or reject codes provided by NLP before finalizing the structured report. Furthermore, users can suggest RadLex® codes from free text that is not correctly coded with NLP or can suggest to change the modifier. Analyzing free-text fields took 1.23 s on average. Hybrid reporting enables coding of free-text information in our MRRT-compliant templates and thus increases the amount of categorized data that can be stored in the database. This enhances the possibilities for further analyses, such as correlating clinical information with radiological findings or storing high-quality structured information for machine-learning approaches.

Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports

MOTIVATION

The development of deep, bidirectional transformers such as Bidirectional Encoder Representations from Transformers (BERT) led to an outperformance of several Natural Language Processing (NLP) benchmarks. Especially in radiology, large amounts of free-text data are generated in daily clinical workflow. These report texts could be of particular use for the generation of labels in machine learning, especially for image classification. However, as report texts are mostly unstructured, advanced NLP methods are needed to enable accurate text classification. While neural networks can be used for this purpose, they must first be trained on large amounts of manually labelled data to achieve good results. In contrast, BERT models can be pre-trained on unlabelled data and then only require fine tuning on a small amount of manually labelled data to achieve even better results.

RESULTS

Using BERT to identify the most important findings in intensive care chest radiograph reports, we achieve areas under the receiver operation characteristics curve of 0.98 for congestion, 0.97 for effusion, 0.97 for consolidation and 0.99 for pneumothorax, surpassing the accuracy of previous approaches with comparatively little annotation effort. Our approach could therefore help to improve information extraction from free-text medical reports. Availability  and implementationWe make the source code for fine-tuning the BERT-models freely available at https://github.com/fast-raidiology/bert-for-radiology.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Digital twins, artificial intelligence, and machine learning technology to identify a real personalized motion axis of the tibiotalar joint for robotics in total ankle arthroplasty

PURPOSE

Axial alignment of the talar implant in total ankle arthroplasty remains a major issue, since the real axis of motion of each patient is impossible to determine with usual techniques. Further knowledge regarding individual axis of motion of the ankle is therefore needed.

MATERIAL AND METHODS

Therefore, digital twins, artificial intelligence, and machine learning technology were used to identify a real personalized motion axis of the tibiotalar joint. Three-dimensional (3D) models of distal extremities were generated using computed tomography data of normal patients. Digital twins were used to reproduce the mobility of the ankles, and the real ankle of the patients was matched to the digital twin with machine learning technology.

RESULTS

The results showed that a personalized axis can be obtained for each patient. When the origin of the axis is the centre of mass of the talus, this axis can be represented in a geodesic system. The mean value of the axis is a line passing in first approximation through the centre of the sphere (with a variation of 3 mm from the centre of the mass of the talus) and through a point with the coordinates 91.6° west and 7.4° north (range 84° to 98° west; - 2° to 12° north). This study improves the understanding of the axis of the ankle, as well as its relationship to the possibility to use the geodesic system for robotic in ankle arthroplasty.

CONCLUSION

The consideration of a personalized axis of the ankle might be helpful for better understanding of ankle surgery and particularly total ankle arthroplasty.

Structured reporting of brain MRI following mechanical thrombectomy in acute ischemic stroke patients

Background

To compare the quality of free-text reports (FTR) and structured reports (SR) of brain magnetic resonance imaging (MRI) examinations in patients following mechanical thrombectomy for acute stroke treatment.

Methods

A template for SR of brain MRI examinations based on decision trees was designed and developed in house and applied to twenty patients with acute ischemic stroke in addition to FTR. Two experienced stroke neurologists independently evaluated the quality of FTR and SR regarding clarity, content, presence of key features, information extraction, and overall report quality. The statistical analysis for the differences between FTR and SR was performed using the Mann–Whitney U-test or the Chi-squared test.

Results

Clarity (p < 0.001), comprehensibility (p < 0.001), inclusion of relevant findings (p = 0.016), structure (p = 0.005), and satisfaction with the content of the report for immediate patient management (p < 0.001) were evaluated significantly superior for the SR by both neurologist raters. One rater additionally found the explanation of the patient’s clinical symptoms (p = 0.003), completeness (p < 0.009) and length (p < 0.001) of SR to be significantly superior compared to FTR and stated that there remained no open questions, requiring further consultation of the radiologist (p < 0.001). Both neurologists preferred SR over FTR.

Conclusions

The use of SR for brain magnetic resonance imaging may increase the report quality and satisfaction of the referring physicians in acute ischemic stroke patients following mechanical thrombectomy.

Trial registration Retrospectively registered.

[Artificial Intelligence in Radiology - Definition, Potential and Challenges]

Artificial Intelligence in Radiology - Definition, Potential and Challenges Abstract. Artificial Intelligence (AI) is omnipresent. It has neatly permeated our daily life, even if we are not always fully aware of its ubiquitous presence. The healthcare sector in particular is experiencing a revolution which will change our daily routine considerably in the near future. Due to its advanced digitization and its historical technical affinity radiology is especially prone to these developments. But what exactly is AI and what makes AI so potent that established medical disciplines such as radiology worry about their future job perspectives? What are the assets of AI in radiology today - and what are the major challenges? This review article tries to give some answers to these questions.

The role of structured reporting and structured operation planning in functional endoscopic sinus surgery

Computed tomography (CT) scans represent the gold standard in the planning of functional endoscopic sinus surgeries (FESS). Yet, radiologists and otolaryngologists have different perspectives on these scans. In general, residents often struggle with aspects involved in both reporting and operation planning. The aim of this study was to compare the completeness of structured reports (SR) of preoperative CT images and structured operation planning (SOP) to conventional reports (CR) and conventional operation planning (COP) to potentially improve future treatment decisions on an individual level. In total, 30 preoperative CT scans obtained for surgical planning of patients scheduled for FESS were evaluated using SR and CR by radiology residents. Subsequently, otolaryngology residents performed a COP using free texts and a SOP using a specific template. All radiology reports and operation plannings were evaluated by two experienced FESS surgeons regarding their completeness for surgical planning. User satisfaction of otolaryngology residents was assessed by using visual analogue scales. Overall radiology report completeness was significantly higher using SRs regarding surgically important structures compared to CRs (84.4 vs. 22.0%, p<0.001). SOPs produced significantly higher completeness ratings (97% vs. 39.4%, p<0.001) regarding pathologies and anatomical variances. Moreover, time efficiency was not significantly impaired by implementation of SR (148 s vs. 160 s, p = 0.61) and user satisfaction was significantly higher for SOP (VAS 8.1 vs. 4.1, p<0.001). Implementation of SR and SOP results in a significantly increased completeness of radiology reports and operation planning for FESS. Consequently, the combination of both facilitates surgical planning and may decrease potential risks during FESS.

Contextual Structured Reporting in Radiology: Implementation and Long-Term Evaluation in Improving the Communication of Critical Findings

Structured reporting contributes to the completeness of radiology reports and improves quality. Both the content and the structure are essential for successful implementation of structured reporting. Contextual structured reporting is tailored to a specific scenario and can contain information retrieved from the context. Critical findings detected by imaging need urgent communication to the referring physician. According to guidelines, the occurrence of this communication should be documented in the radiology reports and should contain when, to whom and how was communicated. In free-text reporting, one or more of these required items might be omitted. We developed a contextual structured reporting template to ensure complete documentation of the communication of critical findings. The WHEN and HOW items were included automatically, and the insertion of the WHO-item was facilitated by the template. A pre- and post-implementation study demonstrated a substantial improvement in guideline adherence. The template usage improved in the long-term post-implementation study compared with the short-term results. The two most often occurring categories of critical findings are “infection / inflammation” and “oncology”, corresponding to the a large part of urgency level 2 (to be reported within 6 h) and level 3 (to be reported within 6 days), respectively. We conclude that contextual structured reporting is feasible for required elements in radiology reporting and for automated insertion of context-dependent data. Contextual structured reporting improves guideline adherence for communication of critical findings.

Emergent Comprehensive Imaging of the Major Trauma Patient: A New Paradigm for Improved Clinical Decision-Making

Modern advances in the medical imaging layered onto sophisticated trauma resuscitation strategies in highly organized regionalized trauma systems have created a paradigm shift in the management of severely injured patients. Although immediate exploratory surgery to identify and control life-threatening injuries still has its place, accelerated image acquisition and interpretation procedures now make it rare for trauma surgeons in major centers to venture into damage control surgery unaided by computed tomography (CT) or other imaging, particularly in cases of blunt trauma. Indeed, because of the high incidence of clinically occult injuries associated with major mechanism trauma, and even lower energy trauma in frail or elderly patients, CT imaging has become as invaluable as physical examination, if not more so, in critical decision-making in support of optimal outcomes. In particular, whole-body computed tomography (WBCT) completed promptly after initial assessment of a major trauma provides a quick, comprehensive survey of injuries that enables better surgical planning, obviates the need for multiple subsequent studies, and permits specialized reconstructions when needed. For those at risk for problematic occult injury after modest trauma, WBCT facilitates safer discharge planning and simplified follow-up. Through standardized guidelines, streamlined protocols, synoptic reporting, accessible web-based platforms, and active collaboration with clinicians, radiologists dedicated to trauma and emergency imaging enable clearer understanding of complex injuries in high-risk patients which leads to superior clinical decision-making. Whereas dated dogma has long warned that the CT scanner is the last place to take a challenging trauma patient, modern practice suggests that, more often than not, early comprehensive imaging can be done safely and efficiently and is in the patient's best interest. This article outlines how the role of diagnostic imaging for major trauma has evolved considerably in recent years.

Bridging the "last mile" gap between AI implementation and operation: "data awareness" that matters

Interest in the application of machine learning (ML) techniques to medicine is growing fast and wide because of their ability to endow decision support systems with so-called artificial intelligence, particularly in those medical disciplines that extensively rely on digital imaging. Nonetheless, achieving a pragmatic and ecological validation of medical AI systems in real-world settings is difficult, even when these systems exhibit very high accuracy in laboratory settings. This difficulty has been called the “last mile of implementation.” In this review of the concept, we claim that this metaphorical mile presents two chasms: the hiatus of human trust and the hiatus of machine experience. The former hiatus encompasses all that can hinder the concrete use of AI at the point of care, including availability and usability issues, but also the contradictory phenomena of cognitive ergonomics, such as automation bias (overreliance on technology) and prejudice against the machine (clearly the opposite). The latter hiatus, on the other hand, relates to the production and availability of a sufficient amount of reliable and accurate clinical data that is suitable to be the “experience” with which a machine can be trained. In briefly reviewing the existing literature, we focus on this latter hiatus of the last mile, as it has been largely neglected by both ML developers and doctors. In doing so, we argue that efforts to cross this chasm require data governance practices and a focus on data work, including the practices of data awareness and data hygiene. To address the challenge of bridging the chasms in the last mile of medical AI implementation, we discuss the six main socio-technical challenges that must be overcome in order to build robust bridges and deploy potentially effective AI in real-world clinical settings.