Error in radiology-where are we now?

Deep learning MR reconstruction in knees and ankles in children and young adults. Is it ready for clinical use?

OBJECTIVE

To evaluate the diagnostic performance and image quality of accelerated Turbo Spin Echo sequences using deep-learning (DL) reconstructions compared to conventional sequences in knee and ankle MRIs of children and young adults.

MATERIALS AND METHODS

IRB-approved prospective study consisting of 49 MRIs from 48 subjects (10 males, mean age 16.4 years, range 7-29 years), with each MRI consisting of both conventional and DL sequences. Sequences were evaluated blindly to determine predictive values, sensitivity, and specificity of DL sequences using conventional sequences and knee arthroscopy (if available) as references. Physeal patency and appearance were evaluated. Qualitative parameters were compared. Presence of undesired image alterations was assessed.

RESULTS

The prevalence of abnormal findings in the knees and ankles were 11.7% (75/640), and 11.5% (19/165), respectively. Using conventional sequences as reference, sensitivity and specificity of DL sequences in knees were 90.7% and 99.3%, and in ankles were 100.0% and 100.0%. Using arthroscopy as reference, sensitivity and specificity of DL sequences were 80.0% and 95.8%, and of conventional sequences were 80.0% and 97.9%. Agreement of physeal status was 100.0%. DL sequences were qualitatively "same-or-better" compared to conventional (p < 0.032), except for pixelation artifact for the PDFS sequence (p = 0.233). No discrete image alteration was identified in the knee DL sequences. In the ankle, we identified one DL artifact involving a tendon (0.8%, 1/125). DL sequences were faster than conventional sequences by a factor of 2 (p < 0.001).

CONCLUSION

In knee and ankle MRIs, DL sequences provided similar diagnostic performance and "same-or-better" image quality than conventional sequences at half the acquisition time.

Automatic segmentation of ameloblastoma on ct images using deep learning with limited data

BACKGROUND

Ameloblastoma, a common benign tumor found in the jaw bone, necessitates accurate localization and segmentation for effective diagnosis and treatment. However, the traditional manual segmentation method is plagued with inefficiencies and drawbacks. Hence, the implementation of an AI-based automatic segmentation approach is crucial to enhance clinical diagnosis and treatment procedures.

METHODS

We collected CT images from 79 patients diagnosed with ameloblastoma and employed a deep learning neural network model for training and testing purposes. Specifically, we utilized the Mask R-CNN neural network structure and implemented image preprocessing and enhancement techniques. During the testing phase, cross-validation methods were employed for evaluation, and the experimental results were verified using an external validation set. Finally, we obtained an additional dataset comprising 200 CT images of ameloblastoma from a different dental center to evaluate the model's generalization performance.

RESULTS

During extensive testing and evaluation, our model successfully demonstrated the capability to automatically segment ameloblastoma. The DICE index achieved an impressive value of 0.874. Moreover, when the IoU threshold ranged from 0.5 to 0.95, the model's AP was 0.741. For a specific IoU threshold of 0.5, the model achieved an AP of 0.914, and for another IoU threshold of 0.75, the AP was 0.826. Our validation using external data confirms the model's strong generalization performance.

CONCLUSION

In this study, we successfully applied a neural network model based on deep learning that effectively performs automatic segmentation of ameloblastoma. The proposed method offers notable advantages in terms of efficiency, accuracy, and speed, rendering it a promising tool for clinical diagnosis and treatment.

Can we predict discordant RECIST 1.1 evaluations in double read clinical trials?

Background

In lung clinical trials with imaging, blinded independent central review with double reads is recommended to reduce evaluation bias and the Response Evaluation Criteria In Solid Tumor (RECIST) is still widely used. We retrospectively analyzed the inter-reader discrepancies rate over time, the risk factors for discrepancies related to baseline evaluations, and the potential of machine learning to predict inter-reader discrepancies.

Materials and methods

We retrospectively analyzed five BICR clinical trials for patients on immunotherapy or targeted therapy for lung cancer. Double reads of 1724 patients involving 17 radiologists were performed using RECIST 1.1. We evaluated the rate of discrepancies over time according to four endpoints: progressive disease declared (PDD), date of progressive disease (DOPD), best overall response (BOR), and date of the first response (DOFR). Risk factors associated with discrepancies were analyzed, two predictive models were evaluated.

Results

At the end of trials, the discrepancy rates between trials were not different. On average, the discrepancy rates were 21.0%, 41.0%, 28.8%, and 48.8% for PDD, DOPD, BOR, and DOFR, respectively. Over time, the discrepancy rate was higher for DOFR than DOPD, and the rates increased as the trial progressed, even after accrual was completed. It was rare for readers to not find any disease, for less than 7% of patients, at least one reader selected non-measurable disease only (NTL). Often the readers selected some of their target lesions (TLs) and NTLs in different organs, with ranges of 36.0-57.9% and 60.5-73.5% of patients, respectively. Rarely (4-8.1%) two readers selected all their TLs in different locations. Significant risk factors were different depending on the endpoint and the trial being considered. Prediction had a poor performance but the positive predictive value was higher than 80%. The best classification was obtained with BOR.

Conclusion

Predicting discordance rates necessitates having knowledge of patient accrual, patient survival, and the probability of discordances over time. In lung cancer trials, although risk factors for inter-reader discrepancies are known, they are weakly significant, the ability to predict discrepancies from baseline data is limited. To boost prediction accuracy, it would be necessary to enhance baseline-derived features or create new ones, considering other risk factors and looking into optimal reader associations.

Repeatability of radiographic assessments for feline hip dysplasia suggest consensus scores in radiology are more uncertain than commonly assumed

Variation in the diagnostic interpretation of radiographs is a well-recognised problem in human and veterinary medicine. One common solution is to create a 'consensus' score based on a majority or unanimous decision from multiple observers. While consensus approaches are generally assumed to improve diagnostic repeatability, the extent to which consensus scores are themselves repeatable has rarely been examined. Here we use repeated assessments by three radiologists of 196 hip radiographs from 98 cats within a health-screening programme to examine intra-observer, inter-observer, majority-consensus and unanimous-consensus repeatability scores for feline hip dysplasia. In line with other studies, intra-observer and inter-observer repeatability was moderate (63-71%), and related to the reference assessment and time taken to reach a decision. Consensus scores did show reduced variation between assessments compared to individuals, but consensus repeatability was far from perfect. Only 75% of majority consensus scores were in agreement between assessments, and based on Bayesian multinomial modelling we estimate that unanimous consensus scores can have repeatabilities as low as 83%. These results clearly show that consensus scores in radiology can have large uncertainties, and that future studies in both human and veterinary medicine need to include consensus-uncertainty estimates if we are to properly interpret radiological diagnoses and the extent to which consensus scores improve diagnostic accuracy.

The role of radiologist in the changing world of healthcare: a White Paper of the European Society of Radiology (ESR)

Radiology as a specialty has been enormously successful since its beginnings, moving over time from an adjunct to clinical decision-making to a crucial component of multidisciplinary patient care. However, this increased centrality of radiology and reliance on our services carries within it dangers, prominent among them being the danger of our being viewed as deliverers of a commodity, and the risk of our becoming overwhelmed by increasing workload, unable to interact sufficiently with patients and referrers due to pressure of work. With this White Paper, the Board of Directors of the European Society of Radiology (ESR) seeks to briefly explain the position of the radiologist in the modern healthcare environment, considering our duties and contributions as doctors, protectors, communicators, innovators, scientists and teachers. This statement is intended to serve as a summary of the breadth of our responsibilities and roles, and to assist radiologists in countering misunderstanding of who we are and what we do.

Communicating with patients in the age of online portals-challenges and opportunities on the horizon for radiologists

The deployment of electronic patient portals increasingly allows patients throughout Europe to consult and share their radiology reports and images securely and timely online. Technical solutions and rules for releasing reports and images on patient portals may differ among institutions, regions and countries, and radiologists should therefore be familiar with the criteria by which reports and images are made available to their patients. Radiologists may also be solicited by patients who wish to discuss complex or critical imaging findings directly with the imaging expert who is responsible for the diagnosis. This emphasises the importance of radiologists’ communication skills as well as appropriate and efficient communication pathways and methods including electronic tools. Radiologists may also have to think about adapting reports as their final product in order to enable both referrers and patients to understand imaging findings. Actionable reports for a medical audience require structured, organ-specific terms and quantitative information, whereas patient-friendly summaries should preferably be based on consumer health language and include explanatory multimedia support or hyperlinks. Owing to the cultural and linguistic diversity in Europe dedicated solutions will require close collaboration between radiologists, patient representatives and software developers; software tools using artificial intelligence and natural language processing could potentially be useful in this context. By engaging actively in the challenges that are associated with increased communication with their patients, radiologists will not only have the opportunity to contribute to patient-centred care, but also to enhance the clinical relevance and the visibility of their profession.

Evaluation of neuroradiology emergency MRI interpretations: low discrepancy rates between on-call radiology residents' preliminary interpretations and neuroradiologists' final reports

OBJECTIVES

To evaluate the performance of on-call radiology residents in interpreting alone brain and spine MRI studies performed after hours, to describe their mistakes, and to identify influencing factors that increased the occurrence of errors.

METHODS

A total of 328 MRI examinations performed during a 13-month period (from December 1, 2019, to January 1, 2021) were prospectively included. Discrepancies between the preliminary interpretation of on-call radiology residents and the final reports of attending neuroradiologists were noted and classified according to a three-level score: level 1 (perfect interpretation or minor correction), level 2 (important correction without immediate change in patient management), or level 3 (major correction with immediate change in patient management). Categorical data were compared using Fisher's exact test.

RESULTS

The overall discrepancy rate (level-2 and level-3 errors) was 16%; the rate of major discrepancies (only level-3 errors) was 5.5%. The major-discrepancy rate of second-year residents, when compared with that of senior residents, was significantly higher (p = 0.02). Almost all of the level-3 errors concerned cerebrovascular pathology. The most common level-2 errors involved undescribed aneurysms. We found no significant difference in the major-discrepancy rate regarding time since the beginning of the shift.

CONCLUSIONS

The great majority of examinations were correctly interpreted. The rate of major discrepancies in our study was comparable to the data in the literature, and there was no adverse clinical outcome. The level of residency has an effect on the rate of serious errors in residents' reports.

KEY POINTS

• The rate of major discrepancies between preliminary MRI interpretations by on-call radiology residents and final reports by attending neuroradiologists is low, and comparable to discrepancy rates reported for head CT interpretations. • The youngest residents made significantly more serious errors when compared to senior residents. • There was no adverse clinical outcome in patient morbidity as a result of an initial misdiagnosis.

Automated Color-Coding of Lesion Changes in Contrast-Enhanced 3D T1-Weighted Sequences for MRI Follow-up of Brain Metastases

BACKGROUND AND PURPOSE

MR imaging is the technique of choice for follow-up of patients with brain metastases, yet the radiologic assessment is often tedious and error-prone, especially in examinations with multiple metastases or subtle changes. This study aimed to determine whether using automated color-coding improves the radiologic assessment of brain metastases compared with conventional reading.

MATERIALS AND METHODS

One hundred twenty-one pairs of follow-up examinations of patients with brain metastases were assessed. Two radiologists determined the presence of progression, regression, mixed changes, or stable disease between the follow-up examinations and indicated subjective diagnostic certainty regarding their decisions in a conventional reading and a second reading using automated color-coding after an interval of 8 weeks.

RESULTS

The rate of correctly classified diagnoses was higher (91.3%, 221/242, versus 74.0%, 179/242, P < .01) when using automated color-coding, and the median Likert score for diagnostic certainty improved from 2 (interquartile range, 2-3) to 4 (interquartile range, 3-5) (P < .05) compared with the conventional reading. Interrater agreement was excellent (κ = 0.80; 95% CI, 0.71-0.89) with automated color-coding compared with a moderate agreement (κ = 0.46; 95% CI, 0.34-0.58) with the conventional reading approach. When considering the time required for image preprocessing, the overall average time for reading an examination was longer in the automated color-coding approach (91.5 [SD, 23.1] seconds versus 79.4 [SD, 34.7 ] seconds, P < .001).

CONCLUSIONS

Compared with the conventional reading, automated color-coding of lesion changes in follow-up examinations of patients with brain metastases significantly increased the rate of correct diagnoses and resulted in higher diagnostic certainty.

CAR Peer Learning Guide

Peer learning is a quality initiative used to identify potential areas of practice improvement, both on a patient level and on a systemic level. Opportunities for peer learning include review of prior imaging studies, evaluation of cases from multidisciplinary case conferences, and review of radiology trainees' call cases. Peer learning is non-punitive and focuses on promoting life-long learning. It seeks to identify and disseminate learning opportunities and areas for systems improvement compared to traditional peer review. Learning opportunities arise from peer learning through both individual communication of cases reviewed for routine work, as well as through anonymous presentation of aggregate cases in an educational format. In conjunction with other tools such as root cause analysis, peer learning can be used to guide future practice improvement opportunities. This guide provides definitions of terms and a synthetic evidence review regarding peer review and peer learning, as well as medicolegal and jurisdictional considerations. Important aspects of what makes an effective peer learning program and best practices for implementing such a program are presented. The guide is intended to be a living document that will be updated regularly as new data emerges and peer learning continues to evolve in radiology practices.

Radiology Reporting Errors: Learning from Report Addenda

Background The addition of new information to a completed radiology report in the form of an "addendum" conveys a variety of information, ranging from less significant typographical errors to serious omissions and misinterpretations. Understanding the reasons for errors and their clinical implications will lead to better clinical governance and radiology practice. Aims This article assesses the common reasons which lead to addenda generation to completed reports and their clinical implications. Subjects and Methods Retrospective study was conducted by reviewing addenda to computed tomography (CT), ultrasound, and magnetic resonance imaging reports between January 2018 to June 2018, to note the frequency and classification of report addenda. Results Rate of addenda generation was 1.1% ( n = 1,076) among the 97,003 approved cross-sectional radiology reports. Errors contributed to 71.2% ( n = 767) of addenda, most commonly communication (29.3%, n = 316) and observational errors (20.8%, n = 224), and 28.7% were nonerrors aimed at providing additional clinically relevant information. Majority of the addenda (82.3%, n = 886) did not have a significant clinical impact. CT and ultrasound reports accounted for 36.9% ( n = 398) and 35.2% ( n = 379) share, respectively. A time gap of 1 to 7 days was noted for 46.8% ( n = 504) addenda and 37.6% ( n = 405) were issued in less than a day. Radiologists with more than 6-year experience created majority (1.5%, n = 456) of addenda. Those which were added to reports generated during emergency hours contributed to 23.2% ( n = 250) of the addenda. Conclusion The study has identified the prevalence of report addenda in a radiology practice involving picture archiving and communication system in a tertiary care center in India. The etiology included both errors and non-errors. Results of this audit were used to generate a checklist and put protocols that will help decrease serious radiology misses and common errors.

Visual Illusions in Radiology: Untrue Perceptions in Medical Images and Their Implications for Diagnostic Accuracy

Errors in radiologic interpretation are largely the result of failures of perception. This remains true despite the increasing use of computer-aided detection and diagnosis. We surveyed the literature on visual illusions during the viewing of radiologic images. Misperception of anatomical structures is a potential cause of error that can lead to patient harm if disease is seen when none is present. However, visual illusions can also help enhance the ability of radiologists to detect and characterize abnormalities. Indeed, radiologists have learned to exploit certain perceptual biases in diagnostic findings and as training tools. We propose that further detailed study of radiologic illusions would help clarify the mechanisms underlying radiologic performance and provide additional heuristics to improve radiologist training and reduce medical error.

Radiologists and Clinical Trials: Part 1 The Truth About Reader Disagreements

The debate over human visual perception and how medical images should be interpreted have persisted since X-rays were the only imaging technique available. Concerns over rates of disagreement between expert image readers are associated with much of the clinical research and at times driven by the belief that any image endpoint variability is problematic. The deeper understanding of the reasons, value, and risk of disagreement are somewhat siloed, leading, at times, to costly and risky approaches, especially in clinical trials. Although artificial intelligence promises some relief from mistakes, its routine application for assessing tumors within cancer trials is still an aspiration. Our consortium of international experts in medical imaging for drug development research, the Pharma Imaging Network for Therapeutics and Diagnostics (PINTAD), tapped the collective knowledge of its members to ground expectations, summarize common reasons for reader discordance, identify what factors can be controlled and which actions are likely to be effective in reducing discordance. Reinforced by an exhaustive literature review, our work defines the forces that shape reader variability. This review article aims to produce a singular authoritative resource outlining reader performance's practical realities within cancer trials, whether they occur within a clinical or an independent central review.

Radiology and patient communication: if not now, then when?

KEY POINTS

• Communication with patients in radiology is, in general, indirect using the referrer as a conduit. • Direct patient communication may be beneficial for radiology departments and radiologists to improve patient awareness about the nature of our role and also to provide correct and measured information about the nature and frequency of discrepancies in radiology.