Bias in Radiology: The How and Why of Misses and Misinterpretations

Spectrum of Cognitive Biases in Diagnostic Radiology

Cognitive biases are systematic thought processes involving the use of a filter of personal experiences and preferences arising from the tendency of the human brain to simplify information processing, especially when taking in vast amounts of data such as from imaging studies. These biases encompass a wide spectrum of thought processes and frequently overlap in their concepts, with multiple biases usually in operation when interpretive and perceptual errors occur in radiology. The authors review the gamut of cognitive biases that occur in radiology. These biases are organized according to their expected stage of occurrence while the radiologist reads and interprets an imaging study. In addition, the authors propose several additional cognitive biases that have not yet, to their knowledge, been defined in the radiologic literature but are applicable to diagnostic radiology. Case examples are used to illustrate potential biases and their impact, with emergency radiology serving as the clinical paradigm, given the associated high imaging volumes, wide diversity of imaging examinations, and rapid pace, which can further increase a radiologist's reliance on biases and heuristics. Potential strategies to recognize and overcome one's personal biases at each stage of image interpretation are also discussed. Awareness of such biases and their unintended effects on imaging interpretations and patient outcomes may help make radiologists cognizant of their own biases that can result in diagnostic errors. Identification of cognitive bias in departmental and systematic quality improvement practices may represent another tool to prevent diagnostic errors in radiology. ©RSNA, 2024 See the invited commentary by Larson in this issue.

Practical Strategies to Retain Radiologists

Since the great resignation associated with the coronavirus disease 2019 pandemic, radiology practices are now challenged with maintaining adequate radiology staffing requirements to cope with increasing clinical workload requirements. The authors describe practical strategies for radiology practice leaders to retain radiologists in the current challenging job market, while mitigating their burnout.

Outcome of patients with multivessel occlusion stroke after endovascular treatment

INTRODUCTION

Little is known about the implications of multivessel occlusions (MVO) in large vessel occlusion stroke patients who undergo endovascular treatment (EVT).

PATIENTS AND METHODS

We report data from the MR CLEAN Registry: a prospective, observational study on all stroke patients who underwent EVT in the Netherlands (March 2014-November 2017). We included patients with an intracranial target occlusion in the anterior circulation. An MVO was defined as an MCA occlusion (M1/M2) or intracranial ICA/ICA-T occlusion, with a concurrent second occlusion in the ACA or PCA territory confirmed on baseline CTA. To compare outcomes, we performed a 10:1 propensity score matching analysis with a logistic regression model including potential confounders. Outcome measures included 90-day functional outcome (modified Rankin Scale, mRS) and mortality.

RESULTS

Of 2946 included patients, 71 patients (2.4%) had an MVO (87% concurrent ACA occlusion, 10% PCA occlusion, 3% ⩾3 occlusions). These patients were matched to 71 non-MVO patients. Before matching, MVO patients had a higher baseline NIHSS (median 18 vs 16, p = 0.001) and worse collateral status (absent collaterals: 17% vs 6%, p < 0.001) compared to non-MVO patients. After matching, MVO patients had worse functional outcome at 90 days (median mRS 5 vs 3, cOR 0.39; 95%CI 0.25-0.62). Mortality was higher in MVO patients (46% vs 27%, OR 2.11, 95%CI 1.24-3.57).

DISCUSSION AND CONCLUSION

MVOs on baseline imaging were uncommon in LVO stroke patients undergoing EVT, but were associated with poor functional outcome.

Natural history and growth prediction model of pancreatic serous cystic neoplasms

OBJECTIVE

Serous cystic neoplasms (SCN) are benign pancreatic cystic neoplasms that may require resection based on local complications and rate of growth. We aimed to develop a predictive model for the growth curve of SCNs to aid in the clinical decision making of determining need for surgical resection.

METHODS

Utilizing a prospectively maintained pancreatic cyst database from a single institution, patients with SCNs were identified. Diagnosis confirmation included imaging, cyst aspiration, pathology, or expert opinion. Cyst size diameter was measured by radiology or surgery. Patients with interval imaging ≥3 months from diagnosis were included. Flexible restricted cubic splines were utilized for modeling of non-linearities in time and previous measurements. Model fitting and analysis were performed using R (V3.50, Vienna, Austria) with the rms package.

RESULTS

Among 203 eligible patients from 1998 to 2021, the mean initial cyst size was 31 mm (range 5-160 mm), with a mean follow-up of 72 months (range 3-266 months). The model effectively captured the non-linear relationship between cyst size and time, with both time and previous cyst size (not initial cyst size) significantly predicting current cyst growth (p < 0.01). The root mean square error for overall prediction was 10.74. Validation through bootstrapping demonstrated consistent performance, particularly for shorter follow-up intervals.

CONCLUSION

SCNs typically have a similar growth rate regardless of initial size. An accurate predictive model can be used to identify rapidly growing outliers that may warrant surgical intervention, and this free model (https://riskcalc.org/SerousCystadenomaSize/) can be incorporated in the electronic medical record.

Communicating difficult news in pediatric radiology

Effective communication is an important aspect of providing quality healthcare. Radiology staff may be called upon to communicate results, even when it is difficult. Simple, efficient communication models can be learned to reduce clinician distress and support patients and families through stressful times. These practical tips ensure essential communication elements are met, including assessing patient and family perspectives, clearly delivering medical information, responding with empathy, and collaborating around next steps. Models such as "Ask-Tell-Ask," "SPIKES," and "NURSE" can be used to disclose results, communicate serious news, and respond to emotion. Lastly, clinicians can also utilize "I wish… I worry… I wonder" statements to align with a patient's hopes and acknowledge a family's concerns, ultimately allowing healthcare professionals to support and guide families through challenging situations.

Understanding and Mitigating Bias in Imaging Artificial Intelligence

Artificial intelligence (AI) algorithms are prone to bias at multiple stages of model development, with potential for exacerbating health disparities. However, bias in imaging AI is a complex topic that encompasses multiple coexisting definitions. Bias may refer to unequal preference to a person or group owing to preexisting attitudes or beliefs, either intentional or unintentional. However, cognitive bias refers to systematic deviation from objective judgment due to reliance on heuristics, and statistical bias refers to differences between true and expected values, commonly manifesting as systematic error in model prediction (ie, a model with output unrepresentative of real-world conditions). Clinical decisions informed by biased models may lead to patient harm due to action on inaccurate AI results or exacerbate health inequities due to differing performance among patient populations. However, while inequitable bias can harm patients in this context, a mindful approach leveraging equitable bias can address underrepresentation of minority groups or rare diseases. Radiologists should also be aware of bias after AI deployment such as automation bias, or a tendency to agree with automated decisions despite contrary evidence. Understanding common sources of imaging AI bias and the consequences of using biased models can guide preventive measures to mitigate its impact. Accordingly, the authors focus on sources of bias at stages along the imaging machine learning life cycle, attempting to simplify potentially intimidating technical terminology for general radiologists using AI tools in practice or collaborating with data scientists and engineers for AI tool development. The authors review definitions of bias in AI, describe common sources of bias, and present recommendations to guide quality control measures to mitigate the impact of bias in imaging AI. Understanding the terms featured in this article will enable a proactive approach to identifying and mitigating bias in imaging AI. Published under a CC BY 4.0 license. Test Your Knowledge questions for this article are available in the supplemental material. See the invited commentary by Rouzrokh and Erickson in this issue.

Endovascular treatment of patients with stroke caused by anterior cerebral artery occlusions

BACKGROUND

Occlusion of the anterior cerebral artery (ACA) is uncommon but may lead to significant disability. The benefit of endovascular treatment (EVT) for ACA occlusions remains uncertain.

METHODS

We included patients treated with EVT and compared patients with ACA occlusions with patients who had internal carotid artery (ICA) or proximal (M1/M2) middle cerebral artery (MCA) occlusions from the MR CLEAN Registry. Primary outcome was the modified Rankin Scale score (mRS). Secondary outcomes were functional independence (mRS 0-2), National Institutes of Health Stroke Scale (NIHSS) score, delta-NIHSS (baseline minus NIHSS score at 24-48 h), and successful recanalization (expanded thrombolysis in cerebral infarction (eTICI) score 2b-3). Safety outcomes were symptomatic intracranial hemorrhage (sICH), periprocedural complications, and mortality.

RESULTS

Of 5193 patients, 11 (0.2%) had primary ACA occlusions. Median NIHSS at baseline was lower in patients with ACA versus ICA/MCA occlusions (11, IQR 9-14; versus 15, IQR 11-19). Functional outcome did not differ from patients with ICA/MCA occlusions. Functional independence was 4/11 (36%) in patients with ACA versus 1949/4815 (41%) in ICA/MCA occlusions; median delta-NIHSS was - 1 (IQR - 7 to 2) and - 4 (IQR - 9 to 0), respectively. Successful recanalization was 4/9 (44%), versus 3083/4787 (64%) in ICA/MCA occlusions. Mortality was 3/11 (27%) versus 1263/4815 (26%). One patient with ACA occlusion had sICH; no other complications occurred.

CONCLUSION

In this cohort ACA occlusions were uncommon. Functional outcome did not differ between patients with ACA occlusions and ICA/MCA occlusions. Prospective research is needed to determine feasibility, safety, and outcomes of EVT for ACA occlusions.

Analysis of coping capacities and cognitive biases of novice drivers-A questionnaire-based study

Coping capacity is a key aspect of driver-vehicle interaction when drivers observe and make decisions, and is of great importance for drivers. However, different drivers have different self-cognition and assess their driving abilities differently, especially for novice drivers. Based on questionnaire data, this study has investigated the coping capacities of drivers in both static environments and dynamic environments. With the ANOVA analysis method and the structural equation model (SEM), this study has verified the effects of gender and driving factors (driving years, driving frequency, driving time) on drivers' coping capacities based on drivers' self-assessment scores and mutual assessment scores. Drivers' self-assessment scores show significant effects of all factors on drivers' coping capacities, and drivers' mutual assessment scores show significant effects of all factors, excluding driving time, on drivers' coping capacities. Also, it has been found that all drivers in the driving year group have cognitive biases. It seems that first-year drivers are always overconfident with their driving skills, while drivers with a driving experience of more than three years usually score driving skills of themselves and other drivers most conservatively. With increased exposure to various traffic conditions, experienced drivers are more aware of their limitations in dealing with complex traffic situations, while novice drivers do not know their lack of capability to properly respond to any unexpected situation they could encounter.

Development of a Multimodal Machine Learning-Based Prognostication Model for Traumatic Brain Injury Using Clinical Data and Computed Tomography Scans: A CENTER-TBI and CINTER-TBI Study

Computed tomography (CT) is an important imaging modality for guiding prognostication in patients with traumatic brain injury (TBI). However, because of the specialized expertise necessary, timely and dependable TBI prognostication based on CT imaging remains challenging. This study aimed to enhance the efficiency and reliability of TBI prognostication by employing machine learning (ML) techniques on CT images. A retrospective analysis was conducted on the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) data set (n = 1016). An ML-driven binary classifier was developed to predict favorable or unfavorable outcomes at 6 months post-injury. The prognostic performance was assessed using the area under the curve (AUC) over fivefold cross-validation and compared with conventional models that depend on clinical variables and CT scoring systems. An external validation was performed using the Comparative Indian Neurotrauma Effectiveness Research in Traumatic Brain Injury (CINTER-TBI) data set (n = 348). The developed model achieved superior performance without the necessity for manual CT assessments (AUC = 0.846 [95% CI: 0.843-0.849]) compared with the model based on the clinical and laboratory variables (AUC = 0.817 [95% CI: 0.814-0.820]) and established CT scoring systems requiring manual interpretations (AUC = 0.829 [95% CI: 0.826-0.832] for Marshall and 0.838 [95% CI: 0.835-0.841] for International Mission for Prognosis and Analysis of Clinical Trials in TBI [IMPACT]). The external validation demonstrated the prognostic capacity of the developed model to be significantly better (AUC = 0.859 [95% CI: 0.857-0.862]) than the model using clinical variables (AUC = 0.809 [95% CI: 0.798-0.820]). This study established an ML-based model that provides efficient and reliable TBI prognosis based on CT scans, with potential implications for earlier intervention and improved patient outcomes.

Machine Learning and Bias in Medical Imaging: Opportunities and Challenges

Bias in health care has been well documented and results in disparate and worsened outcomes for at-risk groups. Medical imaging plays a critical role in facilitating patient diagnoses but involves multiple sources of bias including factors related to access to imaging modalities, acquisition of images, and assessment (ie, interpretation) of imaging data. Machine learning (ML) applied to diagnostic imaging has demonstrated the potential to improve the quality of imaging-based diagnosis and the precision of measuring imaging-based traits. Algorithms can leverage subtle information not visible to the human eye to detect underdiagnosed conditions or derive new disease phenotypes by linking imaging features with clinical outcomes, all while mitigating cognitive bias in interpretation. Importantly, however, the application of ML to diagnostic imaging has the potential to either reduce or propagate bias. Understanding the potential gain as well as the potential risks requires an understanding of how and what ML models learn. Common risks of propagating bias can arise from unbalanced training, suboptimal architecture design or selection, and uneven application of models. Notwithstanding these risks, ML may yet be applied to improve gain from imaging across all 3A's (access, acquisition, and assessment) for all patients. In this review, we present a framework for understanding the balance of opportunities and challenges for minimizing bias in medical imaging, how ML may improve current approaches to imaging, and what specific design considerations should be made as part of efforts to maximize the quality of health care for all.

Study Smarter: Applying the Science of Learning to Radiology

Lifelong learning is critical to a successful career in radiology, but many learners use inefficient or ineffective studying strategies developed from personal experience. Decades of cognitive psychology research have identified several techniques that consistently improve knowledge consolidation and retrieval. To improve their knowledge and ultimately patient care, radiologists should strive to understand and adopt these learning techniques. The first part of this article reviews several evidence-based learning principles, including active retrieval and the testing effect, spaced repetition, interleaving, deliberate practice, and growth mind-set. The second part provides practical suggestions on how to incorporate these principles into radiology learning, both during training and beyond.

Post-mortem CT in the investigation of homicides

AIM

To investigate the reliability of post-mortem computed tomography (PMCT) in a case series of homicides involving blunt-force, sharp-force, and ballistic trauma.

MATERIALS AND METHODS

The study investigates 16 homicide cases that underwent PMCT before autopsy. Two radiologists assessed the PMCT examinations and the data were compared to the forensic pathology findings. Data were organised in broad categories: foreign bodies, external injuries, soft-tissue and organ injuries, fractures, air in cavities, fluid collections, random pathology, and wound track. Findings were organised by systems: head and neck, thorax, abdomen and pelvis, extremities. Cohen's kappa statistics were used to assess observer agreement.

RESULTS

Six gunshot-related homicides (37.5%), seven sharp-force-related homicides (43.75%), two blunt-force-related deaths (12.5%), and one homicide due to mechanical asphyxia (1.25%) were analysed. A total of 64 fractures were reported by the pathologists, 67 by radiologist 1 and 68 by radiologist 2. Agreement was deemed substantial in all cases. Pathologists failed to report gas in cavities while radiologists underreported superficial injuries.

CONCLUSION

An overall observation was that less accurate findings were produced by the blinded radiologist in comparison to the non-blinded one. The extremeness of homicides obscured the interpretation of PMCT leading to the observed discrepancies. The combination of PMCT and autopsies is deemed optimal when investigating homicidal events.

Association between clinical history in the radiographic request and diagnostic accuracy of thorax radiographs in dogs: A retrospective case-control study

BACKGROUND

The effect of clinical history on the interpretation of radiographs has been widely researched in human medicine. There is, however, no data on this topic in veterinary medicine.

HYPOTHESIS/OBJECTIVES

Diagnostic accuracy would improve when history was supplied.

ANIMALS

Thirty client-owned dogs with abnormal findings on thoracic radiographs and confirmation of the disease, and 30 healthy client-owned controls were drawn retrospectively.

METHODS

Retrospective case-control study. Sixty radiographic studies of the thorax were randomized and interpreted by 6 radiologists; first, with no access to the clinical information; and a second time with access to all pertinent clinical information and signalment.

RESULTS

A significant increase in diagnostic accuracy was noted when clinical information was provided (64.4% without and 75.2% with clinical information; P = .002). There was no significant difference in agreement between radiologists when comparing no clinical information and with clinical information (Kappa 0.313 and 0.300, respectively).

CONCLUSIONS AND CLINICAL IMPORTANCE

The addition of pertinent clinical information to the radiographic request significantly improves the diagnostic accuracy of thorax radiographs of dogs and is recommended as standard practice.

Deep learning for the detection of benign and malignant pulmonary nodules in non-screening chest CT scans

BACKGROUND

Outside a screening program, early-stage lung cancer is generally diagnosed after the detection of incidental nodules in clinically ordered chest CT scans. Despite the advances in artificial intelligence (AI) systems for lung cancer detection, clinical validation of these systems is lacking in a non-screening setting.

METHOD

We developed a deep learning-based AI system and assessed its performance for the detection of actionable benign nodules (requiring follow-up), small lung cancers, and pulmonary metastases in CT scans acquired in two Dutch hospitals (internal and external validation). A panel of five thoracic radiologists labeled all nodules, and two additional radiologists verified the nodule malignancy status and searched for any missed cancers using data from the national Netherlands Cancer Registry. The detection performance was evaluated by measuring the sensitivity at predefined false positive rates on a free receiver operating characteristic curve and was compared with the panel of radiologists.

RESULTS

On the external test set (100 scans from 100 patients), the sensitivity of the AI system for detecting benign nodules, primary lung cancers, and metastases is respectively 94.3% (82/87, 95% CI: 88.1-98.8%), 96.9% (31/32, 95% CI: 91.7-100%), and 92.0% (104/113, 95% CI: 88.5-95.5%) at a clinically acceptable operating point of 1 false positive per scan (FP/s). These sensitivities are comparable to or higher than the radiologists, albeit with a slightly higher FP/s (average difference of 0.6).

CONCLUSIONS

The AI system reliably detects benign and malignant pulmonary nodules in clinically indicated CT scans and can potentially assist radiologists in this setting.

Diagnostic error and bias in the department of radiology: a pictorial essay

Diagnostic imaging is an essential and indispensable part of medical diagnosis and treatment, and diagnostic errors or biases are also common in the department of radiology, sometimes even having a severe impact on the diagnosis and treatment of patients. There are various reasons for diagnostic errors or biases in imaging. In this review, we analyze and summarize the causes of diagnostic imaging errors and biases based on practical cases. We propose solutions for dealing with diagnostic imaging errors and reducing their probability, thereby helping radiologists in their clinical practice.Critical relevance statement Diagnostic errors or bias contribute to most medical errors in the radiology department. Solutions for dealing with diagnostic imaging errors are pivotal for patients.Key points• Diagnostic errors or bias contribute to most medical errors in radiology department.• Solutions for dealing with diagnostic imaging errors are pivotal for patients.• This review summarizes the causes of diagnostic errors and offers solutions to them.

Self-supervised-RCNN for medical image segmentation with limited data annotation

Many successful methods developed for medical image analysis based on machine learning use supervised learning approaches, which often require large datasets annotated by experts to achieve high accuracy. However, medical data annotation is time-consuming and expensive, especially for segmentation tasks. To overcome the problem of learning with limited labeled medical image data, an alternative deep learning training strategy based on self-supervised pretraining on unlabeled imaging data is proposed in this work. For the pretraining, different distortions are arbitrarily applied to random areas of unlabeled images. Next, a Mask-RCNN architecture is trained to localize the distortion location and recover the original image pixels. This pretrained model is assumed to gain knowledge of the relevant texture in the images from the self-supervised pretraining on unlabeled imaging data. This provides a good basis for fine-tuning the model to segment the structure of interest using a limited amount of labeled training data. The effectiveness of the proposed method in different pretraining and fine-tuning scenarios was evaluated based on the Osteoarthritis Initiative dataset with the aim of segmenting effusions in MRI datasets of the knee. Applying the proposed self-supervised pretraining method improved the Dice score by up to 18% compared to training the models using only the limited annotated data. The proposed self-supervised learning approach can be applied to many other medical image analysis tasks including anomaly detection, segmentation, and classification.

Anatomical study of the medial calcaneal nerve using high-resolution ultrasound

OBJECTIVES

To determine whether high-resolution ultrasound (US) can identify the course and relations of the medial calcaneal nerve (MCN).

METHODS

This investigation was initially undertaken in eight cadaveric specimens and followed by a high-resolution US study in 20 healthy adult volunteers (40 nerves) by two musculoskeletal radiologists in consensus. The location and course of the MCN as well as its relationship to adjacent anatomical structures were evaluated.

RESULTS

The MCN was consistently identified by US along its entire course. The mean cross-sectional area of the nerve was 1 mm2 (range 0.5-2). The level at which the MCN branched from the tibial nerve was variable, located a mean of 7 mm (range - 7-60) proximal to the tip of the medial malleolus. At the level of the medial retromalleolar fossa, the MCN was located inside the proximal tarsal tunnel a mean of 8 mm (range 0-16) posterior to the medial malleolus. More distally, the nerve was depicted in the subcutaneous tissue at the surface of the abductor hallucis fascia with a mean direct distance to the fascia of 1.5 mm (range 0.4-2.8).

CONCLUSIONS

High-resolution US can identify the MCN at the level of the medial retromalleolar fossa, as well as more distally in the subcutaneous tissue at the surface of the abductor hallucis fascia. In the setting of heel pain, precise sonographic mapping of the MCN course may enable the radiologist to make diagnosis of nerve compression or neuroma, and perform selective US-guided treatments.

CLINICAL RELEVANCE STATEMENT

In the setting of heel pain, sonography is an attractive tool for diagnosing compression neuropathy or neuroma of the medial calcaneal nerve, and enables the radiologist to perform selective image-guided treatments such as diagnostic blocks and injections.

KEY POINTS

• The MCN is a small cutaneous nerve which rises from the tibial nerve in the medial retromalleolar fossa to the medial side of the heel. • The MCN can be depicted by high-resolution ultrasound along its entire course. • In the setting of heel pain, precise sonographic mapping of the MCN course may enable the radiologist to make diagnosis of neuroma or nerve entrapment, and perform selective ultrasound-guided treatments such as steroid injection or tarsal tunnel release.

Analysis of perceptual errors in skull-base pathology

PURPOSE

Medical errors result in significant mortality and morbidity. The purpose of this study is to analyze skull-base errors at a single tertiary institution, identify common anatomic sites of errors, and offer strategies to reduce errors in this region.

METHODS

A Neuroradiology Quality Assurance Database of radiologic errors was searched for attending physician computer tomography and magnetic resonance imaging errors in skull-base pathology from 2014 to 2020. Data were limited to CT and MRI reports. Errors were separated into four subcategories (tumor, trauma, vascular, and congenital) and further divided by relevant anatomic site.

RESULTS

A total of 90 skull-based errors were identified. Most errors were perceptual (87%), with common study types including MRI Brain (39%) and CT Head (24%). Most common errors were tumors (55%), followed by trauma (24%), vascular (10%), and congenital (7%). Six anatomic sites were identified and encompassed over half of errors (58%): sella, occipital bone, cerebellopontine angle/internal auditory canal (CPA/IAC), foramen magnum and clivus, cavernous sinus, and dural venous sinus.

SUMMARY

Most of the skull-base errors were perceptual. Placing a strong emphasis on both the pathology and closely examining its critical anatomic site (sella, occipital bone, CPA/IAC, foramen magnum and clivus, cavernous sinus, and dural venous sinus) could potentially reduce up to 60% of errors in these regions.

Evaluation of CI electrode position from imaging: comparison of an automated technique with the established manual method

BACKGROUND

A manual evaluation of the CI electrode position from CT and DVT scans may be affected by diagnostic errors due to cognitive biases. The aim of this study was to compare the CI electrode localization using an automated method (image-guided cochlear implant programming, IGCIP) with the clinically established manual method.

METHODS

This prospective experimental study was conducted on a dataset comprising N=50 subjects undergoing cochlear implantation with a Nucleus® CI532 or CI632 Slim Modiolar electrode. Scalar localization, electrode-to-modiolar axis distances (EMD) and angular insertion depth (aDOI) were compared between the automated IGCIP tool and the manual method. Two raters made the manual measurements, and the interrater reliability (±1.96·SD) was determined as the reference for the method comparison. The method comparison was performed using a correlation analysis and a Bland-Altman analysis.

RESULTS

Concerning the scalar localization, all electrodes were localized both manually and automatically in the scala tympani. The interrater differences ranged between ±0.2 mm (EMD) and ±10° (aDOI). There was a bias between the automatic and manual method in measuring both localization parameters, which on the one hand was smaller than the interrater variations. On the other hand, this bias depended on the magnitude of the EMD respectively aDOI. A post-hoc analysis revealed that the deviations between the methods were likely due to a different selection of mid-modiolar axis.

CONCLUSIONS

The IGCIP is a promising tool for automated processing of CT and DVT scans and has useful functionality such as being able to segment the cochlear using post-operative scans. When measuring EMD, the IGCIP tool is superior to the manual method because the smallest possible distance to the axis is determined depending on the cochlear turn, whereas the manual method selects the helicotrema as the reference point rigidly. Functionality to deal with motion artifacts and measurements of aDOI according to the consensus approach are necessary, otherwise the IGCIP is not unrestrictedly ready for clinical use.

Investigating the impact of cognitive biases in radiologists' image interpretation: A scoping review

RATIONALE AND OBJECTIVE

Image interpretation is a fundamental aspect of radiology. The treatment and management of patients relies on accurate and timely imaging diagnosis. However, errors in radiological reports can negatively impact on patient health outcomes. These misdiagnoses can be caused by several different errors, but cognitive biases account for 74 % of all image interpretation errors. There are many biases that can impact on a radiologist's perception and cognitive processes. Several recent narrative reviews have discussed these cognitive biases and have offered possible strategies to mitigate their effects. However, these strategies remain untested. Therefore, the purpose of this scoping review is to evaluate the current knowledge on the extent that cognitive biases impact on medical image interpretation.

MATERIAL AND METHODS

Scopus and Medline Databases were searched using relevant keywords to identify papers published between 2012 and 2022. A subsequent hand search of the narrative reviews was also performed. All studies collected were screened and assessed against the inclusion and exclusion criteria.

RESULTS

Twenty-four publications were included and categorised into five main themes: satisfaction of search, availability bias, hindsight bias, framing bias and other biases. From these studies, there were mixed results regarding the impact of cognitive biases, highlighting the need for further investigation in this area. Moreover, the limited and untested debiasing methods offered by a minority of the publications and narrative reviews also suggests the need for further research. The potential of role of artificial intelligence is also highlighted to further assist radiologists in identifying and mitigating these cognitive biases.

CONCLUSION

Cognitive biases can impact radiologists' image interpretation, however the effectiveness of debiasing strategies remain largely untested.

Impact of Shift Volume on Neuroradiology Diagnostic Errors at a Large Tertiary Academic Center

BACKGROUND AND PURPOSE

Medical errors can result in significant morbidity and mortality. The goal of our study is to evaluate correlation between shift volume and errors made by attending neuroradiologists at an academic medical center, using a large data set.

MATERIALS AND METHODS

CT and MRI reports from our Neuroradiology Quality Assurance database (years 2014 - 2020) were searched for attending physician errors. Data were collected on shift volume, category of missed findings, error type, interpretation setting, exam type, clinical significance.

RESULTS

654 reports contained diagnostic error. There was a significant difference between mean volume of interpreted studies on shifts when an error was made compared with shifts in which no error was documented (46.58 (SD=22.37) vs 34.09 (SD=18.60), p<0.00001); and between shifts when perceptual error was made compared with shifts when interpretive errors were made (49.50 (SD=21.9) vs 43.26 (SD=21.75), p=0.0094). 59.6% of errors occurred in the emergency/inpatient setting, 84% were perceptual and 91.1% clinically significant. Categorical distribution of errors was: vascular 25.8%, brain 23.4%, skull base 13.8%, spine 12.4%, head/neck 11.3%, fractures 10.2%, other 3.1%. Errors were detected most often on brain MRI (25.4%), head CT (18.7%), head/neck CTA (13.8%), spine MRI (13.7%).

CONCLUSION

Errors were associated with higher volume shifts, were primarily perceptual and clinically significant. We need National guidelines establishing a range of what is a safe number of interpreted cross-sectional studies per day.

Unsupervised abnormality detection in neonatal MRI brain scans using deep learning

Analysis of 3D medical imaging data has been a large topic of focus in the area of Machine Learning/Artificial Intelligence, though little work has been done in algorithmic (particularly unsupervised) analysis of neonatal brain MRI's. A myriad of conditions can manifest at an early age, including neonatal encephalopathy (NE), which can result in lifelong physical consequences. As such, there is a dire need for better biomarkers of NE and other conditions. The objective of the study is to improve identification of anomalies and prognostication of neonatal MRI brain scans. We introduce a framework designed to support the analysis and assessment of neonatal MRI brain scans, the results of which can be used as an aid to neuroradiologists. We explored the efficacy of the framework through iterations of several deep convolutional Autoencoder (AE) unsupervised modeling architectures designed to learn normalcy of the neonatal brain structure. We tested this framework on the developing human connectome project (dHCP) dataset with 97 patients that were previously categorized by severity. Our framework demonstrated the model's ability to identify and distinguish subtle morphological signatures present in brain structures. Normal and abnormal neonatal brain scans can be distinguished with reasonable accuracy, correctly categorizing them in up to 83% of cases. Most critically, new brain anomalies originally missed during the radiological reading were identified and corroborated by a neuroradiologist. This framework and our modeling approach demonstrate an ability to improve prognostication of neonatal brain conditions and are able to localize new anomalies.

Observer agreement in single computerized tomography use for diagnosing paediatric head and neck malignancies at Uganda Cancer Institute

BACKGROUND

In the Ugandan setting, investigation for PHNM with CT uses a protocol with both unenhanced and contrast enhanced procedures hence doubling the ionizing radiation exposure. The purpose of this study was to determine the feasibility of single CT procedures in diagnosing PHNM.

METHODS

This was a cross-sectional study using CT images from patients, aged fifteen years and below, investigated for head and neck malignancies at the Uganda Cancer Institute. Three radiologists, observers A, B and C, with 12, 5 and 2 years of experience, respectively, participated in the study. They independently reported contrast enhanced images (protocol A), unenhanced images (protocol B), then both unenhanced and contrast enhanced images (protocol C) in 2 months intervals. Inter- and intra- observer agreement was determined using Gwen's Agreement coefficient.

RESULTS

Seventy-three CT scans of 36 boys and 37 girls, with a median age of 9 (3-13) years, were used. Intra-and inter-observer agreement on primary tumour location ranged from substantial to almost perfect with the highest intra-observer agreement observed when protocols A and C were compared. Inter-observer agreement for tumour calcifications was substantial for protocol A. Observers A and C demonstrated an almost perfect intra-observer agreement when protocols A and C were compared. There was a substantial inter-observer agreement on diagnosis for all protocols.

CONCLUSIONS

In our setting and examining a limited number of CT images, we demonstrated that contrast-enhanced CT scans provide sufficient information with no evidence of additional value of unenhanced images. Using contrast-enhanced images alone reduced the radiation exposure significantly.

Why don't we inform patients about the risk of diagnostic errors?

The principles of autonomy and informed consent dictate that patients who undergo a radiological examination should actually be informed about the risk of diagnostic errors. Implementing such a policy could potentially increase the quality of care. However, due to the vast number of radiological examinations that are performed in each hospital each day, financial constraints, and the risk of losing trust, patients, and income if the requirement for informed consent is not imposed by law on a state or national level, it may be challenging to inform patients about the risk of diagnostic errors. Future research is necessary to determine if and how an informed consent procedure for diagnostic errors can be implemented in clinical practice.

Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses

A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis, going beyond the most likely options. We introduce a new task, "less likely brainstorming," that asks a model to generate outputs that humans think are relevant but less likely to happen. We explore the task in two settings: a brain MRI interpretation generation setting and an everyday commonsense reasoning setting. We found that a baseline approach of training with less likely hypotheses as targets generates outputs that humans evaluate as either likely or irrelevant nearly half of the time; standard MLE training is not effective. To tackle this problem, we propose a controlled text generation method that uses a novel contrastive learning strategy to encourage models to differentiate between generating likely and less likely outputs according to humans. We compare our method with several state-of-the-art controlled text generation models via automatic and human evaluations and show that our models' capability of generating less likely outputs is improved.

Under-reporting of forensic findings: craniocervical emergency imaging in cases of survived hanging

To determine the diagnostic bias between clinical and forensic radiology in cases of nonfatal hanging and determine and describe typical underreported imaging findings. In a retrospective, single-center study, all patients admitted for attempted suicide with near-hanging or fatal hanging between January 2008 and December 2020 who received CT or MRI of head and neck were reviewed and missed findings in the original report were documented. A binary regression with disagreement as dependent variable was fitted for the imaging modality, fatality, age, and sex. A total of 123 hanging incidents were retrospectively analyzed. The vast majority (n = 108; 87.8%) had attempted suicide with a nonfatal outcome. Fatal outcome occurred in 15 (12.0%). The extra- and intracranial injuries documented on CT and MRI scans were laryngeal (n = 8; 6.5%), soft tissue (n = 42; 34.1%), and vascular injuries (n = 1; 0.8%). Intracranial pathology was evident on 18 (14.6%) scans. Disagreement occurred in 36 (29.3%) cases and represented 52 (69.2%) of all cases with a radiological finding. Disagreement was strongly associated with fatality (OR: 2.7-44.9.4, p = 0.0012). In most cases, nonfatal hangings cause no or only minor injuries. Fatal cases are associated with a greater probability of missed minor imaging findings. This suggests that findings deemed clinically irrelevant are probably not reported in such severe emergency cases. This association indicates that minor abnormalities are underreported when major pathologies are evident on imaging in victims of strangulation.

A Multicenter Assessment of Interreader Reliability of LI-RADS Version 2018 for MRI and CT

Background Various limitations have impacted research evaluating reader agreement for Liver Imaging Reporting and Data System (LI-RADS). Purpose To assess reader agreement of LI-RADS in an international multicenter multireader setting using scrollable images. Materials and Methods This retrospective study used deidentified clinical multiphase CT and MRI and reports with at least one untreated observation from six institutions and three countries; only qualifying examinations were submitted. Examination dates were October 2017 to August 2018 at the coordinating center. One untreated observation per examination was randomly selected using observation identifiers, and its clinically assigned features were extracted from the report. The corresponding LI-RADS version 2018 category was computed as a rescored clinical read. Each examination was randomly assigned to two of 43 research readers who independently scored the observation. Agreement for an ordinal modified four-category LI-RADS scale (LR-1, definitely benign; LR-2, probably benign; LR-3, intermediate probability of malignancy; LR-4, probably hepatocellular carcinoma [HCC]; LR-5, definitely HCC; LR-M, probably malignant but not HCC specific; and LR-TIV, tumor in vein) was computed using intraclass correlation coefficients (ICCs). Agreement was also computed for dichotomized malignancy (LR-4, LR-5, LR-M, and LR-TIV), LR-5, and LR-M. Agreement was compared between research-versus-research reads and research-versus-clinical reads. Results The study population consisted of 484 patients (mean age, 62 years ± 10 [SD]; 156 women; 93 CT examinations, 391 MRI examinations). ICCs for ordinal LI-RADS, dichotomized malignancy, LR-5, and LR-M were 0.68 (95% CI: 0.61, 0.73), 0.63 (95% CI: 0.55, 0.70), 0.58 (95% CI: 0.50, 0.66), and 0.46 (95% CI: 0.31, 0.61) respectively. Research-versus-research reader agreement was higher than research-versus-clinical agreement for modified four-category LI-RADS (ICC, 0.68 vs 0.62, respectively; P = .03) and for dichotomized malignancy (ICC, 0.63 vs 0.53, respectively; P = .005), but not for LR-5 (P = .14) or LR-M (P = .94). Conclusion There was moderate agreement for LI-RADS version 2018 overall. For some comparisons, research-versus-research reader agreement was higher than research-versus-clinical reader agreement, indicating differences between the clinical and research environments that warrant further study. © RSNA, 2023 Supplemental material is available for this article. See also the editorials by Johnson and Galgano and Smith in this issue.

MAVRIC based T2 mapping assessment of infrapatellar fat pad scarring in patients with total knee arthroplasty

The infrapatellar fat pad (IPFP) has been implicated as a source of postoperative knee pain. Imaging the IPFP is challenging in patients with total knee arthroplasty (TKA) due to metallic susceptibility artifact. Multi-Acquisition Variable-Resonance Image Combination (MAVRIC)-based T2 Mapping has been developed to mitigate this artifact and can generate quantitative T2 data. Objectives of this study were to (1) measure T2 values of the IPFP in patients with TKAs using a MAVRIC based T2 mapping technique and (2) determine if IPFP T2 values are related to the degree of fat pad scarring or clinical magnetic resonance imaging (MRI) findings. Twenty-eight subjects (10 males, 18 females, Age: 66 + 7.2 years [Mean ± standard deviations]) undergoing clinical MRIs were sequentially recruited. Morphological imaging and quantitative T2 mapping sequences were performed on a clinical 1.5 T scanner. The morphologic images were graded for the presence and severity of fat pad scarring and clinical outcomes. T2 values were calculated in the total fat pad volume, a normal regions of interest (ROI), and an abnormal ROI. T2 values were shortened in the total IPFP volume (p = 0.001) and within abnormal regions (p = 0.003) in subjects with more severe IPFP scarring. The difference between T2 values in normal-abnormal regions was greater in subjects with severe versus no scarring (+1426.1%, p = 0.008). T2 values were elevated in patients with MRI findings of osteolysis (+32.3%, p = 0.02). These findings indicate that MAVRIC-based T2 Mapping may be used as a quantitative biomarker of postoperative IPFP scarring in individuals following TKA.

Reliability of radiologists' first impression when interpreting a screening mammogram

Previous studies showed that radiologists can detect the gist of an abnormality in a mammogram based on a half-second image presentation through global processing of screening mammograms. This study investigated the intra- and inter-observer reliability of the radiologists' initial impressions about the abnormality (or "gist signal"). It also examined if a subset of radiologists produced more reliable and accurate gist signals. Thirty-nine radiologists provided their initial impressions on two separate occasions, viewing each mammogram for half a second each time. The intra-class correlation (ICC) values showed poor to moderate intra-reader reliability. Only 13 radiologists had an ICC of 0.6 or above, which is considered the minimum standard for reliability, and only three radiologists had an ICC exceeding 0.7. The median value for the weighted Cohen's Kappa was 0.478 (interquartile range = 0.419-0.555). The Mann-Whitney U-test showed that the "Gist Experts", defined as those who outperformed others, had significantly higher ICC values (p = 0.002) and weighted Cohen's Kappa scores (p = 0.026). However, even for these experts, the intra-radiologist agreements were not strong, as an ICC of at least 0.75 indicates good reliability and the signal from none of the readers reached this level of reliability as determined by ICC values. The inter-reader reliability of the gist signal was poor, with an ICC score of 0.31 (CI = 0.26-0.37). The Fleiss Kappa score of 0.106 (CI = 0.105-0.106), indicating only slight inter-reader agreement, confirms the findings from the ICC analysis. The intra- and inter-reader reliability analysis showed that the radiologists' initial impressions are not reliable signals. In particular, the absence of an abnormal gist does not reliably signal a normal case, so radiologists should keep searching. This highlights the importance of "discovery scanning," or coarse screening to detect potential targets before ending the visual search.

Evidence of a cognitive bias in the quantification of COVID-19 with CT: an artificial intelligence randomised clinical trial

Chest computed tomography (CT) has played a valuable, distinct role in the screening, diagnosis, and follow-up of COVID-19 patients. The quantification of COVID-19 pneumonia on CT has proven to be an important predictor of the treatment course and outcome of the patient although it remains heavily reliant on the radiologist's subjective perceptions. Here, we show that with the adoption of CT for COVID-19 management, a new type of psychophysical bias has emerged in radiology. A preliminary survey of 40 radiologists and a retrospective analysis of CT data from 109 patients from two hospitals revealed that radiologists overestimated the percentage of lung involvement by 10.23 ± 4.65% and 15.8 ± 6.6%, respectively. In the subsequent randomised controlled trial, artificial intelligence (AI) decision support reduced the absolute overestimation error (P < 0.001) from 9.5% ± 6.6 (No-AI analysis arm, n = 38) to 1.0% ± 5.2 (AI analysis arm, n = 38). These results indicate a human perception bias in radiology that has clinically meaningful effects on the quantitative analysis of COVID-19 on CT. The objectivity of AI was shown to be a valuable complement in mitigating the radiologist's subjectivity, reducing the overestimation tenfold.Trial registration: https://Clinicaltrial.gov . Identifier: NCT05282056, Date of registration: 01/02/2022.

Diagnostic errors in musculoskeletal oncology and possible mitigation strategies

The objective of this paper is to explore sources of diagnostic error in musculoskeletal oncology and potential strategies for mitigating them using case examples. As musculoskeletal tumors are often obvious, the diagnostic errors in musculoskeletal oncology are frequently cognitive. In our experience, the most encountered cognitive biases in musculoskeletal oncologic imaging are as follows: (1) anchoring bias, (2) premature closure, (3) hindsight bias, (4) availability bias, and (5) alliterative bias. Anchoring bias results from failing to adjust an early impression despite receiving additional contrary information. Premature closure is the cognitive equivalent of "satisfaction of search." Hindsight bias occurs when we retrospectively overestimate the likelihood of correctly interpreting the examination prospectively. In availability bias, the radiologist judges the probability of a diagnosis based on which diagnosis is most easily recalled. Finally, alliterative bias occurs when a prior radiologist's impression overly influences the diagnostic thinking of another radiologist on a subsequent exam. In addition to cognitive biases, it is also important for radiologists to acknowledge their feelings when making a diagnosis to recognize positive and negative impact of affect on decision making. While errors decrease with radiologist experience, the lack of application of medical knowledge is often the primary source of error rather than a deficiency of knowledge, emphasizing the need to foster clinical reasoning skills and assist cognition. Possible solutions for reducing error exist at both the individual and the system level and include (1) improvement in knowledge and experience, (2) improvement in clinical reasoning and decision-making skills, and (3) improvement in assisting cognition.

Reporting and management of incidental lung findings on computed tomography: beyond lung nodules

Non-nodular incidental lung findings can broadly be categorised as airway- or airspace-related abnormalities and diffuse parenchymal abnormalities. Airway-related abnormalities include bronchial dilatation and thickening, foci of low attenuation, emphysema, and congenital variants. Diffuse parenchymal abnormalities relate to the spectrum of diffuse parenchymal lung diseases cover a spectrum from interstitial lung abnormalities (ILAs) and pulmonary cysts to established diffuse parenchymal lung abnormalities such as the idiopathic interstitial pneumonias and cystic lung diseases. In this review, we discuss the main manifestations of these incidental findings, paying attention to their prevalence and importance, descriptors to use when reporting, the limits of what can be considered "normal", and conclude each section with some pragmatic reporting recommendations. We also highlight technical and patient factors which can lead to spurious abnormalities.

Audit of Prior Screening Mammograms of Screen-Detected Cancers: Implications for the Delay in Breast Cancer Detection

When cancer is detected in a screening mammogram, on occasion retrospective review of prior screening (pre-index) mammograms indicates a likely presence of cancer. These missed cancers during pre-index screens constitute a delay in detection and diagnosis. This study was undertaken to quantify the missed cancer rate by auditing pre-index screens to improve the quality of mammography screening practice. From a cohort of 135 screen-detected cancers, 120 pre-index screening mammograms could be retrieved and served as the study sample. A consensus read by 2 radiologists who interpreted the pre-index screens in an unblinded manner with full knowledge of cancer location, cancer type, lesion type, and pathology served as the truth or reference standard. Five radiologists interpreted the pre-index screens in a blinded manner. Established performance metrics such as sensitivity and specificity were quantified for each reader in interpreting these pre-index screens in a blinded manner. All five radiologists detected lesions in 8/120 (6.7%) screens. Excluding the 2 readers whose performance was close to random, all the 3 remaining readers detected lesions in 13 pre-index screens. This indicates that there is a delay in diagnosis by at least one cycle from 8/120 (6.7%) to 13/120 (10.8%). There were no observable trends in terms of either the cancer type or the lesion type. Auditing prior screening mammograms in screen-detected cancers can help in identifying the proportion of cases that were missed during interpretation and help in quantifying the delay in breast cancer detection.

The effect of clinical history on diagnostic performance of endodontic cone-beam CT interpretation

AIM

To assess the effect of clinical history on the interpretation of endodontic disease in dental cone-beam computed tomography (CBCT).

MATERIALS AND METHODS

A reader performance study of an image test-set was undertaken using a factorial, free-response, crossover design, accounting for the independent variables: case type, case severity, reader type, and reading modality. Twenty-three readers interpreted 60 CBCT images twice over two reading sessions using a balanced design, once with access to clinical history and once without, where 30 in each session included history. Lesion localisations, specificity, false-positive marks and the weighted alternative free-response receiver operating characteristic (wAFROC1) figure of merit were calculated.

RESULTS

Clinical history had no significant effect on specificity and false-positive rates in non-diseased cases (p>0.05), but improved lesion localisation in subtle and obvious diseased cases (p<0.01). wAFROC1 values were higher with clinical history for subtle (0.58 versus 0.48; p<0.001) and obvious (0.77 versus 0.71; p=0.006) diseased categories. No associations were observed between clinical history and both readers' years of experience and reading volume in the non-diseased categories. Readers with fewer (p=0.03) and moderate (p=0.008) years of experience and low (p=0.002) CBCT reading volume demonstrated better lesion localisation in subtle diseased cases when clinical history was available.

CONCLUSIONS

Clinical history improved the interpretation of CBCT images with disease without affecting the interpretation of images without disease. Less and moderately experienced readers and low-volume readers benefitted more from availability of clinical history.

Diagnostic Accuracy of AI for Opportunistic Screening of Abdominal Aortic Aneurysm in CT: A Systematic Review and Narrative Synthesis

In this review, we focused on the applicability of artificial intelligence (AI) for opportunistic abdominal aortic aneurysm (AAA) detection in computed tomography (CT). We used the academic search system PubMed as the primary source for the literature search and Google Scholar as a supplementary source of evidence. We searched through 2 February 2022. All studies on automated AAA detection or segmentation in noncontrast abdominal CT were included. For bias assessment, we developed and used an adapted version of the QUADAS-2 checklist. We included eight studies with 355 cases, of which 273 (77%) contained AAA. The highest risk of bias and level of applicability concerns were observed for the "patient selection" domain, due to the 100% pathology rate in the majority (75%) of the studies. The mean sensitivity value was 95% (95% CI 100-87%), the mean specificity value was 96.6% (95% CI 100-75.7%), and the mean accuracy value was 95.2% (95% CI 100-54.5%). Half of the included studies performed diagnostic accuracy estimation, with only one study having data on all diagnostic accuracy metrics. Therefore, we conducted a narrative synthesis. Our findings indicate high study heterogeneity, requiring further research with balanced noncontrast CT datasets and adherence to reporting standards in order to validate the high sensitivity value obtained.

U"AI" Testing: User Interface and Usability Testing of a Chest X-ray AI Tool in a Simulated Real-World Workflow

Purpose: To observe interactions of practicing radiologists with a chest x-ray AI tool and evaluate its usability and impact on workflow efficiency. Methods: Using a simulated clinical workflow and remote multi-monitor screensharing, we prospectively assessed the interactions of 10 staff radiologists (5-33 years of experience) with a PACS-embedded, regulatory-approved chest x-ray AI tool. Qualitatively, we collected feedback using a think-aloud method and post-testing semi-structured interview; transcript themes were categorized by: (1) AI tool features, (2) deployment considerations, and (3) broad human-AI interactions. Quantitatively, we used time-stamped video recordings to compare reporting and decision-making efficiency with and without AI assistance. Results: For AI tool features, radiologists appreciated the simple binary classification (normal vs abnormal) and found the heatmap essential to understand what the AI considered abnormal; users were uncertain of how to interpret confidence values. Regarding deployment considerations, radiologists thought the tool would be especially helpful for identifying subtle diagnoses; opinions were mixed on whether the tool impacted perceived efficiency, accuracy, and confidence. Considering general human-AI interactions, radiologists shared concerns about automation bias especially when relying on an automated triage function. Regarding decision-making and workflow efficiency, participants began dictating 5 seconds later (42% increase, P = .02) and took 14 seconds longer to complete cases (33% increase, P = .09) with AI assistance. Conclusions: Radiologist usability testing provided insights into effective AI tool features, deployment considerations, and human-AI interactions that can guide successful AI deployment. Early AI adoption may increase radiologists' decision-making and total reporting time but improves with experience.

Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center

BACKGROUND AND PURPOSE

Diagnostic errors affect 2%-8% of neuroradiology studies, resulting in significant potential morbidity and mortality. This retrospective analysis of a large database at a single tertiary academic institution focuses on diagnostic misses in cerebrovascular pathology and suggests error-reduction strategies.

MATERIALS AND METHODS

CT and MR imaging reports from a consecutive database spanning 2015-2020 were searched for errors of attending physicians in cerebrovascular pathology. Data were collected on missed findings, study types, and interpretation settings. Errors were categorized as ischemic, arterial, venous, hemorrhagic, and "other."

RESULTS

A total of 245,762 CT and MR imaging neuroradiology examinations were interpreted during the study period. Vascular diagnostic errors were present in 165 reports, with a mean of 49.6 (SD, 23.3) studies on the shifts when an error was made, compared with 34.9 (SD, 19.2) on shifts without detected errors (P < .0001). Seventy percent of examinations occurred in the hospital setting; 93.3% of errors were perceptual; 6.7% were interpretive; and 93.9% (n = 155) were clinically significant (RADPEER 2B or 3B). The distribution of errors was arterial and ischemic each with 33.3%, hemorrhagic with 21.8%, and venous with 7.5%. Most errors involved brain MR imaging (30.3%) followed by head CTA (27.9%) and noncontrast head CT (26.1%). The most common misses were acute/subacute infarcts (25.1%), followed by aneurysms (13.7%) and subdural hematomas (9.7%).

CONCLUSIONS

Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.

Histopathological diagnosis of cutaneous melanocytic lesions: blinded and nonblinded second opinions offer similar improvement in diagnostic accuracy

BACKGROUND

Previous studies of second opinions in the diagnosis of melanocytic skin lesions have examined blinded second opinions, which do not reflect usual clinical practice. The current study, conducted in the USA, investigated both blinded and nonblinded second opinions for their impact on diagnostic accuracy.

METHODS

In total, 100 melanocytic skin biopsy cases, ranging from benign to invasive melanoma, were interpreted by 74 dermatopathologists. Subsequently, 151 dermatopathologists performed nonblinded second and third reviews. We compared the accuracy of single reviewers, second opinions obtained from independent, blinded reviewers and second opinions obtained from sequential, nonblinded reviewers. Accuracy was defined with respect to a consensus reference diagnosis.

RESULTS

The mean case-level diagnostic accuracy of single reviewers was 65.3% (95% CI 63.4-67.2%). Second opinions arising from sequential, nonblinded reviewers significantly improved accuracy to 69.9% (95% CI 68.0-71.7%; P < 0.001). Similarly, second opinions arising from blinded reviewers improved upon the accuracy of single reviewers (69.2%; 95% CI 68.0-71.7%). Nonblinded reviewers were more likely than blinded reviewers to give diagnoses in the same diagnostic classes as the first diagnosis. Nonblinded reviewers tended to be more confident when they agreed with previous reviewers, even with inaccurate diagnoses.

CONCLUSION

We found that both blinded and nonblinded second reviewers offered a similar modest improvement in diagnostic accuracy compared with single reviewers. Obtaining second opinions with knowledge of previous reviews tends to generate agreement among reviews, and may generate unwarranted confidence in an inaccurate diagnosis. Combining aspects of both blinded and nonblinded review in practice may leverage the advantages while mitigating the disadvantages of each approach. Specifically, a second pathologist could give an initial diagnosis blinded to the results of the first pathologist, with subsequent nonblinded discussion between the two pathologists if their diagnoses differ.

Possible Bias in Supervised Deep Learning Algorithms for CT Lung Nodule Detection and Classification

Simple Summary

Artificial Intelligence (AI) algorithms can assist clinicians in their daily tasks by automatically detecting and/or classifying nodules in chest CT scans. Bias of such algorithms is one of the reasons why implementation of them in clinical practice is still not widely adopted. There is no published review on the bias that these algorithms may contain. This review aims to present different types of bias in such algorithms and present possible ways to mitigate them. Only then it would be possible to ensure that these algorithms work as intended under many different clinical settings.

Abstract

Artificial Intelligence (AI) algorithms for automatic lung nodule detection and classification can assist radiologists in their daily routine of chest CT evaluation. Even though many AI algorithms for these tasks have already been developed, their implementation in the clinical workflow is still largely lacking. Apart from the significant number of false-positive findings, one of the reasons for that is the bias that these algorithms may contain. In this review, different types of biases that may exist in chest CT AI nodule detection and classification algorithms are listed and discussed. Examples from the literature in which each type of bias occurs are presented, along with ways to mitigate these biases. Different types of biases can occur in chest CT AI algorithms for lung nodule detection and classification. Mitigation of them can be very difficult, if not impossible to achieve completely.

Influence of Prior Imaging Information on Diagnostic Accuracy for Focal Skeletal Processes—A Retrospective Analysis of the Consistency between Biopsy-Verified Imaging Diagnoses

Introduction: Comparing imaging examinations with those previously obtained is considered mandatory in imaging guidelines. To our knowledge, no studies are available on neither the influence, nor the sequence, of prior imaging and reports on diagnostic accuracy using biopsy as the reference standard. Such data are important to minimize diagnostic errors and to improve the preparation of diagnostic imaging guidelines. The aim of our study was to provide such data. Materials and methods: A retrospective cohort of 216 consecutive skeletal biopsies from patients with at least 2 different imaging modalities (X-ray, CT and MRI) performed within 6 months of biopsy was identified. The diagnostic accuracy of the individual imaging modality was assessed. Finally, the possible influence of the sequence of imaging modalities was investigated. Results: No significant difference in the accuracy of the imaging modalities was shown, being preceded by another imaging modality or not. However, the sequence analyses indicate sequential biases, particularly if MRI was the first imaging modality. Conclusion: The sequence of the imaging modalities seems to influence the diagnostic accuracy against a pathology reference standard. Further studies are needed to establish evidence-based guidelines for the strategy of using previous imaging and reports to improve diagnostic accuracy.

MRA-detected intracranial atherosclerotic disease in patients with TIA and minor stroke

OBJECTIVES

Patients with TIA and minor stroke commonly undergo CT and CTA in the emergency department with subsequent MRI with MRA for further workup. The purpose of this study was to review outpatient MRIs for TIA/stroke patients to assess the additional benefit, if any, of the MRA sequence in the detection of intracranial atherosclerotic disease in patients for whom CTA had already been performed.

METHODS

The radiology reports of outpatient MRIs of the brain for TIA/minor stroke patients were retrospectively reviewed via the hospital PACS system. Following this, the imaging report from the patient's initial presentation to the emergency department was reviewed. This index imaging and subsequent MRI were compared to assess the incidence of new vascular findings detected on the MRA sequence in patients for whom CTA had already been performed. Where new lesions had been identified at follow-up, the imaging was retroactively reviewed to assess if they were present on the index imaging.

RESULTS

Two hundred seven consecutive patients were reviewed. Significant (> 50%) intracranial atherosclerotic disease was present on MRA in 18 patients (8.7%). This was a new finding in 11 patients. Five had initial CTA where the atherosclerosis was not detected. All 5 of these cases were located in the posterior cerebral arteries. Incidental aneurysms were seen in 14 (6.7%); 12 were a new finding at time of MRI.

CONCLUSION

The MRA sequence provides additional value by increasing the detection of clinically important intracranial atherosclerotic disease which may inform management in patients with minor stroke and TIA.

Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists

RATIONALE AND OBJECTIVES

Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers.

MATERIALS AND METHODS

This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012-October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1).

RESULTS

Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%-18% (Chi-square 45.69, p < 0.00001); mass 16%-44% (Chi-square 498.42, p < 0.00001); asymmetry 13%-47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%-20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%-8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%-15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%-8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%-5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%-10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer.

CONCLUSION

Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

Impact of explainable artificial intelligence assistance on clinical decision-making of novice dental clinicians

Objective

Despite artificial intelligence (AI) being used increasingly in healthcare, implementation challenges exist leading to potential biases during the clinical decision process of the practitioner. The interaction of AI with novice clinicians was investigated through an identification task, an important component of diagnosis, in dental radiography. The study evaluated the performance, efficiency, and confidence level of dental students on radiographic identification of furcation involvement (FI), with and without AI assistance.

Materials and Methods

Twenty-two third- and 19 fourth-year dental students (DS3 and DS4, respectively) completed remotely administered surveys to identify FI lesions on a series of dental radiographs. The control group received radiographs without AI assistance while the test group received the same radiographs and AI-labeled radiographs. Data were appropriately analyzed using the Chi-square, Fischer's exact, analysis of variance, or Kruskal-Wallis tests.

Results

Performance between groups with and without AI assistance was not statistically significant except for 1 question where tendency was to err with AI-generated answer (P < .05). The efficiency of task completion and confidence levels was not statistically significant between groups. However, both groups with and without AI assistance believed the use of AI would improve the clinical decision-making.

Discussion

Dental students detecting FI in radiographs with AI assistance had a tendency towards over-reliance on AI.

Conclusion

AI input impacts clinical decision-making, which might be particularly exaggerated in novice clinicians. As it is integrated into routine clinical practice, caution must be taken to prevent overreliance on AI-generated information.

Enhancing medical-imaging artificial intelligence through holistic use of time-tested key imaging and clinical parameters: Future insights

Much of the published literature in Radiology-related Artificial Intelligence (AI) focuses on single tasks, such as identifying the presence or absence or severity of specific lesions. Progress comparable to that achieved for general-purpose computer vision has been hampered by the unavailability of large and diverse radiology datasets containing different types of lesions with possibly multiple kinds of abnormalities in the same image. Also, since a diagnosis is rarely achieved through an image alone, radiology AI must be able to employ diverse strategies that consider all available evidence, not just imaging information. Using key imaging and clinical signs will help improve their accuracy and utility tremendously. Employing strategies that consider all available evidence will be a formidable task; we believe that the combination of human and computer intelligence will be superior to either one alone. Further, unless an AI application is explainable, radiologists will not trust it to be either reliable or bias-free; we discuss some approaches aimed at providing better explanations, as well as regulatory concerns regarding explainability (“transparency”). Finally, we look at federated learning, which allows pooling data from multiple locales while maintaining data privacy to create more generalizable and reliable models, and quantum computing, still prototypical but potentially revolutionary in its computing impact.