Accuracy of Diagnostic Procedures: Has It Improved Over the Past Five Decades?

Let us talk about mistakes

Unfortunately, errors and mistakes are part of life. Errors and mistakes can harm patients and incur unplanned costs. Errors may arise from various sources, which may be classified as systematic, latent, or active. Intrinsic and extrinsic factors also contribute to incorrect decisions. In addition to cognitive biases, our personality, socialization, personal chronobiology, and way of thinking (heuristic versus analytical) are influencing factors. Factors such as overload from private situations, long commuting times, and the complex environment of information technology must also be considered. The objective of this paper is to define and classify errors and mistakes in radiology, to discuss the influencing factors, and to present strategies for prevention. Hierarchical responsibilities and team "well-being" are also discussed.

Applications and potential of machine, learning augmented chest X-ray interpretation in cardiology

The chest X-ray (CXR) has a wide range of clinical indications in the field of cardiology, from the assessment of acute pathology to disease surveillance and screening. Despite many technological advancements, CXR interpretation error rates have remained constant for decades. The application of machine learning has the potential to substantially improve clinical workflow efficiency, pathology detection accuracy, error rates and clinical decision making in cardiology. To date, machine learning has been developed to improve image processing, facilitate pathology detection, optimize the clinical workflow, and facilitate risk stratification. This review explores the current and potential future applications of machine learning for chest radiography to facilitate clinical decision making in cardiology. It maps the current state of the science and considers additional potential use cases from the perspective of clinicians and technologists actively engaged in the development and deployment of deep learning driven clinical decision support systems.

Application of deep learning algorithms in classification and localization of implant cutout for the postoperative hip

OBJECTIVE

This study aims to explore the feasibility of employing convolutional neural networks for detecting and localizing implant cutouts on anteroposterior pelvic radiographs.

MATERIALS AND METHODS

The research involves the development of two Deep Learning models. Initially, a model was created for image-level classification of implant cutouts using 40191 pelvic radiographs obtained from a single institution. The radiographs were partitioned into training, validation, and hold-out test datasets in a 6/2/2 ratio. Performance metrics including the area under the receiver operator characteristics curve (AUROC), sensitivity, and specificity were calculated using the test dataset. Additionally, a second object detection model was trained to localize implant cutouts within the same dataset. Bounding box visualizations were generated on images predicted as cutout-positive by the classification model in the test dataset, serving as an adjunct for assessing algorithm validity.

RESULTS

The classification model had an accuracy of 99.7%, sensitivity of 84.6%, specificity of 99.8%, AUROC of 0.998 (95% CI: 0.996, 0.999) and AUPRC of 0.774 (95% CI: 0.646, 0.880). From the pelvic radiographs predicted as cutout-positive, the object detection model could achieve 95.5% localization accuracy on true positive images, but falsely generated 14 results from the 15 false-positive predictions.

CONCLUSION

The classification model showed fair accuracy for detection of implant cutouts, while the object detection model effectively localized cutout. This serves as proof of concept of using a deep learning-based approach for classification and localization of implant cutouts from pelvic radiographs.

Deep learning improves physician accuracy in the comprehensive detection of abnormalities on chest X-rays

Chest X-rays are the most commonly performed medical imaging exam, yet they are often misinterpreted by physicians. Here, we present an FDA-cleared, artificial intelligence (AI) system which uses a deep learning algorithm to assist physicians in the comprehensive detection and localization of abnormalities on chest X-rays. We trained and tested the AI system on a large dataset, assessed generalizability on publicly available data, and evaluated radiologist and non-radiologist physician accuracy when unaided and aided by the AI system. The AI system accurately detected chest X-ray abnormalities (AUC: 0.976, 95% bootstrap CI: 0.975, 0.976) and generalized to a publicly available dataset (AUC: 0.975, 95% bootstrap CI: 0.971, 0.978). Physicians showed significant improvements in detecting abnormalities on chest X-rays when aided by the AI system compared to when unaided (difference in AUC: 0.101, p < 0.001). Non-radiologist physicians detected abnormalities on chest X-ray exams as accurately as radiologists when aided by the AI system and were faster at evaluating chest X-rays when aided compared to unaided. Together, these results show that the AI system is accurate and reduces physician errors in chest X-ray evaluation, which highlights the potential of AI systems to improve access to fast, high-quality radiograph interpretation.

Application of artificial intelligence in lung cancer screening: A real-world study in a Chinese physical examination population

BACKGROUND

With the rapid increase of chest computed tomography (CT) images, the workload faced by radiologists has increased dramatically. It is undeniable that the use of artificial intelligence (AI) image-assisted diagnosis system in clinical treatment is a major trend in medical development. Therefore, in order to explore the value and diagnostic accuracy of the current AI system in clinical application, we aim to compare the detection and differentiation of benign and malignant pulmonary nodules between AI system and physicians, so as to provide a theoretical basis for clinical application.

METHODS

Our study encompassed a cohort of 23 336 patients who underwent chest low-dose spiral CT screening for lung cancer at the Health Management Center of West China Hospital. We conducted a comparative analysis between AI-assisted reading and manual interpretation, focusing on the detection and differentiation of benign and malignant pulmonary nodules.

RESULTS

The AI-assisted reading exhibited a significantly higher screening positive rate and probability of diagnosing malignant pulmonary nodules compared with manual interpretation (p < 0.001). Moreover, AI scanning demonstrated a markedly superior detection rate of malignant pulmonary nodules compared with manual scanning (97.2% vs. 86.4%, p < 0.001). Additionally, the lung cancer detection rate was substantially higher in the AI reading group compared with the manual reading group (98.9% vs. 90.3%, p < 0.001).

CONCLUSIONS

Our findings underscore the superior screening positive rate and lung cancer detection rate achieved through AI-assisted reading compared with manual interpretation. Thus, AI exhibits considerable potential as an adjunctive tool in lung cancer screening within clinical practice settings.

Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluation

OBJECTIVES

To evaluate an artificial intelligence (AI)-assisted double reading system for detecting clinically relevant missed findings on routinely reported chest radiographs.

METHODS

A retrospective study was performed in two institutions, a secondary care hospital and tertiary referral oncology centre. Commercially available AI software performed a comparative analysis of chest radiographs and radiologists' authorised reports using a deep learning and natural language processing algorithm, respectively. The AI-detected discrepant findings between images and reports were assessed for clinical relevance by an external radiologist, as part of the commercial service provided by the AI vendor. The selected missed findings were subsequently returned to the institution's radiologist for final review.

RESULTS

In total, 25,104 chest radiographs of 21,039 patients (mean age 61.1 years ± 16.2 [SD]; 10,436 men) were included. The AI software detected discrepancies between imaging and reports in 21.1% (5289 of 25,104). After review by the external radiologist, 0.9% (47 of 5289) of cases were deemed to contain clinically relevant missed findings. The institution's radiologists confirmed 35 of 47 missed findings (74.5%) as clinically relevant (0.1% of all cases). Missed findings consisted of lung nodules (71.4%, 25 of 35), pneumothoraces (17.1%, 6 of 35) and consolidations (11.4%, 4 of 35).

CONCLUSION

The AI-assisted double reading system was able to identify missed findings on chest radiographs after report authorisation. The approach required an external radiologist to review the AI-detected discrepancies. The number of clinically relevant missed findings by radiologists was very low.

CLINICAL RELEVANCE STATEMENT

The AI-assisted double reader workflow was shown to detect diagnostic errors and could be applied as a quality assurance tool. Although clinically relevant missed findings were rare, there is potential impact given the common use of chest radiography.

KEY POINTS

• A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions. • Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations. • Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist's reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance.

Assessment of the comparative agreement between chest radiographs and CT scans in intensive care units

PURPOSE

Chest radiographs in critically ill patients can be difficult to interpret due to technical and clinical factors. We sought to determine the agreement of chest radiographs and CT scans, and the inter-observer variation of chest radiograph interpretation, in intensive care units (ICUs).

METHODS

Chest radiographs and corresponding thoracic computerised tomography (CT) scans (as reference standard) were collected from 45 ICU patients. All radiographs were analysed by 20 doctors (radiology consultants, radiology trainees, ICU consultants, ICU trainees) from 4 different centres, blinded to CT results. Specificity/sensitivity were determined for pleural effusion, lobar collapse and consolidation/atelectasis. Separately, Fleiss' kappa for multiple raters was used to determine inter-observer variation for chest radiographs.

RESULTS

The median sensitivity and specificity of chest radiographs for detecting abnormalities seen on CTs scans were 43.2% and 85.9% respectively. Diagnostic sensitivity for pleural effusion was significantly higher among radiology consultants but no specialty/experience distinctions were observed for specificity. Median inter-observer kappa coefficient among assessors was 0.295 ("fair").

CONCLUSIONS

Chest radiographs commonly miss important radiological features in critically ill patients. Inter-observer agreement in chest radiograph interpretation is only "fair". Consultant radiologists are least likely to miss thoracic radiological abnormalities. The consequences of misdiagnosis by chest radiographs remain to be determined.

Screening for lung cancer using thin-slice low-dose computed tomography in southwestern China: a population-based real-world study

OBJECTIVES

Lung cancer is one of the most common malignant tumors threatening human life and health. At present, low-dose computed tomography (LDCT) screening for the high-risk population to achieve early diagnosis and treatment of lung cancer has become the first choice recommended by many authoritative international medical organizations. To further optimize the lung cancer screening method, we conducted a real-world study of LDCT lung cancer screening in a large sample of a healthy physical examination population, comparing differences in lung nodules and lung cancer detection between thin and thick-slice LDCT scanning.

METHODS

A total of 29 296 subjects who underwent low-dose thick-slice CT scanning (5 mm thickness) from January 2015 to December 2015 and 28 058 subjects who underwent low-dose thin-slice CT scanning (1 mm thickness) from January 2018 to December 2018 in West China Hospital were included. The positive detection rate, detection rate of lung cancer, pathological stage of lung cancer, and mortality rate of lung cancer were analyzed and compared between the two groups.

RESULTS

The positive rate of LDCT screening in the thin-slice scanning group was significantly higher than that in the thick-slice scanning group (20.1% vs. 14.4%, p < 0.001). In addition, the lung cancer detection rate in the thin-slice LDCT screening positive group was significantly higher than that in the thick-slice scanning group (78.0% vs. 52.9%, p < 0.001).

CONCLUSIONS

The screening positive rate of low-dose thin-slice CT scanning is higher and more early-stage lung cancer (IA1 stage) can be detected in the screen-positive group.

Influence of Prior Imaging Review on Recommendations for Additional Diagnostic Testing: Retrospective Analysis of Imaging Reports in a Chiropractic Radiology Practice

OBJECTIVE

The purpose of this study was to assess the influence of prior imaging review on recommendations for additional diagnostic testing in an academic chiropractic radiology practice. A secondary aim was to explore the influence of prior imaging review on radiographic interpretation.

METHODS

We retrospectively reviewed radiology reports generated from July 18, 2022, to July 18, 2023, from the Palmer College of Chiropractic main campus (Davenport, Iowa) clinic system. Imaging interpretation included an automated search for prior images in an internal picture archival and communication system (PACS). Images from regional health system databases were available and sought by radiologists when (1) unclear radiologic findings had potential clinical implications or (2) prior imaging could clarify potential problems detected in a clinical history. Data were abstracted to a secure adaptive electronic questionnaire and analyzed descriptively.

RESULTS

We reviewed 1712 radiographic and 165 musculoskeletal diagnostic ultrasound reports for 1552 unique individuals (811 [52.3%] females and 741 [47.7%] males) with a mean age of 42.1 years (range, 2-93 years). Prior imaging was described in 417 (22.2%) reports; 246 (58.9%) indicated images from internal PACS, 192 (46.0%) indicated images from external PACS, and 21 noted both internal and external PACS. Prior imaging findings were credited with answering a clinical question in 98 (23.5%), and a radiographic question in 228 (54.7%) of 417 reports. The process negated the need for follow-up diagnostic testing recommendations in 119 (28.5%) instances, leading to additional imaging recommendations in 19 (4.6%).

CONCLUSION

Data obtained in this study suggest that comparing current and previous imaging may help reduce unnecessary additional imaging or follow-up diagnostic testing recommendations. Prior imaging review may also facilitate diagnostic confidence and interpretation clarity.

Fatigue and vigilance in medical experts detecting breast cancer

An abundance of laboratory-based experiments has described a vigilance decrement of reducing accuracy to detect targets with time on task, but there are few real-world studies, none of which have previously controlled the environment to control for bias. We describe accuracy in clinical practice for 360 experts who examined >1 million women's mammograms for signs of cancer, whilst controlling for potential biases. The vigilance decrement pattern was not observed. Instead, test accuracy improved over time, through a reduction in false alarms and an increase in speed, with no significant change in sensitivity. The multiple-decision model explains why experts miss targets in low prevalence settings through a change in decision threshold and search quit threshold and propose it should be adapted to explain these observed patterns of accuracy with time on task. What is typically thought of as standard and robust research findings in controlled laboratory settings may not directly apply to real-world environments and instead large, controlled studies in relevant environments are needed.

Diagnostic error and bias in the department of radiology: a pictorial essay

Diagnostic imaging is an essential and indispensable part of medical diagnosis and treatment, and diagnostic errors or biases are also common in the department of radiology, sometimes even having a severe impact on the diagnosis and treatment of patients. There are various reasons for diagnostic errors or biases in imaging. In this review, we analyze and summarize the causes of diagnostic imaging errors and biases based on practical cases. We propose solutions for dealing with diagnostic imaging errors and reducing their probability, thereby helping radiologists in their clinical practice.Critical relevance statement Diagnostic errors or bias contribute to most medical errors in the radiology department. Solutions for dealing with diagnostic imaging errors are pivotal for patients.Key points• Diagnostic errors or bias contribute to most medical errors in radiology department.• Solutions for dealing with diagnostic imaging errors are pivotal for patients.• This review summarizes the causes of diagnostic errors and offers solutions to them.

Serial dependence in perception across naturalistic generative adversarial network-generated mammogram

Purpose

Human perception and decisions are biased toward previously seen stimuli. This phenomenon is known as serial dependence and has been extensively studied for the last decade. Recent evidence suggests that clinicians' judgments of mammograms might also be impacted by serial dependence. However, the stimuli used in previous psychophysical experiments on this question, consisting of artificial geometric shapes and healthy tissue backgrounds, were unrealistic. We utilized realistic and controlled generative adversarial network (GAN)-generated radiographs to mimic images that clinicians typically encounter.

Approach

Mammograms from the digital database for screening mammography (DDSM) were utilized to train a GAN. This pretrained GAN was then adopted to generate a large set of authentic-looking simulated mammograms: 20 circular morph continuums, each with 147 images, for a total of 2940 images. Using these stimuli in a standard serial dependence experiment, participants viewed a random GAN-generated mammogram on each trial and subsequently matched the GAN-generated mammogram encountered using a continuous report. The characteristics of serial dependence from each continuum were analyzed.

Results

We found that serial dependence affected the perception of all naturalistic GAN-generated mammogram morph continuums. In all cases, the perceptual judgments of GAN-generated mammograms were biased toward previously encountered GAN-generated mammograms. On average, perceptual decisions had 7% categorization errors that were pulled in the direction of serial dependence.

Conclusions

Serial dependence was found even in the perception of naturalistic GAN-generated mammograms created by a GAN. This supports the idea that serial dependence could, in principle, contribute to decision errors in medical image perception tasks.

Veterinary radiologic error rate as determined by necropsy

A large-scale postmortem auditing of antemortem imaging diagnoses has yet to be accomplished in veterinary medicine. For this retrospective, observational, single-center, diagnostic accuracy study, necropsy reports for patients of The Schwarzman Animal Medical Center were collected over a 1-year period. Each necropsy diagnosis was determined to be either correctly diagnosed or discrepant with its corresponding antemortem diagnostic imaging, and discrepancies were categorized. The radiologic error rate was calculated to include only clinically significant missed diagnoses (lesion was not reported but was retrospectively visible on the image) and misinterpretations (lesion was noted but was incorrectly diagnosed). Nonerror discrepancies, such as temporal indeterminacy, microscopic limitations, sensitivity limitations, and study-type limitations were not included in the error rate. A total of 1099 necropsy diagnoses had corresponding antemortem imaging; 440 diagnoses were classified as major diagnoses, of which 176 were discrepant, for a major discrepancy rate of 40%, similar to reports in people. Seventeen major discrepancies were diagnoses that were missed or misinterpreted by the radiologist, for a calculated radiologic error rate of 4.6%, comparable with error rates of 3%-5% reported in people. From 2020 to 2021, nearly half of all clinically significant abnormalities noted at necropsy went undetected by antemortem imaging, though most discrepancies owed to factors other than radiologic error. Identifying common patterns of misdiagnosis and discrepancy will help radiologists refine their analysis of imaging studies to potentially reduce interpretive error.

Clinical and medico-legal considerations in endodontics

Endodontic treatment can be challenging for a number of reasons, including the microscopic nature of the clinical environment, reliance on tactile sensation and lack of direct visualization of the work being performed. Commonly, endodontic patients present with pain and distress, which can exacerbate an already difficult clinical situation. Complications may might arise prior to, or during treatment, despite practising with the utmost care and skill. Preventing and managing these complications can take considerable time and energy, and oftentimes assistance from or referral to more experienced colleagues is required. The aim of this review is to discuss medico-legal considerations in endodontics, with clinical correlations and a focus on the Australian legal landscape. [Correction added on 18 October 2023, after first online publication: The abstract was amended from a structured to an unstructured abstract.].

Commonly Missed Findings on Chest Radiographs: Causes and Consequences

Chest radiography (CXR) continues to be the most frequently performed imaging examination worldwide, yet it remains prone to frequent errors in interpretation. These pose potential adverse consequences to patients and are a leading motivation for medical malpractice lawsuits. Commonly missed CXR findings and the principal causes of these errors are reviewed and illustrated. Perceptual errors are the predominant source of these missed findings. The medicolegal implications of such errors are explained. Awareness of commonly missed CXR findings, their causes, and their consequences are important in developing approaches to reduce and mitigate these errors.

Factors affecting interpretation of dental radiographs

OBJECTIVES

To identify the factors influencing errors in the interpretation of dental radiographs.

METHODS

A protocol was registered on Prospero. All studies published until May 2022 were included in this review. The search of the electronic databases spanned Ovid Medline, PubMed, EMBASE, Web of Science and Scopus. The quality of the studies was assessed using the MMAT tool. Due to the heterogeneity of the included studies, a meta-analysis was not conducted.

RESULTS

The search yielded 858 articles, of which eight papers met the inclusion and exclusion criteria and were included in the systematic review. These studies assessed the factors influencing the accuracy of the interpretation of dental radiographs. Six factors were identified as being significant that affected the occurrence of interpretation errors. These include clinical experience, clinical knowledge, and technical ability, case complexity, time pressure, location and duration of dental education and training and cognitive load.

CONCLUSIONS

The occurrence of interpretation errors has not been widely investigated in dentistry. The factors identified in this review are interlinked. Further studies are needed to better understand the extent of the occurrence of interpretive errors and their impact on the practice of dentistry.

Call to Action: Creating Resources for Radiology Technologists to Capture Higher Quality Portable Chest X-rays

Background Patient rotation, foreign body overlying anatomy, and anatomy out of field of view can have detrimental impacts on the diagnostic quality of portable chest x-rays (PCXRs), especially as the number of PCXR imaging increases due to the coronavirus disease 2019 (COVID-19) pandemic. Although preventable, these "quality failures" are common and may lead to interpretative and diagnostic errors for the radiologist. Aims In this study, we present a baseline quality failure rate of PCXR imaging as observed at our institution. We also conduct a focus group highlighting the key issues that lead to the problematic images and discuss potential interventions targeting technologists that can be implemented to address imaging quality failure rate. Materials and methods A total of 500 PCXRs for adult patients admitted to a large university hospital between July 12, 2021, and July 25, 2021, were obtained for evaluation of quality. The PCXRs were evaluated by radiology residents for failures in technical image quality. The images were categorized into various metrics including the degree of rotation and obstruction of anatomical structures. After collecting the data, a focus group involving six managers of the technologist department at our university hospital was conducted to further illuminate the key barriers to quality PCXRs faced at our institution.. Results  Out of the 500 PCXRs evaluated, 231 were problematic (46.2%). 43.5% of the problematic films with a repeat PCXR within one week showed that there was a technical problem impacting the ability to detect pathology. Most problematic films also occurred during the night shift (48%). Key issues that lead to poor image quality included improper patient positioning, foreign objects covering anatomy, and variances in technologists' training. Three interventions were proposed to optimize technologist performance that can lower quality failure rates of PCXRs. These include a longitudinal educational curriculum involving didactic sessions, adding nursing support to assist technologists, and adding an extra layer of verification by internal medicine residents before sending the films to the radiologist. The rationale for these interventions is discussed in detail so that a modified version can be implemented in other hospital systems.  Conclusion This study illustrates the high baseline error rate in image quality of PCXRs at our institution and demonstrates the need to improve on image quality. Poor image quality negatively impacts the interpretive accuracy of radiologists and therefore leads to wrong diagnoses. Increasing educational resources and support for technologists can lead to higher image quality and radiologist accuracy.

Diagnostic captioning: a survey

Diagnostic captioning (DC) concerns the automatic generation of a diagnostic text from a set of medical images of a patient collected during an examination. DC can assist inexperienced physicians, reducing clinical errors. It can also help experienced physicians produce diagnostic reports faster. Following the advances of deep learning, especially in generic image captioning, DC has recently attracted more attention, leading to several systems and datasets. This article is an extensive overview of DC. It presents relevant datasets, evaluation measures, and up-to-date systems. It also highlights shortcomings that hinder DC’s progress and proposes future directions.

Resident-attending discrepancy rates for two consecutive versus nonconsecutive weeks of overnight shifts

PURPOSE

Recent Accreditation Council for Graduate Medical Education policy changes no longer limit the number of consecutive night shifts allowed for trainees. Few studies have examined radiology resident overnight performance over time. This study aimed to compare significant resident-attending discrepancy rates for residents working 2 nonconsecutive versus consecutive weeks of overnight shifts. The authors hypothesized significantly increased week-two discrepancies in the consecutive group.

METHODS

For 2020, a retrospective analysis of significant overnight resident-attending discrepancy rates over a 24-week period using database searches was performed for residents self-selecting 2 nonconsecutive versus consecutive weeks. The nonconsecutive group typically had a 7-day mix of days off and day shifts between their night shift weeks. Paired and unpaired t tests were performed with p < 0.05 considered significant.

RESULTS

For the 24 sets of 2 weeks covered by two residents at a time, eight were nonconsecutive and 16 were consecutive. The nonconsecutive group had 75.0% R4 coverage compared to 37.5% for the consecutive group. There were no significant study volume differences between the groups. A total of 27,906 studies (35.3% cross-sectional [CT and MR], 54.9% radiograph plus fluoroscopy, 9.8% US) were performed with 223 discrepancies (0.80%). Overall discrepancies for the nonconsecutive versus consecutive groups were 39/4505 (0.87%) versus 59/9462 (0.62%; p = 0.32) for week one and 46/4732 (1.0%) versus 79/9207 (0.86%; p = 0.60) for week two with no significant differences between the groups by modality.

CONCLUSION

Residents self-selecting 2 consecutive weeks of overnight shifts do not have increased resident-attending discrepancy rates compared to 2 nonconsecutive weeks.

Costs Versus Quality in Imaging Examination Decisions

PURPOSE

Emerging price transparency tools allow consumers to access individualized out-of-pocket cost (OOPC) estimates, but many lack quality metrics. The aim of this study was to evaluate how potential patients weigh imaging OOPC versus measures of quality when selecting an imaging center for a hypothetical health condition (back pain).

METHODS

Surveying 1,310 Amazon Mechanical Turk volunteers, the authors evaluated how potential patients weigh MRI OOPC ($50 vs $400 vs unknown cost at the time of the examination, with billed OOPC responsibility varying between $50 and $3,500) versus service quality surrogates using three different quality indicators (examination results accuracy, physician recommendation of an imaging center on the basis of familiarity, and facility online star ratings) in their decisions when selecting a radiology center for imaging of two hypothetical clinical conditions (mild and severe back pain), using ranking-based conjoint analyses.

RESULTS

A total of 1,025 eligible respondents completed the survey. Respondents expressed higher preference for perceived quality over cost in hypothetical severe back pain scenarios, resulting in a relative importance of 65.8% (95% confidence interval [CI], 62.2%-69.4%) for improved imaging results accuracy from 87% to 96%, 63.9% (95% CI, 60.3%-67.5%) for provider recommendations of the facility, and 80.1% (95% CI, 74.2%-85.9%) for an increase in online review star ratings from 2.5 to 4.5 (out of 5) compared with an increased cost from $50 to $400. For mild back pain, there was no statistical difference in respondents' preference for perceived quality and cost.

CONCLUSIONS

Incorporating quality metrics into price transparency tools is important. Further research is needed to identify metrics that are most comparable and easily obtainable across imaging centers that remain important to patients.

Magnetic resonance imaging-related programmable ventriculoperitoneal shunt valve setting changes occur often

PURPOSE

Patients with programmable ventriculoperitoneal (VP) shunt valves undergo multiple skull radiographs to evaluate for setting changes resulting from MRI. Our purpose was to determine the rates of inadvertent, MRI-related, programmable VP shunt valve setting changes.

MATERIALS AND METHODS

In this retrospective cohort with a study period of January 2015-December 2018, we reviewed the pre- and post-MRI skull radiographs of patients with programmable VP shunts and collected the following data: Demographics, commercial type of the valve used, magnetic field strength of the MRI device used, and whether a setting change occurred. We used the chi-square test to identify variables associated with valve setting change.

RESULTS

We identified 210 MRI exposure events in 156 patients, and an MRI-related valve setting change rate of 56.7%. The setting change rate was significantly higher with higher magnetic field strength (p = 0.03), and with Medtronic Strata™ valves compared to Codman Hakim™ valves (p < 0.0001).

CONCLUSION

Inadvertent, MRI-related shunt valve setting changes are frequent with valves that lack a locking mechanism. Therefore, we suggest that when feasible, the clinicians could opt to manually reprogram the valves after the MRI to the preferred setting without the need for pre- and post-MRI radiographs. We believe that this protocol modification could help reduce ionizing radiation exposure and cost. Manufacturers may consider incorporating locking mechanisms into the design of such devices in order to reduce the unintended setting change rates.

Advancing Research on Medical Image Perception by Strengthening Multidisciplinary Collaboration

Medical image interpretation is central to detecting, diagnosing, and staging cancer and many other disorders. At a time when medical imaging is being transformed by digital technologies and artificial intelligence, understanding the basic perceptual and cognitive processes underlying medical image interpretation is vital for increasing diagnosticians' accuracy and performance, improving patient outcomes, and reducing diagnostician burnout. Medical image perception remains substantially understudied. In September 2019, the National Cancer Institute convened a multidisciplinary panel of radiologists and pathologists together with researchers working in medical image perception and adjacent fields of cognition and perception for the "Cognition and Medical Image Perception Think Tank." The Think Tank's key objectives were to identify critical unsolved problems related to visual perception in pathology and radiology from the perspective of diagnosticians, discuss how these clinically relevant questions could be addressed through cognitive and perception research, identify barriers and solutions for transdisciplinary collaborations, define ways to elevate the profile of cognition and perception research within the medical image community, determine the greatest needs to advance medical image perception, and outline future goals and strategies to evaluate progress. The Think Tank emphasized diagnosticians' perspectives as the crucial starting point for medical image perception research, with diagnosticians describing their interpretation process and identifying perceptual and cognitive problems that arise. This article reports the deliberations of the Think Tank participants to address these objectives and highlight opportunities to expand research on medical image perception.

Assessment of the effect of a comprehensive chest radiograph deep learning model on radiologist reports and patient outcomes: a real-world observational study

OBJECTIVES

Artificial intelligence (AI) algorithms have been developed to detect imaging features on chest X-ray (CXR) with a comprehensive AI model capable of detecting 124 CXR findings being recently developed. The aim of this study was to evaluate the real-world usefulness of the model as a diagnostic assistance device for radiologists.

DESIGN

This prospective real-world multicentre study involved a group of radiologists using the model in their daily reporting workflow to report consecutive CXRs and recording their feedback on level of agreement with the model findings and whether this significantly affected their reporting.

SETTING

The study took place at radiology clinics and hospitals within a large radiology network in Australia between November and December 2020.

PARTICIPANTS

Eleven consultant diagnostic radiologists of varying levels of experience participated in this study.

PRIMARY AND SECONDARY OUTCOME MEASURES

Proportion of CXR cases where use of the AI model led to significant material changes to the radiologist report, to patient management, or to imaging recommendations. Additionally, level of agreement between radiologists and the model findings, and radiologist attitudes towards the model were assessed.

RESULTS

Of 2972 cases reviewed with the model, 92 cases (3.1%) had significant report changes, 43 cases (1.4%) had changed patient management and 29 cases (1.0%) had further imaging recommendations. In terms of agreement with the model, 2569 cases showed complete agreement (86.5%). 390 (13%) cases had one or more findings rejected by the radiologist. There were 16 findings across 13 cases (0.5%) deemed to be missed by the model. Nine out of 10 radiologists felt their accuracy was improved with the model and were more positive towards AI poststudy.

CONCLUSIONS

Use of an AI model in a real-world reporting environment significantly improved radiologist reporting and showed good agreement with radiologists, highlighting the potential for AI diagnostic support to improve clinical practice.

Serial dependence in the perceptual judgments of radiologists

In radiological screening, clinicians scan myriads of radiographs with the intent of recognizing and differentiating lesions. Even though they are trained experts, radiologists’ human search engines are not perfect: average daily error rates are estimated around 3–5%. A main underlying assumption in radiological screening is that visual search on a current radiograph occurs independently of previously seen radiographs. However, recent studies have shown that human perception is biased by previously seen stimuli; the bias in our visual system to misperceive current stimuli towards previous stimuli is called serial dependence. Here, we tested whether serial dependence impacts radiologists’ recognition of simulated lesions embedded in actual radiographs. We found that serial dependence affected radiologists’ recognition of simulated lesions; perception on an average trial was pulled 13% toward the 1-back stimulus. Simulated lesions were perceived as biased towards the those seen in the previous 1 or 2 radiographs. Similar results were found when testing lesion recognition in a group of untrained observers. Taken together, these results suggest that perceptual judgements of radiologists are affected by previous visual experience, and thus some of the diagnostic errors exhibited by radiologists may be caused by serial dependence from previously seen radiographs.

Technical and interpretive pitfalls of postmortem CT: Five examples of errors revealed by autopsy

Image acquisition of dead bodies, particularly using postmortem computed tomography (PMCT), has become common in forensic investigations worldwide. Meanwhile, in countries such as Japan which have an extremely low rate of autopsy, PMCT is being increasingly used in the clinical field to certify the cause of death (COD) without performing an autopsy or toxicological tests, even in cases of unnatural death. Additionally, these PMCT images are predominantly interpreted by clinical personnel such as emergency physicians or clinicians who are not trained in PMCT interpretation and who work for the police, that is, the so-called police doctors. Many potential pitfalls associated with the use of PMCT have been previously described in textbooks and published papers, including the pitfalls of not performing a complete forensic pathology investigation, and the use of physicians without appropriate PMCT training to interpret PMCT and direct death investigation and certification. We describe five examples in which apparent misdiagnosis of COD based on PMCT misinterpretation was revealed by autopsy. Here are the five examples of errors: (1) Postmortem changes were misinterpreted as COD, (2) resuscitation effects were misinterpreted as COD, (3) COD was determined after an incomplete examination, (4) fatal findings caused by external origin were wrongly interpreted as 'of internal origin' based on PMCT, and (5) non-fatal findings on PMCT were wrongly interpreted as fatal. Interpretation of PMCT by appropriately trained physicians and an accompanying complete forensic investigation, including autopsy when indicated, is necessary to prevent significant errors in COD determination and related potential adverse medicolegal consequences.

Chest radiographs and machine learning - Past, present and future

Despite its simple acquisition technique, the chest X-ray remains the most common first-line imaging tool for chest assessment globally. Recent evidence for image analysis using modern machine learning points to possible improvements in both the efficiency and the accuracy of chest X-ray interpretation. While promising, these machine learning algorithms have not provided comprehensive assessment of findings in an image and do not account for clinical history or other relevant clinical information. However, the rapid evolution in technology and evidence base for its use suggests that the next generation of comprehensive, well-tested machine learning algorithms will be a revolution akin to early advances in X-ray technology. Current use cases, strengths, limitations and applications of chest X-ray machine learning systems are discussed.

Current Status of Radiomics and Deep Learning in Liver Imaging

ABSTRACT

Artificial intelligence is poised to revolutionize medical image. It takes advantage of the high-dimensional quantitative features present in medical images that may not be fully appreciated by humans. Artificial intelligence has the potential to facilitate automatic organ segmentation, disease detection and characterization, and prediction of disease recurrence. This article reviews the current status of artificial intelligence in liver imaging and reviews the opportunities and challenges in clinical implementation.

Characteristics of expert search behavior in volumetric medical image interpretation

Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.

Investigating errors in medical imaging: medical malpractice cases in Finland

Objective

The objectives of the study were to survey patient injury claims concerning medical imaging in Finland in 1991–2017, and to investigate the nature of the incidents, the number of claims, the reasons for the claims, and the decisions made concerning the claims.

Materials and methods

The research material consisted of patient claims concerning imaging, sent to the Finnish Patient Insurance Centre (PVK). The data contained information on injury dates, the examination code, the decision code, the description of the injury, and the medical grounds for decisions.

Results

The number of claims included in the study was 1054, and the average number per year was 87. The most common cause was delayed diagnosis (404 claims, 38.3%). Most of the claims concerned mammography (314, 29.8%), radiography (170, 16.1%), and MRI (162, 15.4%). According to the decisions made by the PVK, there were no delays in 54.6% of the examinations for which claims were made. About 30% of all patient claims received compensation, the most typical reason being medical malpractice (27.7%), followed by excessive injuries and injuries caused by infections, accidents and equipment (2.7%).

Conclusion

Patient injury in imaging examinations and interventions cannot be completely prevented. However, injury data are an important source of information for health care. By analysing claims, we can prevent harm, increase the quality of care, and improve patient safety in medical imaging.

Blind spots on CT imaging of the head: Insights from 5 years of report addenda at a single institution

BACKGROUND

Errors of detection ("misses") are the major source of error in radiology. There is sparse prior literature describing patterns of detection error on CT head imaging.

PURPOSE

The objective of this study was to gain insight to areas on CT head imaging where radiologists are most likely to miss clinically relevant findings.

METHODS

We performed a cross-sectional study of consecutive reports of CT imaging of the head at a single institution spanning 5/1/2013-5/1/2018 (5 years). Detection errors described in addenda were categorized according to anatomic location, type of pathology, and potential impact on management. Blind spots were defined by the most common sites of missed findings.

RESULTS

A total of 165,943 reports for CT head imaging were obtained. Addenda were found in 1658 (~1%) of reports, of which 359 (21.7%) described errors of detection. Within the extracranial soft tissues (n = 73) the most common "misses" were at incidentally imaged parotid glands and the frontal scalp. Within osseous structures (n = 149), blind spots included the nasal and occipital bones. Vascular lesions (n = 47) which passed detection were most common at the distal MCA, carotid terminus and sigmoid sinus/jugular bulb. No predisposition was seen for anatomic subsites within the CSF space (n = 60) and brain parenchyma (n = 65).

CONCLUSIONS

Consistent patterns of blind spots are revealed. Radiologic teaching and search patterns to account for these sites of error may accelerate trainee competence and improve accuracy in the practice of radiology.

Automatic classification of canine thoracic radiographs using deep learning

The interpretation of thoracic radiographs is a challenging and error-prone task for veterinarians. Despite recent advancements in machine learning and computer vision, the development of computer-aided diagnostic systems for radiographs remains a challenging and unsolved problem, particularly in the context of veterinary medicine. In this study, a novel method, based on multi-label deep convolutional neural network (CNN), for the classification of thoracic radiographs in dogs was developed. All the thoracic radiographs of dogs performed between 2010 and 2020 in the institution were retrospectively collected. Radiographs were taken with two different radiograph acquisition systems and were divided into two data sets accordingly. One data set (Data Set 1) was used for training and testing and another data set (Data Set 2) was used to test the generalization ability of the CNNs. Radiographic findings used as non mutually exclusive labels to train the CNNs were: unremarkable, cardiomegaly, alveolar pattern, bronchial pattern, interstitial pattern, mass, pleural effusion, pneumothorax, and megaesophagus. Two different CNNs, based on ResNet-50 and DenseNet-121 architectures respectively, were developed and tested. The CNN based on ResNet-50 had an Area Under the Receive-Operator Curve (AUC) above 0.8 for all the included radiographic findings except for bronchial and interstitial patterns both on Data Set 1 and Data Set 2. The CNN based on DenseNet-121 had a lower overall performance. Statistically significant differences in the generalization ability between the two CNNs were evident, with the CNN based on ResNet-50 showing better performance for alveolar pattern, interstitial pattern, megaesophagus, and pneumothorax.

The role of clinical history in the interpretation of chest radiographs

OBJECTIVE

This review will appraise the literature pertaining to the influences that clinical history has on the action of assessing the chest radiograph.

KEY FINDINGS

There remains conflicting evidence on the impact of clinical history on chest radiography. Some research suggests that clinical history has the potential to influence the reporter in a negative way by limiting their search strategy to a more focussed search. Image interpretation is more accurate when reporters are allowed to conduct a free search of the chest image, untainted by preconceived concepts.

CONCLUSION

Clinical history needs to be accessed appropriately to aid and not stifle accurate image interpretation. Reporters need to be aware of the potential bias clinical history can introduce to their reporting and develop strategies to alleviate this as much as possible.

IMPLICATIONS FOR PRACTICE

A greater understanding of the potential bias of clinical history on the process of image interpretation is required by all reporters. Reporters need to develop an approach and strategy when accessing clinical history. Novice reporters need to be educated regarding the impact of clinical history on their reporting.

What do radiologists look for? Advances and limitations of perceptual learning in radiologic search

Supported by guidance from training during residency programs, radiologists learn clinically relevant visual features by viewing thousands of medical images. Yet the precise visual features that expert radiologists use in their clinical practice remain unknown. Identifying such features would allow the development of perceptual learning training methods targeted to the optimization of radiology training and the reduction of medical error. Here we review attempts to bridge current gaps in understanding with a focus on computational saliency models that characterize and predict gaze behavior in radiologists. There have been great strides toward the accurate prediction of relevant medical information within images, thereby facilitating the development of novel computer-aided detection and diagnostic tools. In some cases, computational models have achieved equivalent sensitivity to that of radiologists, suggesting that we may be close to identifying the underlying visual representations that radiologists use. However, because the relevant bottom-up features vary across task context and imaging modalities, it will also be necessary to identify relevant top-down factors before perceptual expertise in radiology can be fully understood. Progress along these dimensions will improve the tools available for educating new generations of radiologists, and aid in the detection of medically relevant information, ultimately improving patient health.

Arm position on portable neonatal/infant ICU chest radiograph can mimic lamellar effusion

INTRODUCTION/BACKGROUND

Arm malposition in neonatal ICU radiographs may result in overlap of the arm soft tissues and chest wall giving the appearance of lamellar effusions. We aimed to determine the frequency of arm malposition on portable neonatal/infant intensive care unit (N/IICU) chest radiographs and the proportion of these mimicking lamellar effusions.

MATERIAL AND METHODS

We evaluated a subgroup of supine portable chest radiographs performed at the N/IICU. Two reviewers, at a tertiary pediatric hospital located in the USA, evaluated each radiograph in consensus and classified arm position for either side independently as (1) acceptable: arm abducted and separated from the chest and (2) compromised: arm down and in contact with chest soft tissue. The compromised cases were evaluated regarding any overlap between soft tissues of the arm and chest of sufficient degree to mimic a lamellar effusion.

RESULTS

We reviewed 300 radiographs performed at the N/IICU (600 hemithoraces). The mean age was 1.8 ± 1.8 months. Of 600 hemithoraces, 233 (39%) showed arms down and in contact with the chest. In seven (1%) cases, the arm position was compromised and mimicked a lamellar effusion. We identified 32 (5%) true lamellar effusions in the whole sample; in 14 of the 32 cases with lamellar effusion, the radiographs were performed with the arms down.

CONCLUSION

Portable chest radiographs performed in the N/IICU without proper arm abduction represent a potential for misinterpretation of chest radiographs. Although the prevalence of mimickers of lamellar effusion is only around 1%, the prevalence of arms down on a portable chest radiograph is considerably high (39%).

Use of deep learning to detect cardiomegaly on thoracic radiographs in dogs

The purpose of this study was to develop a computer-aided detection (CAD) device based on convolutional neural networks (CNNs) to detect cardiomegaly from plain radiographs in dogs. Right lateral chest radiographs (n = 1465) were retrospectively selected from archives. The radiographs were classified as having a normal cardiac silhouette (No-vertebral heart scale [VHS]-Cardiomegaly) or an enlarged cardiac silhouette (VHS-Cardiomegaly) based on the breed-specific VHS. The database was divided into a training set (1153 images) and a test set (315 images). The diagnostic accuracy of four different CNN models in the detection of cardiomegaly was calculated using the test set. All tested models had an area under the curve >0.9, demonstrating high diagnostic accuracy. There was a statistically significant difference between Model C and the remainder models (Model A vs. Model C, P = 0.0298; Model B vs. Model C, P = 0.003; Model C vs. Model D, P = 0.0018), but there were no significant differences between other combinations of models (Model A vs. Model B, P = 0.395; Model A vs. Model D, P = 0.128; Model B vs. Model D, P = 0.373). Convolutional neural networks could therefore assist veterinarians in detecting cardiomegaly in dogs from plain radiographs.

Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the “Big Three”

Background

Missed vascular events, infections, and cancers account for ~75% of serious harms from diagnostic errors. Just 15 diseases from these “Big Three” categories account for nearly half of all serious misdiagnosis-related harms in malpractice claims. As part of a larger project estimating total US burden of serious misdiagnosis-related harms, we performed a focused literature review to measure diagnostic error and harm rates for these 15 conditions.

Methods

We searched PubMed, Google, and cited references. For errors, we selected high-quality, modern, US-based studies, if available, and best available evidence otherwise. For harms, we used literature-based estimates of the generic (disease-agnostic) rate of serious harms (morbidity/mortality) per diagnostic error and applied claims-based severity weights to construct disease-specific rates. Results were validated via expert review and comparison to prior literature that used different methods. We used Monte Carlo analysis to construct probabilistic plausible ranges (PPRs) around estimates.

Results

Rates for the 15 diseases were drawn from 28 published studies representing 91,755 patients. Diagnostic error (false negative) rates ranged from 2.2% (myocardial infarction) to 62.1% (spinal abscess), with a median of 13.6% [interquartile range (IQR) 9.2–24.7] and an aggregate mean of 9.7% (PPR 8.2–12.3). Serious misdiagnosis-related harm rates per incident disease case ranged from 1.2% (myocardial infarction) to 35.6% (spinal abscess), with a median of 5.5% (IQR 4.6–13.6) and an aggregate mean of 5.2% (PPR 4.5–6.7). Rates were considered face valid by domain experts and consistent with prior literature reports.

Conclusions

Diagnostic improvement initiatives should focus on dangerous conditions with higher diagnostic error and misdiagnosis-related harm rates.

A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence

As the first step in image interpretation is detection, an error in perception can prematurely end the diagnostic process leading to missed diagnoses. Because perceptual errors of this sort-"failure to detect"-are the most common interpretive error (and cause of litigation) in radiology, understanding the nature of perceptual expertise is essential in decreasing radiology's long-standing error rates. In this article, we review what constitutes a perceptual error, the existing models of radiologic image perception, the development of perceptual expertise and how it can be tested, perceptual learning methods in training radiologists, and why understanding perceptual expertise is still relevant in the era of artificial intelligence. Adding targeted interventions, such as perceptual learning, to existing teaching practices, has the potential to enhance expertise and reduce medical error.

Association of volume of self-directed versus assigned interpretive work with diagnostic performance of radiologists: an observational study

OBJECTIVES

To understand the sources of variability in diagnostic performance among experienced radiologists.

DESIGN

All prostate MRI examinations performed between 2016 and 2018 were retrospectively reviewed.

SETTING

University hospital in Japan.

PARTICIPANTS

Data derived from 334 pathology-proven cases (male, mean age: 70 years; range: 35-90 years) that were interpreted by 10 experienced radiologists were subjected to the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

Diagnostic performance measures of the radiologists were compared with candidate factors, including interpretive volume of prostate MRIs, volume of self-directed and assigned total annual interpretive work, and years of experience. The potential influence of fatigue was also evaluated by examining the effect of the report's issue time.

RESULTS

There were 186 prostate cancer cases. Performance was based on accuracy, sensitivity and specificity (86%, 85% and 84%, respectively). While performance was not correlated with the volume of prostate MRIs, per se (ρ=-0.15, p=0.69; ρ=-0.01, p=0.99; ρ=-0.33, p=0.36) or the total MRIs assigned for each radiologist (p>0.6) or years of experience (p>0.4), all measures were strongly correlated with voluntary work represented by the interpretive volume of abdominal CTs (r=0.79, p<0.01; r=0.80, p<0.01; r=0.64, p=0.048). The performance did not differ based on the issue time of the report (morning, afternoon and evening) (χ²(2)=3.65, p=0.16).

CONCLUSIONS

Greater autonomy, represented as enhanced self-directed interpretive work, was most significantly correlated with the performance of prostate MRI interpretation. The lack of a correlation between the performance and assigned volume confirms the complexity of human learning. Together, these findings support the hypothesis that successful promotion of internal drivers could have a pervasive positive impact on improving diagnostic performance.

Light sheet microscopy for histopathology applications

Light sheet microscopy (LSM) is an evolving optical imaging technique with a plane illumination for optical sectioning and volumetric imaging spanning cell biology, embryology, and in vivo live imaging. Here, we focus on emerging biomedical applications of LSM for tissue samples. Decoupling of the light sheet illumination from detection enables high-speed and large field-of-view imaging with minimal photobleaching and phototoxicity. These unique characteristics of the LSM technique can be easily adapted and potentially replace conventional histopathological procedures. In this review, we cover LSM technology from its inception to its most advanced technology; in particular, we highlight the human histopathological imaging applications to demonstrate LSM's rapid diagnostic ability in comparison with conventional histopathological procedures. We anticipate that the LSM technique can become a useful three-dimensional imaging tool for assessing human biopsies in the near future.

Radiograph Interpretation Discrepancies in a Community Hospital Emergency Department

Introduction

In many hospitals, off-hours emergency department (ED) radiographs are not read by a radiologist until the following morning and are instead interpreted by the emergency physician (EP) at the time of service. Studies have found conflicting results regarding the radiographic interpretation discrepancies between EPs and trained radiologists. The aim of this study was to identify the number of radiologic interpretation discrepancies between EPs and radiologists in a community ED setting.

Methods

Using a pre-existing logbook of radiologic discrepancies as well as our institution’s picture archiving and communication system, all off-hours interpretation discrepancies between January 2012 and January 2015 were reviewed and recorded in a de-identified fashion. We recorded the type of radiograph obtained for each patient. Discrepancy grades were recorded based on a pre-existing 1–4 scale defined in the institution’s protocol logbook as Grade 1 (no further action needed); Grade 2 (call to the patient or pharmacy); Grade 3 (return to ED for further treatment, e.g., fracture not splinted); Grade 4 (return to ED for serious risk, e.g., pneumothorax, bowel obstruction). We also recorded the total number of radiographs formally interpreted by EPs during the prescribed time-frame to determine overall agreement between EPs and radiologists.

Results

There were 1044 discrepancies out of 16,111 EP reads, indicating 93.5% agreement. Patients averaged 48.4 ± 25.0 years of age and 53.3% were female; 25.1% were over-calls by EPs. The majority of discrepancies were minor with 75.8% Grade 1 and 22.3% Grade 2. Only 1.7% were Grade 3, which required return to the ED for further treatment. A small number of discrepancies, 0.2%, were Grade 4. Grade 4 discrepancies accounted for two of the 16,111 total reads, equivalent to 0.01%. A slight disagreement in finding between EP and radiologist accounted for 8.3% of discrepancies.

Conclusion

Results suggest that plain radiographic studies can be interpreted by EPs with a very low incidence of clinically significant discrepancies when compared to the radiologist interpretation. Due to rare though significant discrepancies, radiologist interpretation should be performed when available. Further studies are needed to determine the generalizability of this study to EDs with differing volume, patient population, acuity, and physician training.

Analysis of Perceptual Expertise in Radiology - Current Knowledge and a New Perspective

Radiologists rely principally on visual inspection to detect, describe, and classify findings in medical images. As most interpretive errors in radiology are perceptual in nature, understanding the path to radiologic expertise during image analysis is essential to educate future generations of radiologists. We review the perceptual tasks and challenges in radiologic diagnosis, discuss models of radiologic image perception, consider the application of perceptual learning methods in medical training, and suggest a new approach to understanding perceptional expertise. Specific principled enhancements to educational practices in radiology promise to deepen perceptual expertise among radiologists with the goal of improving training and reducing medical error.

A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop

Imaging research laboratories are rapidly creating machine learning systems that achieve expert human performance using open-source methods and tools. These artificial intelligence systems are being developed to improve medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection, computer-aided classification, and radiogenomics. In August 2018, a meeting was held in Bethesda, Maryland, at the National Institutes of Health to discuss the current state of the art and knowledge gaps and to develop a roadmap for future research initiatives. Key research priorities include: 1, new image reconstruction methods that efficiently produce images suitable for human interpretation from source data; 2, automated image labeling and annotation methods, including information extraction from the imaging report, electronic phenotyping, and prospective structured image reporting; 3, new machine learning methods for clinical imaging data, such as tailored, pretrained model architectures, and federated machine learning methods; 4, machine learning methods that can explain the advice they provide to human users (so-called explainable artificial intelligence); and 5, validated methods for image de-identification and data sharing to facilitate wide availability of clinical imaging data sets. This research roadmap is intended to identify and prioritize these needs for academic research laboratories, funding agencies, professional societies, and industry.

256 Shades of gray: uncertainty and diagnostic error in radiology

Radiologists practice in an environment of extraordinarily high uncertainty, which results partly from the high variability of the physical and technical aspects of imaging, partly from the inherent limitations in the diagnostic power of the various imaging modalities, and partly from the complex visual-perceptual and cognitive processes involved in image interpretation. This paper reviews the high level of uncertainty inherent to the process of radiological imaging and image interpretation vis-à-vis the issue of radiological interpretive error, in order to highlight the considerable degree of overlap that exists between these. The scope of radiological error, its many potential causes and various error-reduction strategies in radiology are also reviewed.

Radiologist Quality Assurance by Nonradiologists at Tumor Board

PURPOSE

To explore the use of nonradiologists as a method to efficiently reduce bias in the assessment of radiologist performance using a hepatobiliary tumor board as a case study.

MATERIALS AND METHODS

Institutional review board approval was obtained for this HIPAA-compliant prospective quality assurance (QA) effort. Consecutive patients with CT or MR imaging reviewed at one hepatobiliary tumor board between February 2016 and October 2016 (n = 265) were included. All presentations were assigned prospective anonymous QA scores by an experienced nonradiologist hepatobiliary provider based on contemporaneous comparison of the imaging interpretation at a tumor board and the original interpretation(s): concordant, minor discordance, major discordance. Major discordance was defined as a discrepancy that may affect clinical management. Minor discordance was defined as a discrepancy unlikely to affect clinical management. All discordances and predicted management changes were retrospectively confirmed by the liver tumor program medical director. Logistic regression analyses were performed to determine what factors best predict discordant reporting.

RESULTS

Approximately one-third (30% [79 of 265]) of reports were assigned a discordance, including 51 (19%) minor and 28 (11%) major discordances. The most common related to mass size (41% [32 of 79]), tumor stage and extent (24% [19 of 79]), and assigned LI-RADS v2014 score (22% [17 of 79]). One radiologist had 11.8-fold greater odds of discordance (P = .002). Nine other radiologists were similar (P = .10-.99). Radiologists presenting their own studies had 4.5-fold less odds of discordance (P = .006).

CONCLUSIONS

QA conducted in line with tumor board workflow can enable efficient assessment of radiologist performance. Discordant interpretations are commonly (30%) reported by nonradiologist providers.