Fostering a Healthy AI Ecosystem for Radiology: Conclusions of the 2018 RSNA Summit on AI in Radiology

Improving annotation efficiency for fully labeling a breast mass segmentation dataset

Purpose

Breast cancer remains a leading cause of death for women. Screening programs are deployed to detect cancer at early stages. One current barrier identified by breast imaging researchers is a shortage of labeled image datasets. Addressing this problem is crucial to improve early detection models. We present an active learning (AL) framework for segmenting breast masses from 2D digital mammography, and we publish labeled data. Our method aims to reduce the input needed from expert annotators to reach a fully labeled dataset.

Approach

We create a dataset of 1136 mammographic masses with pixel-wise binary segmentation labels, with the test subset labeled independently by two different teams. With this dataset, we simulate a human annotator within an AL framework to develop and compare AI-assisted labeling methods, using a discriminator model and a simulated oracle to collect acceptable segmentation labels. A UNet model is retrained on these labels, generating new segmentations. We evaluate various oracle heuristics using the percentage of segmentations that the oracle relabels and measure the quality of the proposed labels by evaluating the intersection over union over a validation dataset.

Results

Our method reduces expert annotator input by 44%. We present a dataset of 1136 binary segmentation labels approved by board-certified radiologists and make the 143-image validation set public for comparison with other researchers' methods.

Conclusions

We demonstrate that AL can significantly improve the efficiency and time-effectiveness of creating labeled mammogram datasets. Our framework facilitates the development of high-quality datasets while minimizing manual effort in the domain of digital mammography.

A vendor-agnostic, PACS integrated, and DICOM-compatible software-server pipeline for testing segmentation algorithms within the clinical radiology workflow

Background

Reproducible approaches are needed to bring AI/ML for medical image analysis closer to the bedside. Investigators wishing to shadow test cross-sectional medical imaging segmentation algorithms on new studies in real-time will benefit from simple tools that integrate PACS with on-premises image processing, allowing visualization of DICOM-compatible segmentation results and volumetric data at the radiology workstation.

Purpose

In this work, we develop and release a simple containerized and easily deployable pipeline for shadow testing of segmentation algorithms within the clinical workflow.

Methods

Our end-to-end automated pipeline has two major components- 1. A router/listener and anonymizer and an OHIF web viewer backstopped by a DCM4CHEE DICOM query/retrieve archive deployed in the virtual infrastructure of our secure hospital intranet, and 2. An on-premises single GPU workstation host for DICOM/NIfTI conversion steps, and image processing. DICOM images are visualized in OHIF along with their segmentation masks and associated volumetry measurements (in mL) using DICOM SEG and structured report (SR) elements. Since nnU-net has emerged as a widely-used out-of-the-box method for training segmentation models with state-of-the-art performance, feasibility of our pipleine is demonstrated by recording clock times for a traumatic pelvic hematoma nnU-net model.

Results

Mean total clock time from PACS send by user to completion of transfer to the DCM4CHEE query/retrieve archive was 5 min 32 s (± SD of 1 min 26 s). This compares favorably to the report turnaround times for whole-body CT exams, which often exceed 30 min, and illustrates feasibility in the clinical setting where quantitative results would be expected prior to report sign-off. Inference times accounted for most of the total clock time, ranging from 2 min 41 s to 8 min 27 s. All other virtual and on-premises host steps combined ranged from a minimum of 34 s to a maximum of 48 s.

Conclusion

The software worked seamlessly with an existing PACS and could be used for deployment of DL models within the radiology workflow for prospective testing on newly scanned patients. Once configured, the pipeline is executed through one command using a single shell script. The code is made publicly available through an open-source license at "https://github.com/vastc/," and includes a readme file providing pipeline config instructions for host names, series filter, other parameters, and citation instructions for this work.

Analyzing Barriers and Enablers for the Acceptance of Artificial Intelligence Innovations into Radiology Practice: A Scoping Review

OBJECTIVES

This scoping review was conducted to determine the barriers and enablers associated with the acceptance of artificial intelligence/machine learning (AI/ML)-enabled innovations into radiology practice from a physician's perspective.

METHODS

A systematic search was performed using Ovid Medline and Embase. Keywords were used to generate refined queries with the inclusion of computer-aided diagnosis, artificial intelligence, and barriers and enablers. Three reviewers assessed the articles, with a fourth reviewer used for disagreements. The risk of bias was mitigated by including both quantitative and qualitative studies.

RESULTS

An electronic search from January 2000 to 2023 identified 513 studies. Twelve articles were found to fulfill the inclusion criteria: qualitative studies (n = 4), survey studies (n = 7), and randomized controlled trials (RCT) (n = 1). Among the most common barriers to AI implementation into radiology practice were radiologists' lack of acceptance and trust in AI innovations; a lack of awareness, knowledge, and familiarity with the technology; and perceived threat to the professional autonomy of radiologists. The most important identified AI implementation enablers were high expectations of AI's potential added value; the potential to decrease errors in diagnosis; the potential to increase efficiency when reaching a diagnosis; and the potential to improve the quality of patient care.

CONCLUSIONS

This scoping review found that few studies have been designed specifically to identify barriers and enablers to the acceptance of AI in radiology practice. The majority of studies have assessed the perception of AI replacing radiologists, rather than other barriers or enablers in the adoption of AI. To comprehensively evaluate the potential advantages and disadvantages of integrating AI innovations into radiology practice, gathering more robust research evidence on stakeholder perspectives and attitudes is essential.

MR fingerprinting for semisolid magnetization transfer and chemical exchange saturation transfer quantification

Chemical exchange saturation transfer (CEST) MRI has positioned itself as a promising contrast mechanism, capable of providing molecular information at sufficient resolution and amplified sensitivity. However, it has not yet become a routinely employed clinical technique, due to a variety of confounding factors affecting its contrast-weighted image interpretation and the inherently long scan time. CEST MR fingerprinting (MRF) is a novel approach for addressing these challenges, allowing simultaneous quantitation of several proton exchange parameters using rapid acquisition schemes. Recently, a number of deep-learning algorithms have been developed to further boost the performance and speed of CEST and semi-solid macromolecule magnetization transfer (MT) MRF. This review article describes the fundamental theory behind semisolid MT/CEST-MRF and its main applications. It then details supervised and unsupervised learning approaches for MRF image reconstruction and describes artificial intelligence (AI)-based pipelines for protocol optimization. Finally, practical considerations are discussed, and future perspectives are given, accompanied by basic demonstration code and data.

Should artificial intelligence be used in conjunction with Neuroimaging in the diagnosis of Alzheimer's disease?

Alzheimer's disease (AD) is a progressive, neurodegenerative disorder that affects memory, thinking, behavior, and other cognitive functions. Although there is no cure, detecting AD early is important for the development of a therapeutic plan and a care plan that may preserve cognitive function and prevent irreversible damage. Neuroimaging, such as magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET), has served as a critical tool in establishing diagnostic indicators of AD during the preclinical stage. However, as neuroimaging technology quickly advances, there is a challenge in analyzing and interpreting vast amounts of brain imaging data. Given these limitations, there is great interest in using artificial Intelligence (AI) to assist in this process. AI introduces limitless possibilities in the future diagnosis of AD, yet there is still resistance from the healthcare community to incorporate AI in the clinical setting. The goal of this review is to answer the question of whether AI should be used in conjunction with neuroimaging in the diagnosis of AD. To answer the question, the possible benefits and disadvantages of AI are discussed. The main advantages of AI are its potential to improve diagnostic accuracy, improve the efficiency in analyzing radiographic data, reduce physician burnout, and advance precision medicine. The disadvantages include generalization and data shortage, lack of in vivo gold standard, skepticism in the medical community, potential for physician bias, and concerns over patient information, privacy, and safety. Although the challenges present fundamental concerns and must be addressed when the time comes, it would be unethical not to use AI if it can improve patient health and outcome.

An efficient magnetic resonance image data quality screening dashboard

Purpose

Complex data processing and curation for artificial intelligence applications rely on high‐quality data sets for training and analysis. Manually reviewing images and their associated annotations is a very laborious task and existing quality control tools for data review are generally limited to raw images only. The purpose of this work was to develop an imaging informatics dashboard for the easy and fast review of processed magnetic resonance (MR) imaging data sets; we demonstrated its ability in a large‐scale data review.

Methods

We developed a custom R Shiny dashboard that displays key static snapshots of each imaging study and its annotations. A graphical interface allows the structured entry of review data and download of tabulated review results. We evaluated the dashboard using two large data sets: 1380 processed MR imaging studies from our institution and 285 studies from the 2018 MICCAI Brain Tumor Segmentation Challenge (BraTS).

Results

Studies were reviewed at an average rate of 100/h using the dashboard, 10 times faster than using existing data viewers. For data from our institution, 1181 of the 1380 (86%) studies were of acceptable quality. The most commonly identified failure modes were tumor segmentation (9.6% of cases) and image registration (4.6% of cases). Tumor segmentation without visible errors on the dashboard had much better agreement with reference tumor volume measurements (root‐mean‐square error 12.2 cm³) than did segmentations with minor errors (20.5 cm³) or failed segmentations (27.4 cm³). In the BraTS data, 242 of 285 (85%) studies were acceptable quality after processing. Among the 43 cases that failed review, 14 had unacceptable raw image quality.

Conclusion

Our dashboard provides a fast, effective tool for reviewing complex processed MR imaging data sets. It is freely available for download at https://github.com/EGates1/MRDQED.

A Radiology-focused Review of Predictive Uncertainty for AI Interpretability in Computer-assisted Segmentation

The recent advances and availability of computer hardware, software tools, and massive digital data archives have enabled the rapid development of artificial intelligence (AI) applications. Concerns over whether AI tools can "communicate" decisions to radiologists and primary care physicians is of particular importance because automated clinical decisions can substantially impact patient outcome. A challenge facing the clinical implementation of AI stems from the potential lack of trust clinicians have in these predictive models. This review will expand on the existing literature on interpretability methods for deep learning and review the state-of-the-art methods for predictive uncertainty estimation for computer-assisted segmentation tasks. Last, we discuss how uncertainty can improve predictive performance and model interpretability and can act as a tool to help foster trust. Keywords: Segmentation, Quantification, Ethics, Bayesian Network (BN) © RSNA, 2021.

An Open-Source, Vender Agnostic Hardware and Software Pipeline for Integration of Artificial Intelligence in Radiology Workflow

Although machine learning (ML) has made significant improvements in radiology, few algorithms have been integrated into clinical radiology workflow. Complex radiology IT environments and Picture Archiving and Communication System (PACS) pose unique challenges in creating a practical ML schema. However, clinical integration and testing are critical to ensuring the safety and accuracy of ML algorithms. This study aims to propose, develop, and demonstrate a simple, efficient, and understandable hardware and software system for integrating ML models into the standard radiology workflow and PACS that can serve as a framework for testing ML algorithms. A Digital Imaging and Communications in Medicine/Graphics Processing Unit (DICOM/GPU) server and software pipeline was established at a metropolitan county hospital intranet to demonstrate clinical integration of ML algorithms in radiology. A clinical ML integration schema, agnostic to the hospital IT system and specific ML models/frameworks, was implemented and tested with a breast density classification algorithm and prospectively evaluated for time delays using 100 digital 2D mammograms. An open-source clinical ML integration schema was successfully implemented and demonstrated. This schema allows for simple uploading of custom ML models. With the proposed setup, the ML pipeline took an average of 26.52 s per second to process a batch of 100 studies. The most significant processing time delays were noted in model load and study stability times. The code is made available at " http://bit.ly/2Z121hX ". We demonstrated the feasibility to deploy and utilize ML models in radiology without disrupting existing radiology workflow.