Computer-aided diagnosis: how to move from the laboratory to the clinic

Evaluation and Real-World Performance Monitoring of Artificial Intelligence Models in Clinical Practice: Try It, Buy It, Check It

The pace of regulatory clearance of artificial intelligence (AI) algorithms for radiology continues to accelerate, and numerous algorithms are becoming available for use in clinical practice. End users of AI in radiology should be aware that AI algorithms may not work as expected when used beyond the institutions in which they were trained, and model performance may degrade over time. In this article, we discuss why regulatory clearance alone may not be enough to ensure AI will be safe and effective in all radiological practices and review strategies available resources for evaluating before clinical use and monitoring performance of AI models to ensure efficacy and patient safety.

Neovascularization Detection and Localization in Fundus Images Using Deep Learning

Proliferative Diabetic Retinopathy (PDR) is a severe retinal disease that threatens diabetic patients. It is characterized by neovascularization in the retina and the optic disk. PDR clinical features contain highly intense retinal neovascularization and fibrous spreads, leading to visual distortion if not controlled. Different image processing techniques have been proposed to detect and diagnose neovascularization from fundus images. Recently, deep learning methods are getting popular in neovascularization detection due to artificial intelligence advancement in biomedical image processing. This paper presents a semantic segmentation convolutional neural network architecture for neovascularization detection. First, image pre-processing steps were applied to enhance the fundus images. Then, the images were divided into small patches, forming a training set, a validation set, and a testing set. A semantic segmentation convolutional neural network was designed and trained to detect the neovascularization regions on the images. Finally, the network was tested using the testing set for performance evaluation. The proposed model is entirely automated in detecting and localizing neovascularization lesions, which is not possible with previously published methods. Evaluation results showed that the model could achieve accuracy, sensitivity, specificity, precision, Jaccard similarity, and Dice similarity of 0.9948, 0.8772, 0.9976, 0.8696, 0.7643, and 0.8466, respectively. We demonstrated that this model could outperform other convolutional neural network models in neovascularization detection.

Multiorgan segmentation from partially labeled datasets with conditional nnU-Net

Accurate and robust multiorgan abdominal CT segmentation plays a significant role in numerous clinical applications, such as therapy treatment planning and treatment delivery. Almost all existing segmentation networks rely on fully annotated data with strong supervision. However, annotating fully annotated multiorgan data in CT images is both laborious and time-consuming. In comparison, massive partially labeled datasets are usually easily accessible. In this paper, we propose conditional nnU-Net trained on the union of partially labeled datasets for multiorgan segmentation. The deep model employs the state-of-the-art nnU-Net as the backbone and introduces a conditioning strategy by feeding auxiliary information into the decoder architecture as an additional input layer. This model leverages the prior conditional information to identify the organ class at the pixel-wise level and encourages organs' spatial information recovery. Furthermore, we adopt a deep supervision mechanism to refine the outputs at different scales and apply the combination of Dice loss and Focal loss to optimize the training model. Our proposed method is evaluated on seven publicly available datasets of the liver, pancreas, spleen and kidney, in which promising segmentation performance has been achieved. The proposed conditional nnU-Net breaks down the barriers between nonoverlapping labeled datasets and further alleviates the problem of data hunger in multiorgan segmentation.

Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa

Computer-aided digital chest radiograph interpretation (CAD) can facilitate high-throughput screening for tuberculosis (TB), but its use in population-based active case-finding programs has been limited. In an HIV-endemic area in rural South Africa, we used a CAD algorithm (CAD4TBv5) to interpret digital chest x-rays (CXR) as part of a mobile health screening effort. Participants with TB symptoms or CAD4TBv5 score above the triaging threshold were referred for microbiological sputum assessment. During an initial pilot phase, a low CAD4TBv5 triaging threshold of 25 was selected to maximize TB case finding. We report the performance of CAD4TBv5 in screening 9,914 participants, 99 (1.0%) of whom were found to have microbiologically proven TB. CAD4TBv5 was able to identify TB cases at the same sensitivity but lower specificity as a blinded radiologist, whereas the next generation of the algorithm (CAD4TBv6) achieved comparable sensitivity and specificity to the radiologist. The CXRs of people with microbiologically confirmed TB spanned a range of lung field abnormality, including 19 (19.2%) cases deemed normal by the radiologist. HIV serostatus did not impact CAD4TB's performance. Notably, 78.8% of the TB cases identified during this population-based survey were asymptomatic and therefore triaged for sputum collection on the basis of CAD4TBv5 score alone. While CAD4TBv6 has the potential to replace radiologists for triaging CXRs in TB prevalence surveys, population-specific piloting is necessary to set the appropriate triaging thresholds. Further work on image analysis strategies is needed to identify radiologically subtle active TB.

Performance changes due to differences in training data for cerebral aneurysm detection in head MR angiography images

PURPOSE

The performance of computer-aided detection (CAD) software depends on the quality and quantity of the dataset used for machine learning. If the data characteristics in development and practical use are different, the performance of CAD software degrades. In this study, we investigated changes in detection performance due to differences in training data for cerebral aneurysm detection software in head magnetic resonance angiography images.

MATERIALS AND METHODS

We utilized three types of CAD software for cerebral aneurysm detection in MRA images, which were based on 3D local intensity structure analysis, graph-based features, and convolutional neural network. For each type of CAD software, we compared three types of training pattern, which were two types of training using single-site data and one type of training using multisite data. We also carried out internal and external evaluations.

RESULTS

In training using single-site data, the performance of CAD software largely and unpredictably fluctuated when the training dataset was changed. Training using multisite data did not show the lowest performance among the three training patterns for any CAD software and dataset.

CONCLUSION

The training of cerebral aneurysm detection software using data collected from multiple sites is desirable to ensure the stable performance of the software.

Performance of an AI based CAD system in solid lung nodule detection on chest phantom radiographs compared to radiology residents and fellow radiologists

Background

Despite the decreasing relevance of chest radiography in lung cancer screening, chest radiography is still frequently applied to assess for lung nodules. The aim of the current study was to determine the accuracy of a commercial AI based CAD system for the detection of artificial lung nodules on chest radiograph phantoms and compare the performance to radiologists in training.

Methods

Sixty-one anthropomorphic lung phantoms were equipped with 140 randomly deployed artificial lung nodules (5, 8, 10, 12 mm). A random generator chose nodule size and distribution before a two-plane chest X-ray (CXR) of each phantom was performed. Seven blinded radiologists in training (2 fellows, 5 residents) with 2 to 5 years of experience in chest imaging read the CXRs on a PACS-workstation independently. Results of the software were recorded separately. McNemar test was used to compare each radiologist’s results to the AI-computer-aided-diagnostic (CAD) software in a per-nodule and a per-phantom approach and Fleiss-Kappa was applied for inter-rater and intra-observer agreements.

Results

Five out of seven readers showed a significantly higher accuracy than the AI algorithm. The pooled accuracies of the radiologists in a nodule-based and a phantom-based approach were 0.59 and 0.82 respectively, whereas the AI-CAD showed accuracies of 0.47 and 0.67, respectively. Radiologists’ average sensitivity for 10 and 12 mm nodules was 0.80 and dropped to 0.66 for 8 mm (P=0.04) and 0.14 for 5 mm nodules (P<0.001). The radiologists and the algorithm both demonstrated a significant higher sensitivity for peripheral compared to central nodules (0.66 vs. 0.48; P=0.004 and 0.64 vs. 0.094; P=0.025, respectively). Inter-rater agreements were moderate among the radiologists and between radiologists and AI-CAD software (K’=0.58±0.13 and 0.51±0.1). Intra-observer agreement was calculated for two readers and was almost perfect for the phantom-based (K’=0.85±0.05; K’=0.80±0.02); and substantial to almost perfect for the nodule-based approach (K’=0.83±0.02; K’=0.78±0.02).

Conclusions

The AI based CAD system as a primary reader acts inferior to radiologists regarding lung nodule detection in chest phantoms. Chest radiography has reasonable accuracy in lung nodule detection if read by a radiologist alone and may be further optimized by an AI based CAD system as a second reader.

Medical image-based detection of COVID-19 using Deep Convolution Neural Networks

The demand for automatic detection of Novel Coronavirus or COVID-19 is increasing across the globe. The exponential rise in cases burdens healthcare facilities, and a vast amount of multimedia healthcare data is being explored to find a solution. This study presents a practical solution to detect COVID-19 from chest X-rays while distinguishing those from normal and impacted by Viral Pneumonia via Deep Convolution Neural Networks (CNN). In this study, three pre-trained CNN models (EfficientNetB0, VGG16, and InceptionV3) are evaluated through transfer learning. The rationale for selecting these specific models is their balance of accuracy and efficiency with fewer parameters suitable for mobile applications. The dataset used for the study is publicly available and compiled from different sources. This study uses deep learning techniques and performance metrics (accuracy, recall, specificity, precision, and F1 scores). The results show that the proposed approach produced a high-quality model, with an overall accuracy of 92.93%, COVID-19, a sensitivity of 94.79%. The work indicates a definite possibility to implement computer vision design to enable effective detection and screening measures.

Radiologic Assessment of Osteosarcoma Lung Metastases: State of the Art and Recent Advances

The lung is the most frequent site of osteosarcoma (OS) metastases, which are a critical point in defining a patient’s prognosis. Chest computed tomography (CT) represents the gold standard for the detection of lung metastases even if its sensitivity widely ranges in the literature since lung localizations are often atypical. ESMO guidelines represent one of the major references for the follow-up program of OS patients. The development of new reconstruction techniques, such as the iterative method and the deep learning-based image reconstruction (DLIR), has led to a significant reduction of the radiation dose with the low-dose CT. The improvement of these techniques has great importance considering the young-onset of the disease and the strict chest surveillance during follow-up programs. The use of ¹⁸F-fluorodeoxyglucose (FDG) positron emission tomography (PET)/CT is still controversial, while volume doubling time (VDT) and computer-aided diagnosis (CAD) systems are recent diagnostic tools that could support radiologists for lung nodules evaluation. Their use, well-established for other malignancies, needs to be further evaluated, focusing on OS patients.

Computer-aided diagnosis in the era of deep learning

Computer-aided diagnosis (CAD) has been a major field of research for the past few decades. CAD uses machine learning methods to analyze imaging and/or nonimaging patient data and makes assessment of the patient's condition, which can then be used to assist clinicians in their decision-making process. The recent success of the deep learning technology in machine learning spurs new research and development efforts to improve CAD performance and to develop CAD for many other complex clinical tasks. In this paper, we discuss the potential and challenges in developing CAD tools using deep learning technology or artificial intelligence (AI) in general, the pitfalls and lessons learned from CAD in screening mammography and considerations needed for future implementation of CAD or AI in clinical use. It is hoped that the past experiences and the deep learning technology will lead to successful advancement and lasting growth in this new era of CAD, thereby enabling CAD to deliver intelligent aids to improve health care.

Marginal loss and exclusion loss for partially supervised multi-organ segmentation

Annotating multiple organs in medical images is both costly and time-consuming; therefore, existing multi-organ datasets with labels are often low in sample size and mostly partially labeled, that is, a dataset has a few organs labeled but not all organs. In this paper, we investigate how to learn a single multi-organ segmentation network from a union of such datasets. To this end, we propose two types of novel loss function, particularly designed for this scenario: (i) marginal loss and (ii) exclusion loss. Because the background label for a partially labeled image is, in fact, a 'merged' label of all unlabelled organs and 'true' background (in the sense of full labels), the probability of this 'merged' background label is a marginal probability, summing the relevant probabilities before merging. This marginal probability can be plugged into any existing loss function (such as cross entropy loss, Dice loss, etc.) to form a marginal loss. Leveraging the fact that the organs are non-overlapping, we propose the exclusion loss to gauge the dissimilarity between labeled organs and the estimated segmentation of unlabelled organs. Experiments on a union of five benchmark datasets in multi-organ segmentation of liver, spleen, left and right kidneys, and pancreas demonstrate that using our newly proposed loss functions brings a conspicuous performance improvement for state-of-the-art methods without introducing any extra computation.

Liver segmentation in abdominal CT images via auto-context neural network and self-supervised contour attention

OBJECTIVE

Accurate image segmentation of the liver is a challenging problem owing to its large shape variability and unclear boundaries. Although the applications of fully convolutional neural networks (CNNs) have shown groundbreaking results, limited studies have focused on the performance of generalization. In this study, we introduce a CNN for liver segmentation on abdominal computed tomography (CT) images that focus on the performance of generalization and accuracy.

METHODS

To improve the generalization performance, we initially propose an auto-context algorithm in a single CNN. The proposed auto-context neural network exploits an effective high-level residual estimation to obtain the shape prior. Identical dual paths are effectively trained to represent mutual complementary features for an accurate posterior analysis of a liver. Further, we extend our network by employing a self-supervised contour scheme. We trained sparse contour features by penalizing the ground-truth contour to focus more contour attentions on the failures.

RESULTS

We used 180 abdominal CT images for training and validation. Two-fold cross-validation is presented for a comparison with the state-of-the-art neural networks. The experimental results show that the proposed network results in better accuracy when compared to the state-of-the-art networks by reducing 10.31% of the Hausdorff distance. Novel multiple N-fold cross-validations are conducted to show the best performance of generalization of the proposed network.

CONCLUSION AND SIGNIFICANCE

The proposed method minimized the error between training and test images more than any other modern neural networks. Moreover, the contour scheme was successfully employed in the network by introducing a self-supervising metric.

Colorectal polyp characterization with endocytoscopy: Ready for widespread implementation with artificial intelligence?

Endocytoscopy provides an in-vivo visualization of nuclei and micro-vessels at the cellular level in real-time, facilitating so-called "optical biopsy" or "virtual histology" of colorectal polyps/neoplasms. This functionality is enabled by 520-fold magnification power with endocytoscopy and recent breakthroughs in artificial intelligence (AI) allowing a great advance in endocytoscopic imaging; interpretation of images is now fully supported by AI tool which outputs predictions of polyp histopathology during colonoscopy. The advantage of the use of AI during optical biopsy can be appreciated especially by non-expert endoscopists who to increase performance. This paper provides an overview of the latest evidence on colorectal polyp characterization with endocytoscopy combined with AI and identify the barriers to its widespread implementation.

AI approach of cycle-consistent generative adversarial networks to synthesize PET images to train computer-aided diagnosis algorithm for dementia

OBJECTIVE

An artificial intelligence (AI)-based algorithm typically requires a considerable amount of training data; however, few training images are available for dementia with Lewy bodies and frontotemporal lobar degeneration. Therefore, this study aims to present the potential of cycle-consistent generative adversarial networks (CycleGAN) to obtain enough number of training images for AI-based computer-aided diagnosis (CAD) algorithms for diagnosing dementia.

METHODS

We trained CycleGAN using 43 amyloid-negative and 45 positive images in slice-by-slice.

RESULTS

The CycleGAN can be used to synthesize reasonable amyloid-positive images, and the continuity of slices was preserved.

DISCUSSION

Our results show that CycleGAN has the potential to generate a sufficient number of training images for CAD of dementia.

Deep learning in generating radiology reports: A survey

Substantial progress has been made towards implementing automated radiology reporting models based on deep learning (DL). This is due to the introduction of large medical text/image datasets. Generating radiology coherent paragraphs that do more than traditional medical image annotation, or single sentence-based description, has been the subject of recent academic attention. This presents a more practical and challenging application and moves towards bridging visual medical features and radiologist text. So far, the most common approach has been to utilize publicly available datasets and develop DL models that integrate convolutional neural networks (CNN) for image analysis alongside recurrent neural networks (RNN) for natural language processing (NLP) and natural language generation (NLG). This is an area of research that we anticipate will grow in the near future. We focus our investigation on the following critical challenges: understanding radiology text/image structures and datasets, applying DL algorithms (mainly CNN and RNN), generating radiology text, and improving existing DL based models and evaluation metrics. Lastly, we include a critical discussion and future research recommendations. This survey will be useful for researchers interested in DL, particularly those interested in applying DL to radiology reporting.

Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors

Objective

The objective was to identify barriers and facilitators to the implementation of artificial intelligence (AI) applications in clinical radiology in The Netherlands.

Materials and methods

Using an embedded multiple case study, an exploratory, qualitative research design was followed. Data collection consisted of 24 semi-structured interviews from seven Dutch hospitals. The analysis of barriers and facilitators was guided by the recently published Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework for new medical technologies in healthcare organizations.

Results

Among the most important facilitating factors for implementation were the following: (i) pressure for cost containment in the Dutch healthcare system, (ii) high expectations of AI’s potential added value, (iii) presence of hospital-wide innovation strategies, and (iv) presence of a “local champion.” Among the most prominent hindering factors were the following: (i) inconsistent technical performance of AI applications, (ii) unstructured implementation processes, (iii) uncertain added value for clinical practice of AI applications, and (iv) large variance in acceptance and trust of direct (the radiologists) and indirect (the referring clinicians) adopters.

Conclusion

In order for AI applications to contribute to the improvement of the quality and efficiency of clinical radiology, implementation processes need to be carried out in a structured manner, thereby providing evidence on the clinical added value of AI applications.

Key Points

• Successful implementation of AI in radiology requires collaboration between radiologists and referring clinicians.

• Implementation of AI in radiology is facilitated by the presence of a local champion.

• Evidence on the clinical added value of AI in radiology is needed for successful implementation.

Electronic supplementary material

The online version of this article (10.1007/s00330-020-06946-y) contains supplementary material, which is available to authorized users.

MRI image analysis methods and applications: an algorithmic perspective using brain tumors as an exemplar

The use of magnetic resonance imaging (MRI) in healthcare and the emergence of radiology as a practice are both relatively new compared with the classical specialties in medicine. Having its naissance in the 1970s and later adoption in the 1980s, the use of MRI has grown exponentially, consequently engendering exciting new areas of research. One such development is the use of computational techniques to analyze MRI images much like the way a radiologist would. With the advent of affordable, powerful computing hardware and parallel developments in computer vision, MRI image analysis has also witnessed unprecedented growth. Due to the interdisciplinary and complex nature of this subfield, it is important to survey the current landscape and examine the current approaches for analysis and trend trends moving forward.

Deep Learning for the Classification of Small (≤2 cm) Pulmonary Nodules on CT Imaging: A Preliminary Study

RATIONALE AND OBJECTIVES

We aimed to present a deep learning-based malignancy prediction model (CT-lungNET) that is simpler and faster to use in the diagnosis of small (≤2 cm) pulmonary nodules on nonenhanced chest CT and to preliminarily evaluate its performance and usefulness for human reviewers.

MATERIALS AND METHODS

A total of 173 whole nonenhanced chest CT images containing 208 pulmonary nodules (94 malignant and 11 benign nodules) ranging in size from 5 mm to 20 mm were collected. Pathologically confirmed nodules or nodules that remained unchanged for more than 1 year were included, and 30 benign and 30 malignant nodules were randomly assigned into the test set. We designed CT-lungNET to include three convolutional layers followed by two fully-connected layers and compared its diagnostic performance and processing time with those of AlexNET by using the area under the receiver operating curve (AUROC). An observer performance test was conducted involving eight human reviewers of four different groups (medical students, physicians, radiologic residents, and thoracic radiologists) at test 1 and test 2, referring to the CT-lungNET's malignancy prediction rate with pairwise comparison receiver operating curve analysis.

RESULTS

CT-lungNET showed an improved AUROC (0.85; 95% confidence interval: 0.74-0.93), compared to that of the AlexNET (0.82; 95% confidence interval: 0.71-0.91). The processing speed per one image slice for CT-lungNET was about 10 times faster than that for AlexNET (0.90 vs. 8.79 seconds). During the observer performance test, the classification performance of nonradiologists was increased with the aid of CTlungNET, (mean AUC improvement: 0.13; range: 0.03-0.19) but not significantly so in the radiologists group (mean AUC improvement: 0.02; range: -0.02 to 0.07).

CONCLUSION

CT-lungNET was able to provide better classification results with a significantly shorter amount of processing time as compared to AlexNET in the diagnosis of small pulmonary nodules on nonenhanced chest CT. In this preliminary observer performance test, CT-lungNET may have a role acting as a second reviewer for less experienced reviewers, resulting in enhanced performance in the diagnosis of early lung cancer.

Novel platform for development, training, and validation of computer-assisted detection/diagnosis software

PURPOSE

To build a novel, open-source, purely web-based platform system to address problems in the development and clinical use of computer-assisted detection/diagnosis (CAD) software. The new platform system will replace the existing system for the development and validation of CAD software, Clinical Infrastructure for Radiologic Computation of United Solutions (CIRCUS).

METHODS

In our new system, the two top-level applications visible to users are the web-based image database (CIRCUS DB; database) and the Docker plug-in-based CAD execution platform (CIRCUS CS; clinical server). These applications are built on top of a shared application programming interface server, a three-dimensional image viewer component, and an image repository.

RESULTS

We successfully installed our new system into a Linux server at two clinical sites. A total of 1954 cases were registered in CIRCUS DB. We have been utilizing CIRCUS CS with four Docker-based CAD plug-ins.

CONCLUSIONS

We have successfully built a new version of the CIRCUS system. Our platform was successfully implemented at two clinical sites, and we plan to publish it as an open-source software project.

From Medical Imaging to Radiomics: Role of Data Science for Advancing Precision Health

Treating disease according to precision health requires the individualization of therapeutic solutions as a cardinal step that is part of a process that typically depends on multiple factors. The starting point is the collection and assembly of data over time to assess the patient’s health status and monitor response to therapy. Radiomics is a very important component of this process. Its main goal is implementing a protocol to quantify the image informative contents by first mining and then extracting the most representative features. Further analysis aims to detect potential disease phenotypes through signs and marks of heterogeneity. As multimodal images hinge on various data sources, and these can be integrated with treatment plans and follow-up information, radiomics is naturally centered on dynamically monitoring disease progression and/or the health trajectory of patients. However, radiomics creates critical needs too. A concise list includes: (a) successful harmonization of intra/inter-modality radiomic measurements to facilitate the association with other data domains (genetic, clinical, lifestyle aspects, etc.); (b) ability of data science to revise model strategies and analytics tools to tackle multiple data types and structures (electronic medical records, personal histories, hospitalization data, genomic from various specimens, imaging, etc.) and to offer data-agnostic solutions for patient outcomes prediction; (c) and model validation with independent datasets to ensure generalization of results, clinical value of new risk stratifications, and support to clinical decisions for highly individualized patient management.

Feasibility of integrating computer-aided diagnosis with structured reports of prostate multiparametric MRI

OBJECTIVES

To evaluate the feasibility of integrating computer-aided diagnosis (CAD) with structured reports of prostate multiparametric MRI (mpMRI).

METHODS

This retrospective study enrolled 153 patients who underwent prostate mpMRI for the purpose of targeted biopsy; patients were divided into a group with clinically significant prostate cancer (csPCa, Gleason score ≥ 3 + 4, n = 89) and a group with non-csPCa (n = 64). Ten inexperienced radiologists retrospectively evaluated these cases (single reader per case) twice using structured reports, and they were blinded to the pathologic results. Initially, the readers interpreted mpMRI without CAD. Six weeks later, they evaluated the same cases again with CAD assistance. At each time of image interpretation, lesions detected by the readers were marked on the prostate vector map in structured reports, and a PI-RADS score was given to each lesion. Diagnostic efficacy and reading time were evaluated for the two reading sessions.

RESULTS

With the assistance of CAD, the overall diagnostic efficacy was improved, i.e., the AUC increased from 0.83 to 0.89 (p = 0.018). Specifically, per-patient sensitivity (84.3% vs. 93.3%) and per-lesion sensitivity (76.7% vs. 88.8%) were significantly improved (all p < 0.05). Per-patient specificity with CAD (65.6%) was higher than that without CAD (56.3%), but statistical significance was not reached (p = 0.238). The reading time for each case decreased from 10.9 min to 7.8 min (p < 0.001).

CONCLUSIONS

It is feasible to integrate CAD with structured reports of prostate mpMRI. This reading paradigm can improve the diagnostic sensitivity of csPCa detection and reduce reading time among inexperienced radiologists.

Radiomics in stratification of pancreatic cystic lesions: Machine learning in action

Pancreatic cystic lesions (PCLs) are well-known precursors of pancreatic cancer. Their diagnosis can be challenging as their behavior varies from benign to malignant disease. Precise and timely management of malignant pancreatic cysts might prevent transformation to pancreatic cancer. However, the current consensus guidelines, which rely on standard imaging features to predict cyst malignancy potential, are conflicting and unclear. This has led to an increased interest in radiomics, a high-throughput extraction of comprehensible data from standard of care images. Radiomics can be used as a diagnostic and prognostic tool in personalized medicine. It utilizes quantitative image analysis to extract features in conjunction with machine learning and artificial intelligence (AI) methods like support vector machines, random forest, and convolutional neural network for feature selection and classification. Selected features can then serve as imaging biomarkers to predict high-risk PCLs. Radiomics studies conducted heretofore on PCLs have shown promising results. This cost-effective approach would help us to differentiate benign PCLs from malignant ones and potentially guide clinical decision-making leading to better utilization of healthcare resources. In this review, we discuss the process of radiomics, its myriad applications such as diagnosis, prognosis, and prediction of therapy response. We also discuss the outcomes of studies involving radiomic analysis of PCLs and pancreatic cancer, and challenges associated with this novel field along with possible solutions. Although these studies highlight the potential benefit of radiomics in the prevention and optimal treatment of pancreatic cancer, further studies are warranted before incorporating radiomics into the clinical decision support system.

The feasibility of using citizens to segment anatomy from medical images: Accuracy and motivation

The development of automatic methods for segmenting anatomy from medical images is an important goal for many medical and healthcare research areas. Datasets that can be used to train and test computer algorithms, however, are often small due to the difficulties in obtaining experts to segment enough examples. Citizen science provides a potential solution to this problem but the feasibility of using the public to identify and segment anatomy in a medical image has not been investigated. Our study therefore aimed to explore the feasibility, in terms of performance and motivation, of using citizens for such purposes. Public involvement was woven into the study design and evaluation. Twenty-nine citizens were recruited and, after brief training, asked to segment the spine from a dataset of 150 magnetic resonance images. Participants segmented as many images as they could within three one-hour sessions. Their accuracy was evaluated by comparing them, as individuals and as a combined consensus, to the segmentations of three experts. Questionnaires and a focus group were used to determine the citizens' motivation for taking part and their experience of the study. Citizen segmentation accuracy, in terms of agreement with the expert consensus segmentation, varied considerably between individual citizens. The citizen consensus, however, was close to the expert consensus, indicating that when pooled, citizens may be able to replace or supplement experts for generating large image datasets. Personal interest and a desire to help were the two most common reasons for taking part in the study.

Optimizing Neuro-Oncology Imaging: A Review of Deep Learning Approaches for Glioma Imaging

Radiographic assessment with magnetic resonance imaging (MRI) is widely used to characterize gliomas, which represent 80% of all primary malignant brain tumors. Unfortunately, glioma biology is marked by heterogeneous angiogenesis, cellular proliferation, cellular invasion, and apoptosis. This translates into varying degrees of enhancement, edema, and necrosis, making reliable imaging assessment challenging. Deep learning, a subset of machine learning artificial intelligence, has gained traction as a method, which has seen effective employment in solving image-based problems, including those in medical imaging. This review seeks to summarize current deep learning applications used in the field of glioma detection and outcome prediction and will focus on (1) pre- and post-operative tumor segmentation, (2) genetic characterization of tissue, and (3) prognostication. We demonstrate that deep learning methods of segmenting, characterizing, grading, and predicting survival in gliomas are promising opportunities that may enhance both research and clinical activities.

Artificial intelligence and radiomics in pulmonary nodule management: current status and future applications

Artificial intelligence (AI) has been present in some guise within the field of radiology for over 50 years. The first studies investigating computer-aided diagnosis in thoracic radiology date back to the 1960s, and in the subsequent years, the main application of these techniques has been the detection and classification of pulmonary nodules. In addition, there have been other less intensely researched applications, such as the diagnosis of interstitial lung disease, chronic obstructive pulmonary disease, and the detection of pulmonary emboli. Despite extensive literature on the use of convolutional neural networks in thoracic imaging over the last few decades, we are yet to see these systems in use in clinical practice. The article reviews current state-of-the-art applications of AI and in detection, classification, and follow-up of pulmonary nodules and how deep-learning techniques might influence these going forward. Finally, we postulate the impact of these advancements on the role of radiologists and the importance of radiologists in the development and evaluation of these techniques.

Unsupervised abnormality detection through mixed structure regularization (MSR) in deep sparse autoencoders

PURPOSE

The purpose of this study is to introduce and evaluate the mixed structure regularization (MSR) approach for a deep sparse autoencoder aimed at unsupervised abnormality detection in medical images. Unsupervised abnormality detection based on identifying outliers using deep sparse autoencoders is a very appealing approach for computer-aided detection systems as it requires only healthy data for training rather than expert annotated abnormality. However, regularization is required to avoid overfitting of the network to the training data.

METHODS

We used coronary computed tomography angiography (CCTA) datasets of 90 subjects with expert annotated centerlines. We segmented coronary lumen and wall using an automatic algorithm with manual corrections where required. We defined normal coronary cross section as cross sections with a ratio between lumen and wall areas larger than 0.8. We divided the datasets into training, validation, and testing groups in a tenfold cross-validation scheme. We trained a deep sparse overcomplete autoencoder model for normality modeling with random structure and noise augmentation. We assessed the performance of our deep sparse autoencoder with MSR without denoising (SAE-MSR) and with denoising (SDAE-MSR) in comparison to deep sparse autoencoder (SAE), and deep sparse denoising autoencoder (SDAE) models in the task of detecting coronary artery disease from CCTA data on the test group.

RESULTS

The SDAE-MSR achieved the best aggregated area under the curve (AUC) with a 20% improvement and the best aggregated Average Precision (AP) with a 30% improvement upon the SAE and SDAE (AUC: 0.78 to 0.94, AP: 0.66 to 0.86) in distinguishing between coronary cross sections with mild stenosis (stenosis grade < 0.3) and coronary cross sections with severe stenosis (stenosis grade > 0.7). The improvements were statistically significant (Mann-Whitney U-test, P < 0.001). Similarly, The SDAE-MSR achieved the best aggregated AUC (AP) with an 18% (18%) improvement upon the SAE and SDAE (AUC: 0.71 to 0.84, AP: 0.68 to 0.80). The improvements were statistically significant (Mann-Whitney U-test, P < 0.05).

CONCLUSION

Deep sparse autoencoders with MSR in addition to explicit sparsity regularization term and stochastic corruption of the input data with Gaussian noise have the potential to improve unsupervised abnormality detection using deep-learning compared to common deep autoencoders.

A Hybrid Multilayer Filtering Approach for Thyroid Nodule Segmentation on Ultrasound Images

OBJECTIVES

Speckle noise is the main factor that degrades ultrasound image contrast and segmentation failure. Determining an effective filter can reduce speckle noise and improve segmentation performances. The aim of this study was to define a useful filter to improve the segmentation outcome.

METHODS

Twelve filters, including median, hybrid median (Hmed), Fourier Butterworth, Fourier ideal, wavelet (Wlet), homomorphic Fourier Butterworth, homomorphic Fourier ideal, homomorphic wavelet (Hmp_Wlet), frost, anisotropic diffusion, probabilistic patch-based (PPB), and homogeneous area filters, were used to find the best filter(s) to prepare thyroid nodule segmentation. A receiver operating characteristic (ROC) analysis was used for filter evaluation in the nodule segmentation process. Accordingly, 10 morphologic parameters were measured from segmented regions to find the best parameters that predict the segmentation performance.

RESULTS

The best segmentation performance was reached by using 4 hybrid filters that mainly contain contrast-limited adaptive histogram equalization, Wlet, Hmed, Hmp_Wlet, and PPB filters. The area under the ROC curve for these filters ranged from 0.900 to 0.943 in comparison with the original image, with an area under the curve of 0.685. From 10 morphologic parameters, the area, convex area, equivalent diameter, solidity, and extent can evaluate segmentation performance.

CONCLUSIONS

Hybrid filters that contain contrast-limited adaptive histogram equalization, Wlet, Hmed, Hmp_Wlet, and PPB filters have a high potential to provide good conditions for thyroid nodule segmentation in ultrasound images. In addition to an ROC analysis, morphometry of a segmented region can be used to evaluate segmentation performances.

Artificial intelligence in cancer imaging: Clinical challenges and applications

Judgement, as one of the core tenets of medicine, relies upon the integration of multilayered data with nuanced decision making. Cancer offers a unique context for medical decisions given not only its variegated forms with evolution of disease but also the need to take into account the individual condition of patients, their ability to receive treatment, and their responses to treatment. Challenges remain in the accurate detection, characterization, and monitoring of cancers despite improved technologies. Radiographic assessment of disease most commonly relies upon visual evaluations, the interpretations of which may be augmented by advanced computational analyses. In particular, artificial intelligence (AI) promises to make great strides in the qualitative interpretation of cancer imaging by expert clinicians, including volumetric delineation of tumors over time, extrapolation of the tumor genotype and biological course from its radiographic phenotype, prediction of clinical outcome, and assessment of the impact of disease and treatment on adjacent organs. AI may automate processes in the initial interpretation of images and shift the clinical workflow of radiographic detection, management decisions on whether or not to administer an intervention, and subsequent observation to a yet to be envisioned paradigm. Here, the authors review the current state of AI as applied to medical imaging of cancer and describe advances in 4 tumor types (lung, brain, breast, and prostate) to illustrate how common clinical problems are being addressed. Although most studies evaluating AI applications in oncology to date have not been vigorously validated for reproducibility and generalizability, the results do highlight increasingly concerted efforts in pushing AI technology to clinical use and to impact future directions in cancer care.

CAD system based on B-mode and color Doppler sonographic features may predict if a thyroid nodule is hot or cold

OBJECTIVES

The aim of this study was to evaluate if the analysis of sonographic parameters could predict if a thyroid nodule was hot or cold.

METHODS

Overall, 102 thyroid nodules, including 51 hyperfunctioning (hot) and 51 hypofunctioning (cold) nodules, were evaluated in this study. Twelve sonographic features (i.e., seven B-mode and five Doppler features) were extracted for each nodule type. The isthmus thickness, nodule volume, echogenicity, margin, internal component, microcalcification, and halo sign features were obtained in the B-mode, while the vascularity pattern, resistive index (RI), peak systolic velocity, end diastolic velocity, and peak systolic/end diastolic velocity ratio (SDR) were determined, based on Doppler ultrasounds. All significant features were incorporated in the computer-aided diagnosis (CAD) system to classify hot and cold nodules.

RESULTS

Among all sonographic features, only isthmus thickness, nodule volume, echogenicity, RI, and SDR were significantly different between hot and cold nodules. Based on these features in the training dataset, the CAD system could classify hot and cold nodules with an area under the curve (AUC) of 0.898. Also, in the test dataset, hot and cold nodules were classified with an AUC of 0.833.

CONCLUSIONS

2D sonographic features could differentiate hot and cold thyroid nodules. The CAD system showed a great potential to achieve it automatically.

KEY POINTS

• Cold nodules represent higher volume (p = 0.005), isthmus thickness (p = 0.035), RI (p = 0.020), and SDR (p = 0.044) and appear hypoechogenic (p = 0.010) in US. • Nodule volume with an AUC of 0.685 and resistive index with an AUC of 0.628 showed the highest classification potential among all B-mode and Doppler features respectively. • The proposed CAD system could distinguish hot nodules from cold ones with an AUC of 0.833 (sensitivity 90.00%, specificity 70.00%, accuracy 80.00%, PPV 87.50%, and NPV 75.00%).

Automated Segmentation Methods of Drusen to Diagnose Age-Related Macular Degeneration Screening in Retinal Images

Existing drusen measurement is difficult to use in clinic because it requires a lot of time and effort for visual inspection. In order to resolve this problem, we propose an automatic drusen detection method to help clinical diagnosis of age-related macular degeneration. First, we changed the fundus image to a green channel and extracted the ROI of the macular area based on the optic disk. Next, we detected the candidate group using the difference image of the median filter within the ROI. We also segmented vessels and removed them from the image. Finally, we detected the drusen through Renyi's entropy threshold algorithm. We performed comparisons and statistical analysis between the manual detection results and automatic detection results for 30 cases in order to verify validity. As a result, the average sensitivity was 93.37% (80.95%~100%) and the average DSC was 0.73 (0.3~0.98). In addition, the value of the ICC was 0.984 (CI: 0.967~0.993, p < 0.01), showing the high reliability of the proposed automatic method. We expect that the automatic drusen detection helps clinicians to improve the diagnostic performance in the detection of drusen on fundus image.

OIPAV: an Integrated Software System for Ophthalmic Image Processing, Analysis, and Visualization

Ophthalmic medical images, such as optical coherence tomography (OCT) images and color photo of fundus, provide valuable information for clinical diagnosis and treatment of ophthalmic diseases. In this paper, we introduce a software system specially oriented to ophthalmic images processing, analysis, and visualization (OIPAV) to assist users. OIPAV is a cross-platform system built on a set of powerful and widely used toolkit libraries. Based on the plugin mechanism, the system has an extensible framework. It provides rich functionalities including data I/O, image processing, interaction, ophthalmic diseases detection, data analysis, and visualization. By using OIPAV, users can easily access to the ophthalmic image data manufactured from different imaging devices, facilitate workflows of processing ophthalmic images, and improve quantitative evaluations. With a satisfying function scalability and expandability, the software is applicable for both ophthalmic researchers and clinicians.

Computational Fluid Dynamic Study for aTAA Hemodynamics: An Integrated Image-Based and Radial Basis Functions Mesh Morphing Approach

We present a novel framework for the fluid dynamics analysis of healthy subjects and patients affected by ascending thoracic aorta aneurysm (aTAA). Our aim is to obtain indications about the effect of a bulge on the hemodynamic environment at different enlargements. Three-dimensional (3D) surface models defined from healthy subjects and patients with aTAA, selected for surgical repair, were generated. A representative shape model for both healthy and pathological groups has been identified. A morphing technique based on radial basis functions (RBF) was applied to mold the shape relative to healthy patient into the representative shape of aTAA dataset to enable the parametric simulation of the aTAA formation. Computational fluid dynamics (CFD) simulations were performed by means of a finite volume solver using the mean boundary conditions obtained from three-dimensional (PC-MRI) acquisition. Blood flow helicity and flow descriptors were assessed for all the investigated models. The feasibility of the proposed integrated approach pertaining the coupling between an RBF morphing technique and CFD simulation for aTAA was demonstrated. Significant hemodynamic changes appear at the 60% of the bulge progression. An impingement of the flow toward the bulge was observed by analyzing the normalized flow eccentricity (NFE) index.

Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks

Automatic segmentation of abdominal anatomy on computed tomography (CT) images can support diagnosis, treatment planning, and treatment delivery workflows. Segmentation methods using statistical models and multi-atlas label fusion (MALF) require inter-subject image registrations, which are challenging for abdominal images, but alternative methods without registration have not yet achieved higher accuracy for most abdominal organs. We present a registration-free deep-learning-based segmentation algorithm for eight organs that are relevant for navigation in endoscopic pancreatic and biliary procedures, including the pancreas, the gastrointestinal tract (esophagus, stomach, and duodenum) and surrounding organs (liver, spleen, left kidney, and gallbladder). We directly compared the segmentation accuracy of the proposed method to the existing deep learning and MALF methods in a cross-validation on a multi-centre data set with 90 subjects. The proposed method yielded significantly higher Dice scores for all organs and lower mean absolute distances for most organs, including Dice scores of 0.78 versus 0.71, 0.74, and 0.74 for the pancreas, 0.90 versus 0.85, 0.87, and 0.83 for the stomach, and 0.76 versus 0.68, 0.69, and 0.66 for the esophagus. We conclude that the deep-learning-based segmentation represents a registration-free method for multi-organ abdominal CT segmentation whose accuracy can surpass current methods, potentially supporting image-guided navigation in gastrointestinal endoscopy procedures.

Development and Validation of Segmentation Method for Lung Cancer Volumetry on Chest CT

The set of criteria called Response Evaluation Criteria In Solid Tumors (RECIST) is used to evaluate the remedial effects of lung cancer, whereby the size of a lesion can be measured in one dimension (diameter). Volumetric evaluation is desirable for estimating the size of a lesion accurately, but there are several constraints and limitations to calculating the volume in clinical trials. In this study, we developed a method to detect lesions automatically, with minimal intervention by the user, and calculate their volume. Our proposed method, called a spherical region-growing method (SPRG), uses segmentation that starts from a seed point set by the user. SPRG is a modification of an existing region-growing method that is based on a sphere instead of pixels. The SPRG method detects lesions while preventing leakage to neighboring tissues, because the sphere is grown, i.e., neighboring voxels are added, only when all the voxels meet the required conditions. In this study, two radiologists segmented lung tumors using a manual method and the proposed method, and the results of both methods were compared. The proposed method showed a high sensitivity of 81.68-84.81% and a high dice similarity coefficient (DSC) of 0.86-0.88 compared with the manual method. In addition, the SPRG intraclass correlation coefficient (ICC) was 0.998 (CI 0.997-0.999, p < 0.01), showing that the SPRG method is highly reliable. If our proposed method is used for segmentation and volumetric measurement of lesions, then objective and accurate results and shorter data analysis time are possible.

Artificial intelligence in radiology

Artificial intelligence (AI) algorithms, particularly deep learning, have demonstrated remarkable progress in image-recognition tasks. Methods ranging from convolutional neural networks to variational autoencoders have found myriad applications in the medical image analysis field, propelling it forward at a rapid pace. Historically, in radiology practice, trained physicians visually assessed medical images for the detection, characterization and monitoring of diseases. AI methods excel at automatically recognizing complex patterns in imaging data and providing quantitative, rather than qualitative, assessments of radiographic characteristics. In this Opinion article, we establish a general understanding of AI methods, particularly those pertaining to image-based tasks. We explore how these methods could impact multiple facets of radiology, with a general focus on applications in oncology, and demonstrate ways in which these methods are advancing the field. Finally, we discuss the challenges facing clinical implementation and provide our perspective on how the domain could be advanced.

How random is the random forest? Random forest algorithm on the service of structural imaging biomarkers for Alzheimer's disease: from Alzheimer's disease neuroimaging initiative (ADNI) database

Neuroinformatics is a fascinating research field that applies computational models and analytical tools to high dimensional experimental neuroscience data for a better understanding of how the brain functions or dysfunctions in brain diseases. Neuroinformaticians work in the intersection of neuroscience and informatics supporting the integration of various sub-disciplines (behavioural neuroscience, genetics, cognitive psychology, etc.) working on brain research. Neuroinformaticians are the pathway of information exchange between informaticians and clinicians for a better understanding of the outcome of computational models and the clinical interpretation of the analysis. Machine learning is one of the most significant computational developments in the last decade giving tools to neuroinformaticians and finally to radiologists and clinicians for an automatic and early diagnosis-prognosis of a brain disease. Random forest (RF) algorithm has been successfully applied to high-dimensional neuroimaging data for feature reduction and also has been applied to classify the clinical label of a subject using single or multi-modal neuroimaging datasets. Our aim was to review the studies where RF was applied to correctly predict the Alzheimer's disease (AD), the conversion from mild cognitive impairment (MCI) and its robustness to overfitting, outliers and handling of non-linear data. Finally, we described our RF-based model that gave us the 1st position in an international challenge for automated prediction of MCI from MRI data.

Automated chest screening based on a hybrid model of transfer learning and convolutional sparse denoising autoencoder

Objective

In this paper, we aim to investigate the effect of computer-aided triage system, which is implemented for the health checkup of lung lesions involving tens of thousands of chest X-rays (CXRs) that are required for diagnosis. Therefore, high accuracy of diagnosis by an automated system can reduce the radiologist’s workload on scrutinizing the medical images.

Method

We present a deep learning model in order to efficiently detect abnormal levels or identify normal levels during mass chest screening so as to obtain the probability confidence of the CXRs. Moreover, a convolutional sparse denoising autoencoder is designed to compute the reconstruction error. We employ four publicly available radiology datasets pertaining to CXRs, analyze their reports, and utilize their images for mining the correct disease level of the CXRs that are to be submitted to a computer aided triaging system. Based on our approach, we vote for the final decision from multi-classifiers to determine which three levels of the images (i.e. normal, abnormal, and uncertain cases) that the CXRs fall into.

Results

We only deal with the grade diagnosis for physical examination and propose multiple new metric indices. Combining predictors for classification by using the area under a receiver operating characteristic curve, we observe that the final decision is related to the threshold from reconstruction error and the probability value. Our method achieves promising results in terms of precision of 98.7 and 94.3% based on the normal and abnormal cases, respectively.

Conclusion

The results achieved by the proposed framework show superiority in classifying the disease level with high accuracy. This can potentially save the radiologists time and effort, so as to allow them to focus on higher-level risk CXRs.

The changing face of cancer diagnosis: From computational image analysis to systems biology

ᅟ: KEY POINTS: • Radiomics and radiogenomics will merge radiology, nuclear medicine, pathology and laboratory medicine. • Automation of image data analysis will change the daily routine work. • Image-guided therapy and handling complex radiogenomic data will play a major role.