Fully Convolutional Architectures for Multiclass Segmentation in Chest Radiographs

DR-Net: An Improved Network for Building Extraction from High Resolution Remote Sensing Image

At present, convolutional neural networks (CNN) have been widely used in building extraction from remote sensing imagery (RSI), but there are still some bottlenecks. On the one hand, there are so many parameters in the previous network with complex structure, which will occupy lots of memories and consume much time during training process. On the other hand, low-level features extracted by shallow layers and abstract features extracted by deep layers of artificial neural network cannot be fully fused, which leads to an inaccurate building extraction from RSI. To alleviate these disadvantages, a dense residual neural network (DR-Net) was proposed in this paper. DR-Net uses a deeplabv3+Net encoder/decoder backbone, in combination with densely connected convolution neural network (DCNN) and residual network (ResNet) structure. Compared with deeplabv3+net (containing about 41 million parameters) and BRRNet (containing about 17 million parameters), DR-Net contains about 9 million parameters; So, the number of parameters reduced a lot. The experimental results for both the WHU Building Dataset and Massachusetts Building Dataset, DR-Net show better performance in building extraction than other two state-of-the-art methods. Experiments on WHU building data set showed that Intersection over Union (IoU) increased by 2.4% and F1 score increased by 1.4%; in terms of Massachusetts Building Dataset, IoU increased by 3.8% and F1 score increased by 2.9%.

Bayesian Learning of Shifted-Scaled Dirichlet Mixture Models and Its Application to Early COVID-19 Detection in Chest X-ray Images

Early diagnosis and assessment of fatal diseases and acute infections on chest X-ray (CXR) imaging may have important therapeutic implications and reduce mortality. In fact, many respiratory diseases have a serious impact on the health and lives of people. However, certain types of infections may include high variations in terms of contrast, size and shape which impose a real challenge on classification process. This paper introduces a new statistical framework to discriminate patients who are either negative or positive for certain kinds of virus and pneumonia. We tackle the current problem via a fully Bayesian approach based on a flexible statistical model named shifted-scaled Dirichlet mixture models (SSDMM). This mixture model is encouraged by its effectiveness and robustness recently obtained in various image processing applications. Unlike frequentist learning methods, our developed Bayesian framework has the advantage of taking into account the uncertainty to accurately estimate the model parameters as well as the ability to solve the problem of overfitting. We investigate here a Markov Chain Monte Carlo (MCMC) estimator, which is a computer–driven sampling method, for learning the developed model. The current work shows excellent results when dealing with the challenging problem of biomedical image classification. Indeed, extensive experiments have been carried out on real datasets and the results prove the merits of our Bayesian framework.

Open-Source Automatic Segmentation of Ocular Structures and Biomarkers of Microbial Keratitis on Slit-Lamp Photography Images Using Deep Learning

We propose a fully-automatic deep learning-based algorithm for segmentation of ocular structures and microbial keratitis (MK) biomarkers on slit-lamp photography (SLP) images. The dataset consisted of SLP images from 133 eyes with manual annotations by a physician, P1. A modified region-based convolutional neural network, SLIT-Net, was developed and trained using P1's annotations to identify and segment four pathological regions of interest (ROIs) on diffuse white light images (stromal infiltrate (SI), hypopyon, white blood cell (WBC) border, corneal edema border), one pathological ROI on diffuse blue light images (epithelial defect (ED)), and two non-pathological ROIs on all images (corneal limbus, light reflexes). To assess inter-reader variability, 75 eyes were manually annotated for pathological ROIs by a second physician, P2. Performance was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Using seven-fold cross-validation, the DSC of the algorithm (as compared to P1) for all ROIs was good (range: 0.62-0.95) on all 133 eyes. For the subset of 75 eyes with manual annotations by P2, the DSC for pathological ROIs ranged from 0.69-0.85 (SLIT-Net) vs. 0.37-0.92 (P2). DSCs for SLIT-Net were not significantly different than P2 for segmenting hypopyons (p > 0.05) and higher than P2 for WBCs (p < 0.001) and edema (p < 0.001). DSCs were higher for P2 for segmenting SIs (p < 0.001) and EDs (p < 0.001). HDs were lower for P2 for segmenting SIs (p = 0.005) and EDs (p < 0.001) and not significantly different for hypopyons (p > 0.05), WBCs (p > 0.05), and edema (p > 0.05). This prototype fully-automatic algorithm to segment MK biomarkers on SLP images performed to expectations on an exploratory dataset and holds promise for quantification of corneal physiology and pathology.

Online Learning of Finite and Infinite Gamma Mixture Models for COVID-19 Detection in Medical Images

The accurate detection of abnormalities in medical images (like X-ray and CT scans) is a challenging problem due to images’ blurred boundary contours, different sizes, variable shapes, and uneven density. In this paper, we tackle this problem via a new effective online variational learning model for both mixtures of finite and infinite Gamma distributions. The proposed approach takes advantage of the Gamma distribution flexibility, the online learning scalability, and the variational inference efficiency. Three different batch and online learning methods based on robust texture-based feature extraction are proposed. Our work is evaluated and validated on several real challenging data sets for different kinds of pneumonia infection detection. The obtained results are very promising given that we approach the classification problem in an unsupervised manner. They also confirm the superiority of the Gamma mixture model compared to the Gaussian mixture model for medical images’ classification.

Automated segmentation of retinal nonperfusion area in fluorescein angiography in retinal vein occlusion using convolutional neural networks

PURPOSE

Retinal vein occlusion (RVO) is the second most common cause of vision loss after diabetic retinopathy due to retinal vascular disease. Retinal nonperfusion (RNP), identified on fluorescein angiograms (FA) and appearing as hypofluorescence regions, is one of the most significant characteristics of RVO. Quantification of RNP is crucial for assessing the severity and progression of RVO. However, in current clinical practice, it is mostly conducted manually, which is time-consuming, subjective, and error-prone. The purpose of this study is to develop fully automated methods for segmentation of RNP using convolutional neural networks (CNNs).

METHODS

FA images from 161 patients were analyzed, and RNP areas were annotated by three independent physicians. The optimal method to use multi-physicians' labeled data to train the CNNs was evaluated. An adaptive histogram-based data augmentation method was utilized to boost the CNN performance. CNN methods based on context encoder module were developed for automated segmentation of RNP and compared with existing state-of-the-art methods.

RESULTS

The proposed methods achieved excellent agreements with physicians for segmentation of RNP in FA images. The CNN performance can be improved significantly by the proposed adaptive histogram-based data augmentation method. Using the averaged labels from physicians to train the CNNs achieved the best consensus with all physicians, with a mean accuracy of 0.883±0.166 with fivefold cross-validation.

CONCLUSIONS

We reported CNN methods to segment RNP in RVO in FA images. Our work can help improve clinical workflow, and can be useful for further investigating the association between RNP and retinal disease progression, as well as for evaluating the optimal treatments for the management of RVO.

Automated segmentation and diagnosis of pneumothorax on chest X-rays with fully convolutional multi-scale ScSE-DenseNet: a retrospective study

Background

Pneumothorax (PTX) may cause a life-threatening medical emergency with cardio-respiratory collapse that requires immediate intervention and rapid treatment. The screening and diagnosis of pneumothorax usually rely on chest radiographs. However, the pneumothoraces in chest X-rays may be very subtle with highly variable in shape and overlapped with the ribs or clavicles, which are often difficult to identify. Our objective was to create a large chest X-ray dataset for pneumothorax with pixel-level annotation and to train an automatic segmentation and diagnosis framework to assist radiologists to identify pneumothorax accurately and timely.

Methods

In this study, an end-to-end deep learning framework is proposed for the segmentation and diagnosis of pneumothorax on chest X-rays, which incorporates a fully convolutional DenseNet (FC-DenseNet) with multi-scale module and spatial and channel squeezes and excitation (scSE) modules. To further improve the precision of boundary segmentation, we propose a spatial weighted cross-entropy loss function to penalize the target, background and contour pixels with different weights.

Results

This retrospective study are conducted on a total of eligible 11,051 front-view chest X-ray images (5566 cases of PTX and 5485 cases of Non-PTX). The experimental results show that the proposed algorithm outperforms the five state-of-the-art segmentation algorithms in terms of mean pixel-wise accuracy (MPA) with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.93\pm 0.13$$\end{document}0.93±0.13 and dice similarity coefficient (DSC) with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$0.92\pm 0.14$$\end{document}0.92±0.14, and achieves competitive performance on diagnostic accuracy with 93.45% and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1-score with 92.97%.

Conclusion

This framework provides substantial improvements for the automatic segmentation and diagnosis of pneumothorax and is expected to become a clinical application tool to help radiologists to identify pneumothorax on chest X-rays.

Detection of peripherally inserted central catheter (PICC) in chest X-ray images: A multi-task deep learning model

BACKGROUND AND OBJECTIVE

Peripherally inserted central catheter (PICC) is a novel drug delivery mode which has been widely used in clinical practice. However, long-term retention and some improper actions of patients may cause some severe complications of PICC, such as the drift and prolapse of its catheter. Clinically, the postoperative care of PICC is mainly completed by nurses. However, they cannot recognize the correct position of PICC from X-ray chest images as soon as the complications happen, which may lead to improper treatment. Therefore, it is necessary to identify the position of the PICC catheter as soon as these complications occur. Here we proposed a novel multi-task deep learning framework to detect PICC automatically through X-ray images, which could help nurses to solve this problem.

METHODS

We collected 348 X-ray chest images from 326 patients with visible PICC. Then we proposed a multi-task deep learning framework for line segmentation and tip detection of PICC catheters simultaneously. The proposed deep learning model is composed of an extraction structure and three routes, an up-sampling route for segmentation, an RPNs route, and an RoI Pooling route for detection. We further compared the effectiveness of our model with the models previously proposed.

RESULTS

In the catheter segmentation task, 300 X-ray images were utilized for training the model, then 48 images were tested. In the tip detection task, 154 X-ray images were used for retraining and 20 images were used in the test. Our model achieved generally better results among several popular deep learning models previously proposed.

CONCLUSIONS

We proposed a multi-task deep learning model that could segment the catheter and detect the tip of PICC simultaneously from X-ray chest images. This model could help nurses to recognize the correct position of PICC, and therefore, to handle the potential complications properly.

Post-DAE: Anatomically Plausible Segmentation via Post-Processing With Denoising Autoencoders

We introduce Post-DAE, a post-processing method based on denoising autoencoders (DAE) to improve the anatomical plausibility of arbitrary biomedical image segmentation algorithms. Some of the most popular segmentation methods (e.g. based on convolutional neural networks or random forest classifiers) incorporate additional post-processing steps to ensure that the resulting masks fulfill expected connectivity constraints. These methods operate under the hypothesis that contiguous pixels with similar aspect should belong to the same class. Even if valid in general, this assumption does not consider more complex priors like topological restrictions or convexity, which cannot be easily incorporated into these methods. Post-DAE leverages the latest developments in manifold learning via denoising autoencoders. First, we learn a compact and non-linear embedding that represents the space of anatomically plausible segmentations. Then, given a segmentation mask obtained with an arbitrary method, we reconstruct its anatomically plausible version by projecting it onto the learnt manifold. The proposed method is trained using unpaired segmentation mask, what makes it independent of intensity information and image modality. We performed experiments in binary and multi-label segmentation of chest X-ray and cardiac magnetic resonance images. We show how erroneous and noisy segmentation masks can be improved using Post-DAE. With almost no additional computation cost, our method brings erroneous segmentations back to a feasible space.

Fine-Tuning U-Net for Ultrasound Image Segmentation: Different Layers, Different Outcomes

One way of resolving the problem of scarce and expensive data in deep learning for medical applications is using transfer learning and fine-tuning a network which has been trained on a large data set. The common practice in transfer learning is to keep the shallow layers unchanged and to modify deeper layers according to the new data set. This approach may not work when using a U-Net and when moving from a different domain to ultrasound (US) images due to their drastically different appearance. In this study, we investigated the effect of fine-tuning different sets of layers of a pretrained U-Net for US image segmentation. Two different schemes were analyzed, based on two different definitions of shallow and deep layers. We studied simulated US images, as well as two human US data sets. We also included a chest X-ray data set. The results showed that choosing which layers to fine-tune is a critical task. In particular, they demonstrated that fine-tuning the last layers of the network, which is the common practice for classification networks, is often the worst strategy. It may therefore be more appropriate to fine-tune the shallow layers rather than deep layers in US image segmentation when using a U-Net. Shallow layers learn lower level features which are critical in automatic segmentation of medical images. Even when a large US data set is available, we observed that fine-tuning shallow layers is a faster approach compared to fine-tuning the whole network.

CheXLocNet: Automatic localization of pneumothorax in chest radiographs using deep convolutional neural networks

BACKGROUND

Pneumothorax can lead to a life-threatening emergency. The experienced radiologists can offer precise diagnosis according to the chest radiographs. The localization of the pneumothorax lesions will help to quickly diagnose, which will be benefit for the patients in the underdevelopment areas lack of the experienced radiologists. In recent years, with the development of large neural network architectures and medical imaging datasets, deep learning methods have become a methodology of choice for analyzing medical images. The objective of this study was to the construct convolutional neural networks to localize the pneumothorax lesions in chest radiographs.

METHODS AND FINDINGS

We developed a convolutional neural network, called CheXLocNet, for the segmentation of pneumothorax lesions. The SIIM-ACR Pneumothorax Segmentation dataset was used to train and validate CheXLocNets. The training dataset contained 2079 radiographs with the annotated lesion areas. We trained six CheXLocNets with various hyperparameters. Another 300 annotated radiographs were used to select parameters of these CheXLocNets as the validation set. We determined the optimal parameters by the AP50 (average precision at the intersection over union (IoU) equal to 0.50), a segmentation evaluation metric used by several well-known competitions. Then CheXLocNets were evaluated by a test set (1082 normal radiographs and 290 disease radiographs), based on the classification metrics: area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and positive predictive value (PPV); segmentation metrics: IoU and Dice score. For the classification, CheXLocNet with best sensitivity produced an AUC of 0.87, sensitivity of 0.78 (95% CI 0.73-0.83), and specificity of 0.78 (95% CI 0.76-0.81). CheXLocNet with best specificity produced an AUC of 0.79, sensitivity of 0.46 (95% CI 0.40-0.52), and specificity of 0.92 (95% CI 0.90-0.94). For the segmentation, CheXLocNet with best sensitivity produced an IoU of 0.69 and Dice score of 0.72. CheXLocNet with best specificity produced an IoU of 0.77 and Dice score of 0.79. We combined them to form an ensemble CheXLocNet. The ensemble CheXLocNet produced an IoU of 0.81 and Dice score of 0.82. Our CheXLocNet succeeded in automatically detecting pneumothorax lesions, without any human guidance.

CONCLUSIONS

In this study, we proposed a deep learning network, called, CheXLocNet, for the automatic segmentation of chest radiographs to detect pneumothorax. Our CheXLocNets generated accurate classification results and high-quality segmentation masks for the pneumothorax at the same time. This technology has the potential to improve healthcare delivery and increase access to chest radiograph expertise for the detection of diseases. Furthermore, the segmentation results can offer comprehensive geometric information of lesions, which can benefit monitoring the sequential development of lesions with high accuracy. Thus, CheXLocNets can be further extended to be a reliable clinical decision support tool. Although we used transfer learning in training CheXLocNet, the parameters of CheXLocNet was still large for the radiograph dataset. Further work is necessary to prune CheXLocNet suitable for the radiograph dataset.

Triple attention learning for classification of 14 thoracic diseases using chest radiography

Chest X-ray is the most common radiology examinations for the diagnosis of thoracic diseases. However, due to the complexity of pathological abnormalities and lack of detailed annotation of those abnormalities, computer-aided diagnosis (CAD) of thoracic diseases remains challenging. In this paper, we propose the triple-attention learning (A ³ Net) model for this CAD task. This model uses the pre-trained DenseNet-121 as the backbone network for feature extraction, and integrates three attention modules in a unified framework for channel-wise, element-wise, and scale-wise attention learning. Specifically, the channel-wise attention prompts the deep model to emphasize the discriminative channels of feature maps; the element-wise attention enables the deep model to focus on the regions of pathological abnormalities; the scale-wise attention facilitates the deep model to recalibrate the feature maps at different scales. The proposed model has been evaluated on 112,120images in the ChestX-ray14 dataset with the official patient-level data split. Compared to state-of-the-art deep learning models, our model achieves the highest per-class AUC in classifying 13 out of 14 thoracic diseases and the highest average per-class AUC of 0.826 over 14 thoracic diseases.

Computer-Aided System for the Detection of Multicategory Pulmonary Tuberculosis in Radiographs

The early screening and diagnosis of tuberculosis plays an important role in the control and treatment of tuberculosis infections. In this paper, an integrated computer-aided system based on deep learning is proposed for the detection of multiple categories of tuberculosis lesions in chest radiographs. In this system, the fully convolutional neural network method is used to segment the lung area from the entire chest radiograph for pulmonary tuberculosis detection. Different from the previous analysis of the whole chest radiograph, we focus on the specific tuberculosis lesion areas for the analysis and propose the first multicategory tuberculosis lesion detection method. In it, a learning scalable pyramid structure is introduced into the Faster Region-based Convolutional Network (Faster RCNN), which effectively improves the detection of small-area lesions, mines indistinguishable samples during the training process, and uses reinforcement learning to reduce the detection of false-positive lesions. To compare our method with the current tuberculosis detection system, we propose a classification rule for whole chest X-rays using a multicategory tuberculosis lesion detection model and achieve good performance on two public datasets (Montgomery: AUC = 0.977 and accuracy = 0.926; Shenzhen: AUC = 0.941 and accuracy = 0.902). Our proposed computer-aided system is superior to current systems that can be used to assist radiologists in diagnoses and public health providers in screening for tuberculosis in areas where tuberculosis is endemic.

Automated thalamic nuclei segmentation using multi-planar cascaded convolutional neural networks

PURPOSE

To develop a fast and accurate convolutional neural network based method for segmentation of thalamic nuclei.

METHODS

A cascaded multi-planar scheme with a modified residual U-Net architecture was used to segment thalamic nuclei on conventional and white-matter-nulled (WMn) magnetization prepared rapid gradient echo (MPRAGE) data. A single network was optimized to work with images from healthy controls and patients with multiple sclerosis (MS) and essential tremor (ET), acquired at both 3 T and 7 T field strengths. WMn-MPRAGE images were manually delineated by a trained neuroradiologist using the Morel histological atlas as a guide to generate reference ground truth labels. Dice similarity coefficient and volume similarity index (VSI) were used to evaluate performance. Clinical utility was demonstrated by applying this method to study the effect of MS on thalamic nuclei atrophy.

RESULTS

Segmentation of each thalamus into twelve nuclei was achieved in under a minute. For 7 T WMn-MPRAGE, the proposed method outperforms current state-of-the-art on patients with ET with statistically significant improvements in Dice for five nuclei (increase in the range of 0.05-0.18) and VSI for four nuclei (increase in the range of 0.05-0.19), while performing comparably for healthy and MS subjects. Dice and VSI achieved using 7 T WMn-MPRAGE data are comparable to those using 3 T WMn-MPRAGE data. For conventional MPRAGE, the proposed method shows a statistically significant Dice improvement in the range of 0.14-0.63 over FreeSurfer for all nuclei and disease types. Effect of noise on network performance shows robustness to images with SNR as low as half the baseline SNR. Atrophy of four thalamic nuclei and whole thalamus was observed for MS patients compared to healthy control subjects, after controlling for the effect of parallel imaging, intracranial volume, gender, and age (p < 0.004).

CONCLUSION

The proposed segmentation method is fast, accurate, performs well across disease types and field strengths, and shows great potential for improving our understanding of thalamic nuclei involvement in neurological diseases.

Self-paced DenseNet with boundary constraint for automated multi-organ segmentation on abdominal CT images

Automated multi-organ segmentation on abdominal CT images may replace or complement manual segmentation for clinical applications including image-guided radiation therapy. However, the accuracy of auto-segmentation is challenged by low image contrast, large spatial and inter-patient anatomical variations. In this study, we propose an end-to-end segmentation network, termed self-paced DenseNet, for improved multi-organ segmentation performance, especially for the difficult-to-segment organs. Specifically, a learning-based attention mechanism and dense connection block are seamlessly integrated into the proposed self-paced DenseNet to improve the learning capability and efficiency of the backbone network. To heavily focus on the organs showing low soft-tissue contrast and motion artifacts, a boundary condition is utilized to constrain the network optimization. Additionally, to ease the large learning pace discrepancies of individual organs, a task-wise self-paced-learning strategy is employed to adaptively control the learning paces of individual organs. The proposed self-paced DenseNet was trained and evaluated on a public abdominal CT data set consisting of 90 subjects with manually labeled ground truths of eight organs (including spleen, left kidney, esophagus, gallbladder, stomach, liver, pancreas, and duodenum). For quantitative evaluation, the Dice similarity coefficient (DSC) and average surface distance (ASD) were calculated. An average DSC of 84.46% and ASD of 1.82 mm were achieved on the eight organs, which outperforms the state-of-the-art segmentation methods 2.96% on DSC under the same experimental configuration. Moreover, the proposed segmentation method shows notable improvements on the duodenum and gallbladder, obtaining an average DSC of 69.26% and 80.94% and ASD of 2.14 mm and 2.24 mm, respectively. The results are markedly superior to the average DSC of 63.12% and 76.35% and average ASD of 3.87 mm and 4.33 mm using the vanilla DenseNet, respectively, for the two organs. We demonstrated the effectiveness of the proposed self-paced DenseNet to automatically segment abdominal organs with low boundary conspicuity. The self-paced DenseNet achieved consistently superior segmentation performance on eight abdominal organs with varying segmentation difficulties. The demonstrated computational efficiency (<2 s/CT) makes it well-suited for online applications.

Dense-Unet: a light model for lung fields segmentation in Chest X-Ray images

Automatic and accurate lung segmentation in chest X-ray (CXR) images is fundamental for computer-aided diagnosis systems since the lung is the region of interest in many diseases and also it can reveal useful information by its contours. While deep learning models have reached high performances in the segmentation of anatomical structures, the large number of training parameters is a concern since it increases memory usage and reduces the generalization of the model. To address this, a deep CNN model called Dense-Unet is proposed in which, by dense connectivity between various layers, information flow increases throughout the network. This lets us design a network with significantly fewer parameters while keeping the segmentation robust. To the best of our knowledge, Dense-Unet is the lightest deep model proposed for the segmentation of lung fields in CXR images. The model is evaluated on the JSRT and Montgomery datasets and experiments show that the performance of the proposed model is comparable with state-of-the-art methods.

Lung Region Segmentation in Chest X-Ray Images using Deep Convolutional Neural Networks

Lung cancer is, by far, the leading cause of cancer death in the world. Tools for automated medical imaging analysis development of a Computer-Aided Diagnosis method comprises several tasks. In general, the first one is the segmentation of region of interest, for example, lung region segmentation from Chest X-ray imaging in the task of detecting lung cancer. Deep Convolutional Neural Networks (DCNN) have shown promising results in the task of segmentation in medical images. In this paper, to implement the lung region segmentation task on chest X-ray images, was evaluated three different DCNN architectures in association with different regularization (Dropout, L2, and Dropout + L2) and optimization methods (SGDM, RMSPROP and ADAM). All networks were applied in the Japanese Society of Radiological Technology (JSRT) database. The best results were obtained using Dropout + L2 as regularization method and ADAM as optimization method. Considering the Jaccard Coefficient obtained (0.97967 ± 0.00232) the proposal outperforms the state of the art.Clinical Relevance- The presented method reduces the time that a professional takes to perform lung segmentation, improving the effectiveness.

Assessment of an ensemble of machine learning models toward abnormality detection in chest radiographs

Respiratory diseases account for a significant proportion of deaths and disabilities across the world. Chest X-ray (CXR) analysis remains a common diagnostic imaging modality for confirming intra-thoracic cardiopulmonary abnormalities. However, there remains an acute shortage of expert radiologists, particularly in under-resourced settings, resulting in severe interpretation delays. These issues can be mitigated by a computer-aided diagnostic (CADx) system to supplement decision-making and improve throughput while preserving and possibly improving the standard-of-care. Systems reported in the literature or popular media use handcrafted features and/or data-driven algorithms like deep learning (DL) to learn underlying data distributions. The remarkable success of convolutional neural networks (CNN) toward image recognition tasks has made them a promising choice for automated medical image analyses. However, CNNs suffer from high variance and may overfit due to their sensitivity to training data fluctuations. Ensemble learning helps to reduce this variance by combining predictions of multiple learning algorithms to construct complex, non-linear functions and improve robustness and generalization. This study aims to construct and assess the performance of an ensemble of machine learning (ML) models applied to the challenge of classifying normal and abnormal CXRs and significantly reducing the diagnostic load of radiologists and primary-care physicians.

Automated Quantification of Photoreceptor alteration in macular disease using Optical Coherence Tomography and Deep Learning

Diabetic macular edema (DME) and retina vein occlusion (RVO) are macular diseases in which central photoreceptors are affected due to pathological accumulation of fluid. Optical coherence tomography allows to visually assess and evaluate photoreceptor integrity, whose alteration has been observed as an important biomarker of both diseases. However, the manual quantification of this layered structure is challenging, tedious and time-consuming. In this paper we introduce a deep learning approach for automatically segmenting and characterising photoreceptor alteration. The photoreceptor layer is segmented using an ensemble of four different convolutional neural networks. En-face representations of the layer thickness are produced to characterize the photoreceptors. The pixel-wise standard deviation of the score maps produced by the individual models is also taken to indicate areas of photoreceptor abnormality or ambiguous results. Experimental results showed that our ensemble is able to produce results in pair with a human expert, outperforming each of its constitutive models. No statistically significant differences were observed between mean thickness estimates obtained from automated and manually generated annotations. Therefore, our model is able to reliable quantify photoreceptors, which can be used to improve prognosis and managment of macular diseases.

Artificial Intelligence-Based Diagnosis of Cardiac and Related Diseases

Automatic chest anatomy segmentation plays a key role in computer-aided disease diagnosis, such as for cardiomegaly, pleural effusion, emphysema, and pneumothorax. Among these diseases, cardiomegaly is considered a perilous disease, involving a high risk of sudden cardiac death. It can be diagnosed early by an expert medical practitioner using a chest X-Ray (CXR) analysis. The cardiothoracic ratio (CTR) and transverse cardiac diameter (TCD) are the clinical criteria used to estimate the heart size for diagnosing cardiomegaly. Manual estimation of CTR and other diseases is a time-consuming process and requires significant work by the medical expert. Cardiomegaly and related diseases can be automatically estimated by accurate anatomical semantic segmentation of CXRs using artificial intelligence. Automatic segmentation of the lungs and heart from the CXRs is considered an intensive task owing to inferior quality images and intensity variations using nonideal imaging conditions. Although there are a few deep learning-based techniques for chest anatomy segmentation, most of them only consider single class lung segmentation with deep complex architectures that require a lot of trainable parameters. To address these issues, this study presents two multiclass residual mesh-based CXR segmentation networks, X-RayNet-1 and X-RayNet-2, which are specifically designed to provide fine segmentation performance with a few trainable parameters compared to conventional deep learning schemes. The proposed methods utilize semantic segmentation to support the diagnostic procedure of related diseases. To evaluate X-RayNet-1 and X-RayNet-2, experiments were performed with a publicly available Japanese Society of Radiological Technology (JSRT) dataset for multiclass segmentation of the lungs, heart, and clavicle bones; two other publicly available datasets, Montgomery County (MC) and Shenzhen X-Ray sets (SC), were evaluated for lung segmentation. The experimental results showed that X-RayNet-1 achieved fine performance for all datasets and X-RayNet-2 achieved competitive performance with a 75% parameter reduction.

Detection and visualization of abnormality in chest radiographs using modality-specific convolutional neural network ensembles

Convolutional neural networks (CNNs) trained on natural images are extremely successful in image classification and localization due to superior automated feature extraction capability. In extending their use to biomedical recognition tasks, it is important to note that visual features of medical images tend to be uniquely different than natural images. There are advantages offered through training these networks on large scale medical common modality image collections pertaining to the recognition task. Further, improved generalization in transferring knowledge across similar tasks is possible when the models are trained to learn modality-specific features and then suitably repurposed for the target task. In this study, we propose modality-specific ensemble learning toward improving abnormality detection in chest X-rays (CXRs). CNN models are trained on a large-scale CXR collection to learn modality-specific features and then repurposed for detecting and localizing abnormalities. Model predictions are combined using different ensemble strategies toward reducing prediction variance and sensitivity to the training data while improving overall performance and generalization. Class-selective relevance mapping (CRM) is used to visualize the learned behavior of the individual models and their ensembles. It localizes discriminative regions of interest (ROIs) showing abnormal regions and offers an improved explanation of model predictions. It was observed that the model ensembles demonstrate superior localization performance in terms of Intersection of Union (IoU) and mean Average Precision (mAP) metrics than any individual constituent model.

Multicontext multitask learning networks for mass detection in mammogram

PURPOSE

In this paper, for the purpose of accurate and efficient mass detection, we propose a new deep learning framework, including two major stages: Suspicious region localization (SRL) and Multicontext Multitask Learning (MCMTL).

METHODS

In the first stage, SRL focuses on finding suspicious regions [regions of interest (ROIs)] and extracting multisize patches of these suspicious regions. A set of bounding boxes with different size is used to extract multisize patches, which aim to capture diverse context information. In the second stage, MCMTL networks integrate features from multisize patches of suspicious regions for classification and segmentation simultaneously, where the purpose of this stage is to keep the true positive suspicious regions and to reduce the false positive suspicious regions.

RESULTS

According to the experimental results on two public datasets (i.e., CBIS-DDSM and INBreast), our method achieves the overall performance of 0.812 TPR@2.53 FPI and 0.919 TPR@0.12 FPI on test sets, respectively.

CONCLUSIONS

Our proposed method suggests comparable performance to the state-of-the-art methods.

Assessment of Data Augmentation Strategies Toward Performance Improvement of Abnormality Classification in Chest Radiographs

Image augmentation is a commonly performed technique to prevent class imbalance in datasets to compensate for insufficient training samples, or to prevent model overfitting. Traditional augmentation (TA) techniques include various image transformations, such as rotation, translation, channel splitting, etc. Alternatively, Generative Adversarial Network (GAN), due to its proven ability to synthesize convincingly-realistic images, has been used to perform image augmentation as well. However, it is unclear whether GAN augmentation (GA) strategy provides an advantage over TA for medical image classification tasks. In this paper, we study the usefulness of TA and GA for classifying abnormal chest X-ray (CXR) images. We first trained a progressive-growing GAN (PG-GAN) to synthesize high-resolution CXRs for performing GA. Then, we trained an abnormality classifier using three training sets individually - training set with TA, with GA and with no augmentation (NA). Finally, we analyzed the abnormality classifier's performance for the three training cases, which led to the following conclusions: (1) GAN strategy is not always superior to TA for improving the classifier's performance; (2) in comparison to NA, however, both TA and GA leads to a significant performance improvement; and, (3) increasing the quantity of images in TA and GA strategies also improves the classifier's performance.

Multilabel segmentation of cancer cell culture on vascular structures with deep neural networks

New increasingly complex in vitro cancer cell models are being developed. These new models seem to represent the cell behavior in vivo more accurately and have better physiological relevance than prior models. An efficient testing method for selecting the most optimal drug treatment does not exist to date. One proposed solution to the problem involves isolation of cancer cells from the patients' cancer tissue, after which they are exposed to potential drugs alone or in combinations to find the most optimal medication. To achieve this goal, methods that can efficiently quantify and analyze changes in tested cell are needed. Our study aimed to detect and segment cells and structures from cancer cell cultures grown on vascular structures in phase-contrast microscope images using U-Net neural networks to enable future drug efficacy assessments. We cultivated prostate carcinoma cell lines PC3 and LNCaP on the top of a matrix containing vascular structures. The cells were imaged with a Cell-IQ phase-contrast microscope. Automatic analysis of microscope images could assess the efficacy of tested drugs. The dataset included 36 RGB images and ground-truth segmentations with mutually not exclusive classes. The used method could distinguish vascular structures, cells, spheroids, and cell matter around spheroids in the test images. Some invasive spikes were also detected, but the method could not distinguish the invasive cells in the test images.

Image-to-Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography

Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widely-used for diagnosis, screening, and treatment follow up of diseases related to lungs and heart. The literature in this field of research reports many interesting studies dealing with the challenging tasks of bone suppression and organ segmentation but performed separately, limiting any learning that comes with the consolidation of parameters that could optimize both processes. This study, and for the first time, introduces a multitask deep learning model that generates simultaneously the bone-suppressed image and the organ-segmented image, enhancing the accuracy of tasks, minimizing the number of parameters needed by the model and optimizing the processing time, all by exploiting the interplay between the network parameters to benefit the performance of both tasks. The architectural design of this model, which relies on a conditional generative adversarial network, reveals the process on how the wellestablished pix2pix network (image-to-image network) is modified to fit the need for multitasking and extending it to the new image-to-images architecture. The developed source code of this multitask model is shared publicly on Github as the first attempt for providing the two-task pix2pix extension, a supervised/paired/aligned/registered image-to-images translation which would be useful in many multitask applications. Dilated convolutions are also used to improve the results through a more effective receptive field assessment. The comparison with state-of-the-art al-gorithms along with ablation study and a demonstration video1 are provided to evaluate the efficacy and gauge the merits of the proposed approach.

Contour-aware multi-label chest X-ray organ segmentation

PURPOSE

Segmentation of organs from chest X-ray images is an essential task for an accurate and reliable diagnosis of lung diseases and chest organ morphometry. In this study, we investigated the benefits of augmenting state-of-the-art deep convolutional neural networks (CNNs) for image segmentation with organ contour information and evaluated the performance of such augmentation on segmentation of lung fields, heart, and clavicles from chest X-ray images.

METHODS

Three state-of-the-art CNNs were augmented, namely the UNet and LinkNet architecture with the ResNeXt feature extraction backbone, and the Tiramisu architecture with the DenseNet. All CNN architectures were trained on ground-truth segmentation masks and additionally on the corresponding contours. The contribution of such contour-based augmentation was evaluated against the contour-free architectures, and 20 existing algorithms for lung field segmentation.

RESULTS

The proposed contour-aware segmentation improved the segmentation performance, and when compared against existing algorithms on the same publicly available database of 247 chest X-ray images, the UNet architecture with the ResNeXt50 encoder combined with the contour-aware approach resulted in the best overall segmentation performance, achieving a Jaccard overlap coefficient of 0.971, 0.933, and 0.903 for the lung fields, heart, and clavicles, respectively.

CONCLUSION

In this study, we proposed to augment CNN architectures for CXR segmentation with organ contour information and were able to significantly improve segmentation accuracy and outperform all existing solution using a public chest X-ray database.

An effective approach for CT lung segmentation using mask region-based convolutional neural networks

Computer vision systems have numerous tools to assist in various medical fields, notably in image diagnosis. Computed tomography (CT) is the principal imaging method used to assist in the diagnosis of diseases such as bone fractures, lung cancer, heart disease, and emphysema, among others. Lung cancer is one of the four main causes of death in the world. The lung regions in the CT images are marked manually by a specialist as this initial step is a significant challenge for computer vision techniques. Once defined, the lung regions are segmented for clinical diagnoses. This work proposes an automatic segmentation of the lungs in CT images, using the Convolutional Neural Network (CNN) Mask R-CNN, to specialize the model for lung region mapping, combined with supervised and unsupervised machine learning methods (Bayes, Support Vectors Machine (SVM), K-means and Gaussian Mixture Models (GMMs)). Our approach using Mask R-CNN with the K-means kernel produced the best results for lung segmentation reaching an accuracy of 97.68 ± 3.42% and an average runtime of 11.2 s. We compared our results against other works for validation purposes, and our approach had the highest accuracy and was faster than some state-of-the-art methods.

Degradation Mechanism Detection in Photovoltaic Backsheets by Fully Convolutional Neural Network

Materials and devices age with time. Material aging and degradation has important implications for lifetime performance of materials and systems. While consensus exists that materials should be studied and designed for degradation, materials inspection during operation is typically performed manually by technicians. The manual inspection makes studies prone to errors and uncertainties due to human subjectivity. In this work, we focus on automating the process of degradation mechanism detection through the use of a fully convolutional deep neural network architecture (F-CNN). We demonstrate that F-CNN architecture allows for automated inspection of cracks in polymer backsheets from photovoltaic (PV) modules. The developed F-CNN architecture enabled an end-to-end semantic inspection of the PV module backsheets by applying a contracting path of convolutional blocks (encoders) followed by an expansive path of decoding blocks (decoders). First, the hierarchy of contextual features is learned from the input images by encoders. Next, these features are reconstructed to the pixel-level prediction of the input by decoders. The structure of the encoder and the decoder networks are thoroughly investigated for the multi-class pixel-level degradation type prediction for PV module backsheets. The developed F-CNN framework is validated by reporting degradation type prediction accuracy for the pixel level prediction at the level of 92.8%.

Using deep-learning techniques for pulmonary-thoracic segmentations and improvement of pneumonia diagnosis in pediatric chest radiographs

PURPOSE

To evaluate the efficacy of a deep-learning model to segment the lung and thorax regions in pediatric chest X-rays (CXRs). Validating the diagnosis of bacterial or viral pneumonia could be improved after lung segmentation.

MATERIALS AND METHODS

A clinical-pediatric CXR set including 1351 patients was proposed to develop a deep-learning model for the pulmonary-thoracic segmentations. Model performance was evaluated by Jaccard's similarity coefficient (JSC) and Dice's coefficient (DC). Two adult CXR sets were used to assess the model's generalizability. According to the pulmonary-thoracic ratio, Pearson's correlation coefficient and the Bland-Altman plot were generated to demonstrate the correlation and agreement between manual and automatic segmentations. The receiver operating characteristic curves and areas under the curve (AUCs) were used to compare the pneumonia classification performance based on the lung-extracted images with that based on the original images.

RESULTS

The model achieved JSCs of 0.910 and 0.950, DCs of 0.948 and 0.974 for lung and thorax segmentations, respectively. Pearson's r = 0.96, P < .0001. In the Bland-Altman plot, the mean difference was 0.0025 with a 95% confidence interval of (-0.0451, 0.0501). For testing with two adult CXR sets, the JSCs were 0.903 and 0.888, respectively, while the DCs were 0.948 and 0.937, respectively. After lung segmentation, the AUC of a classifier to identify bacterial or viral pneumonia increased from 0.815 to 0.879.

CONCLUSION

We built a pediatric CXR dataset and exploited a deep-learning model for accurate pulmonary-thoracic segmentations. Lung segmentation can notably improve the diagnosis of bacterial or viral pneumonia.

Identifying pulmonary nodules or masses on chest radiography using deep learning: external validation and strategies to improve clinical practice

AIM

To test the diagnostic performance of a deep learning-based system for the detection of clinically significant pulmonary nodules/masses on chest radiographs.

MATERIALS AND METHODS

Using a retrospective study of 100 patients (47 with clinically significant pulmonary nodules/masses and 53 control subjects without pulmonary nodules), two radiologists verified clinically significantly pulmonary nodules/masses according to chest computed tomography (CT) findings. A computer-aided diagnosis (CAD) software using a deep-learning approach was used to detect pulmonary nodules/masses to determine the diagnostic performance in four algorithms (heat map, abnormal probability, nodule probability, and mass probability).

RESULTS

A total of 100 cases were included in the analysis. Among the four algorithms, mass algorithm could achieve a 76.6% sensitivity (36/47, 11 false negative) and 88.68% specificity (47/53, six false-positive) in the detection of pulmonary nodules/masses at the optimal probability score cut-off of 0.2884. Compared to the other three algorithms, mass probability algorithm had best predictive ability for pulmonary nodule/mass detection at the optimal probability score cut-off of 0.2884 (AUC^Mass: 0.916 versus AUC^{Heat map}: 0.682, p<0.001; AUC^Mass: 0.916 versus AUC^Abnormal: 0.810, p=0.002; AUC^Mass: 0.916 versus AUC^Nodule: 0.813, p=0.014).

CONCLUSION

In conclusion, the deep-learning based computer-aided diagnosis system will likely play a vital role in the early detection and diagnosis of pulmonary nodules/masses on chest radiographs. In future applications, these algorithms could support triage workflow via double reading to improve sensitivity and specificity during the diagnostic process.

Enhanced Point-of-Care Ultrasound Applications by Integrating Automated Feature-Learning Systems Using Deep Learning

Recent applications of artificial intelligence (AI) and deep learning (DL) in health care include enhanced diagnostic imaging modalities to support clinical decisions and improve patients' outcomes. Focused on using automated DL-based systems to improve point-of-care ultrasound (POCUS), we look at DL-based automation as a key field in expanding and improving POCUS applications in various clinical settings. A promising additional value would be the ability to automate training model selections for teaching POCUS to medical trainees and novice sonologists. The diversity of POCUS applications and ultrasound equipment, each requiring specialized AI models and domain expertise, limits the use of DL as a generic solution. In this article, we highlight the most advanced potential applications of AI in POCUS tailored to high-yield models in automated image interpretations, with the premise of improving the accuracy and efficacy of POCUS scans.

Deep Sequential Segmentation of Organs in Volumetric Medical Scans

Segmentation in 3-D scans is playing an increasingly important role in current clinical practice supporting diagnosis, tissue quantification, or treatment planning. The current 3-D approaches based on convolutional neural networks usually suffer from at least three main issues caused predominantly by implementation constraints-first, they require resizing the volume to the lower-resolutional reference dimensions, and second, the capacity of such approaches is very limited due to memory restrictions, and third, all slices of volumes have to be available at any given training or testing time. We address these problems by a U-Net-like architecture consisting of bidirectional convolutional long short-term memory and convolutional, pooling, upsampling, and concatenation layers enclosed into time-distributed wrappers. Our network can either process the full volumes in a sequential manner or segment slabs of slices on demand. We demonstrate performance of our architecture on vertebrae and liver segmentation tasks in 3-D computed tomography scans.

Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks

PURPOSE

We address the automatic segmentation of healthy and cancerous liver tissues (parenchyma, active and necrotic parts of hepatocellular carcinoma (HCC) tumor) on multiphase CT images using a deep learning approach.

METHODS

We devise a cascaded convolutional neural network based on the U-Net architecture. Two strategies for dealing with multiphase information are compared: Single-phase images are concatenated in a multi-dimensional features map on the input layer, or output maps are computed independently for each phase before being merged to produce the final segmentation. Each network of the cascade is specialized in the segmentation of a specific tissue. The performances of these networks taken separately and of the cascaded architecture are assessed on both single-phase and on multiphase images.

RESULTS

In terms of Dice coefficients, the proposed method is on par with a state-of-the-art method designed for automatic MR image segmentation and outperforms previously used technique for interactive CT image segmentation. We validate the hypothesis that several cascaded specialized networks have a higher prediction accuracy than a single network addressing all tasks simultaneously. Although the portal venous phase alone seems to provide sufficient contrast for discriminating tumors from healthy parenchyma, the multiphase information brings significant improvement for the segmentation of cancerous tissues (active versus necrotic part).

CONCLUSION

The proposed cascaded multiphase architecture showed promising performances for the automatic segmentation of liver tissues, allowing to reliably estimate the necrosis rate, a valuable imaging biomarker of the clinical outcome.

A review on lung boundary detection in chest X-rays

PURPOSE

Chest radiography is the most common imaging modality for pulmonary diseases. Due to its wide usage, there is a rich literature addressing automated detection of cardiopulmonary diseases in digital chest X-rays (CXRs). One of the essential steps for automated analysis of CXRs is localizing the relevant region of interest, i.e., isolating lung region from other less relevant parts, for applying decision-making algorithms there. This article provides an overview of the recent literature on lung boundary detection in CXR images.

METHODS

We review the leading lung segmentation algorithms proposed in period 2006-2017. First, we present a review of articles for posterior-anterior view CXRs. Then, we mention studies which operate on lateral views. We pay particular attention to works that focus their efforts on deformed lungs and pediatric cases. We also highlight the radiographic measures extracted from lung boundary and their use in automatically detecting cardiopulmonary abnormalities. Finally, we identify challenges in dataset curation and expert delineation process, and we listed publicly available CXR datasets.

RESULTS

(1) We classified algorithms into four categories: rule-based, pixel classification-based, model-based, hybrid, and deep learning-based algorithms. Based on the reviewed articles, hybrid methods and deep learning-based methods surpass the algorithms in other classes and have segmentation performance as good as inter-observer performance. However, they require long training process and pose high computational complexity. (2) We found that most of the algorithms in the literature are evaluated on posterior-anterior view adult CXRs with a healthy lung anatomy appearance without considering challenges in abnormal CXRs. (3) We also found that there are limited studies for pediatric CXRs. The lung appearance in pediatrics, especially in infant cases, deviates from adult lung appearance due to the pediatric development stages. Moreover, pediatric CXRs are noisier than adult CXRs due to interference by other objects, such as someone holding the child's arms or the child's body, and irregular body pose. Therefore, lung boundary detection algorithms developed on adult CXRs may not perform accurately in pediatric cases and need additional constraints suitable for pediatric CXR imaging characteristics. (4) We have also stated that one of the main challenges in medical image analysis is accessing the suitable datasets. We listed benchmark CXR datasets for developing and evaluating the lung boundary algorithms. However, the number of CXR images with reference boundaries is limited due to the cumbersome but necessary process of expert boundary delineation.

CONCLUSIONS

A reliable computer-aided diagnosis system would need to support a greater variety of lung and background appearance. To our knowledge, algorithms in the literature are evaluated on posterior-anterior view adult CXRs with a healthy lung anatomy appearance, without considering ambiguous lung silhouettes due to pathological deformities, anatomical alterations due to misaligned body positioning, patient's development stage and gross background noises such as holding hands, jewelry, patient's head and legs in CXR. Considering all the challenges which are not very well addressed in the literature, developing lung boundary detection algorithms that are robust to such interference remains a challenging task. We believe that a broad review of lung region detection algorithms would be useful for researchers working in the field of automated detection/diagnosis algorithms for lung/heart pathologies in CXRs.

Computer-aided detection in chest radiography based on artificial intelligence: a survey

As the most common examination tool in medical practice, chest radiography has important clinical value in the diagnosis of disease. Thus, the automatic detection of chest disease based on chest radiography has become one of the hot topics in medical imaging research. Based on the clinical applications, the study conducts a comprehensive survey on computer-aided detection (CAD) systems, and especially focuses on the artificial intelligence technology applied in chest radiography. The paper presents several common chest X-ray datasets and briefly introduces general image preprocessing procedures, such as contrast enhancement and segmentation, and bone suppression techniques that are applied to chest radiography. Then, the CAD system in the detection of specific disease (pulmonary nodules, tuberculosis, and interstitial lung diseases) and multiple diseases is described, focusing on the basic principles of the algorithm, the data used in the study, the evaluation measures, and the results. Finally, the paper summarizes the CAD system in chest radiography based on artificial intelligence and discusses the existing problems and trends.