Multisource Transfer Learning With Convolutional Neural Networks for Lung Pattern Analysis

Real-time gastric polyp detection using convolutional neural networks

Computer-aided polyp detection in gastric gastroscopy has been the subject of research over the past few decades. However, despite significant advances, automatic polyp detection in real time is still an unsolved problem. In this paper, we report on a convolutional neural network (CNN) for polyp detection that is constructed based on Single Shot MultiBox Detector (SSD) architecture and which we call SSD for Gastric Polyps (SSD-GPNet). To take full advantages of feature maps' information from the feature pyramid and to acquire higher accuracy, we re-use information that is abandoned by Max-Pooling layers. In other words, we reuse the lost data from the pooling layers and concatenate that data as extra feature maps to contribute to classification and detection. Meanwhile, in the feature pyramid, we concatenate feature maps of the lower layers and feature maps that are deconvolved from upper layers to make explicit relationships between layers and to effectively increase the number of channels. The results show that our enhanced SSD for gastric polyp detection can realize real-time polyp detection with 50 frames per second (FPS) and can improve the mean average precision (mAP) from 88.5% to 90.4%, with only a little loss in time-performance. And the further experiment shows that SSD-GPNet has excellent performance in improving polyp detection recalls over 10% (p = 0.00053), especially in small polyp detection. This can help endoscopic physicians more easily find missed polyps and decrease the gastric polyp miss rate. It may be applicable in daily clinical practice to reduce the burden on physicians.

Immunotherapy Associated Pulmonary Toxicity: Biology Behind Clinical and Radiological Features

The broader use of immune checkpoint blockade in clinical routine challenges clinicians in the diagnosis and management of side effects which are caused by inflammation generated by the activation of the immune response. Nearly all organs can be affected by immune-related toxicities. However, the most frequently reported are: fatigue, rash, pruritus, diarrhea, nausea/vomiting, arthralgia, decreased appetite and abdominal pain. Although these adverse events are usually mild, reversible and not frequent, an early diagnosis is crucial. Immune-related pulmonary toxicity was most frequently observed in trials of lung cancer and of melanoma patients treated with the combination of the anti-cytotoxic T lymphocyte antigen (CTLA)-4 and the anti-programmed cell death-1 (PD-1) antibodies. The most frequent immune-related adverse event in the lung is represented by pneumonitis due to the development of infiltrates in the interstitium and in the alveoli. Clinical symptoms and radiological patterns are the key elements to be considered for an early diagnosis, rendering the differential diagnosis crucial. Diagnosis of immune-related pneumonitis may imply the temporary or definitive suspension of immunotherapy, along with the start of immuno-suppressive treatments. The aim of this work is to summarize the biological bases, clinical and radiological findings of lung toxicity under immune checkpoint blockade, underlining the importance of multidisciplinary teams for an optimal early diagnosis of this side effect, with the aim to reach an improved patient care.

Convolutional Neural Networks for Radiologic Images: A Radiologist's Guide

Deep learning has rapidly advanced in various fields within the past few years and has recently gained particular attention in the radiology community. This article provides an introduction to deep learning technology and presents the stages that are entailed in the design process of deep learning radiology research. In addition, the article details the results of a survey of the application of deep learning-specifically, the application of convolutional neural networks-to radiologic imaging that was focused on the following five major system organs: chest, breast, brain, musculoskeletal system, and abdomen and pelvis. The survey of the studies is followed by a discussion about current challenges and future trends and their potential implications for radiology. This article may be used as a guide for radiologists planning research in the field of radiologic image analysis using convolutional neural networks.

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.

Deep CNN models for pulmonary nodule classification: Model modification, model integration, and transfer learning

BACKGROUND

Deep learning has made spectacular achievements in analysing natural images, but it faces challenges for medical applications partly due to inadequate images.

OBJECTIVE

Aiming to classify malignant and benign pulmonary nodules using CT images, we explore different strategies to utilize the state-of-the-art deep convolutional neural networks (CNN).

METHODS

Experiments are conducted using the Lung Image Database Consortium image collection (LIDC-IDRI), which is a public database containing 1018 cases. Three strategies are implemented including to 1) modify some state-of-the-art CNN architectures, 2) integrate different CNNs and 3) adopt transfer learning. Totally, 11 deep CNN models are compared using the same dataset.

RESULTS

Study demonstrates that, for the model modification scheme, a concise CifarNet performs better than the other modified CNNs with more complex architectures, achieving an area under ROC curve of AUC = 0.90. Integrated CNN models do not significantly improve the classification performance, but the model complexity is reduced. Transfer learning outperforms the other two schemes and ResNet with fine-tuning leads to the best performance with an AUC = 0.94, as well as the sensitivity of 91% and an overall accuracy of 88%.

CONCLUSIONS

Model modification, model integration, and transfer learning can play important roles to identify and generate optimal deep CNN models in classifying pulmonary nodules based on CT images efficiently. Transfer learning is preferred when applying deep learning to medical imaging applications.

Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning

Bronchoscopy inspection, as a follow-up procedure next to the radiological imaging, plays a key role in the diagnosis and treatment design for lung disease patients. When performing bronchoscopy, doctors have to make a decision immediately whether to perform a biopsy. Because biopsies may cause uncontrollable and life-threatening bleeding of the lung tissue, thus doctors need to be selective with biopsies. In this paper, to help doctors to be more selective on biopsies and provide a second opinion on diagnosis, we propose a computer-aided diagnosis (CAD) system for lung diseases, including cancers and tuberculosis (TB). Based on transfer learning (TL), we propose a novel TL method on the top of DenseNet: sequential fine-tuning (SFT). Compared with traditional fine-tuning (FT) methods, our method achieves the best performance. In a data set of recruited 81 normal cases, 76 TB cases and 277 lung cancer cases, SFT provided an overall accuracy of 82% while other traditional TL methods achieved an accuracy from 70% to 74%. The detection accuracy of SFT for cancers, TB, and normal cases are 87%, 54%, and 91%, respectively. This indicates that the CAD system has the potential to improve lung disease diagnosis accuracy in bronchoscopy and it may be used to be more selective with biopsies.

Gastric Pathology Image Classification Using Stepwise Fine-Tuning for Deep Neural Networks

Deep learning using convolutional neural networks (CNNs) is a distinguished tool for many image classification tasks. Due to its outstanding robustness and generalization, it is also expected to play a key role to facilitate advanced computer-aided diagnosis (CAD) for pathology images. However, the shortage of well-annotated pathology image data for training deep neural networks has become a major issue at present because of the high-cost annotation upon pathologist's professional observation. Faced with this problem, transfer learning techniques are generally used to reinforcing the capacity of deep neural networks. In order to further boost the performance of the state-of-the-art deep neural networks and alleviate insufficiency of well-annotated data, this paper presents a novel stepwise fine-tuning-based deep learning scheme for gastric pathology image classification and establishes a new type of target-correlative intermediate datasets. Our proposed scheme is deemed capable of making the deep neural network imitating the pathologist's perception manner and of acquiring pathology-related knowledge in advance, but with very limited extra cost in data annotation. The experiments are conducted with both well-annotated gastric pathology data and the proposed target-correlative intermediate data on several state-of-the-art deep neural networks. The results congruously demonstrate the feasibility and superiority of our proposed scheme for boosting the classification performance.

EMPHYSEMA CLASSIFICATION USING A MULTI-VIEW CONVOLUTIONAL NETWORK

In this article we propose and validate a fully automatic tool for emphysema classification in Computed Tomography (CT) images. We hypothesize that a relatively simple Convolutional Neural Network (CNN) architecture can learn even better discriminative features from the input data compared with more complex and deeper architectures. The proposed architecture is comprised of only 4 convolutional and 3 pooling layers, where the input corresponds to a 2.5D multiview representation of the pulmonary segment tissue to classify, corresponding to axial, sagittal and coronal views. The proposed architecture is compared to similar 2D CNN and 3D CNN, and to more complex architectures which involve a larger number of parameters (up to six times larger). This method has been evaluated in 1553 tissue samples, and achieves an overall sensitivity of 81.78 % and a specificity of 97.34%, and results show that the proposed method outperforms deeper state-of-the-art architectures particularly designed for lung pattern classification. The method shows satisfactory results in full-lung classification.

Semantic Segmentation of Pathological Lung Tissue With Dilated Fully Convolutional Networks

Early and accurate diagnosis of interstitial lung diseases (ILDs) is crucial for making treatment decisions, but can be challenging even for experienced radiologists. The diagnostic procedure is based on the detection and recognition of the different ILD pathologies in thoracic CT scans, yet their manifestation often appears similar. In this study, we propose the use of a deep purely convolutional neural network for the semantic segmentation of ILD patterns, as the basic component of a computer aided diagnosis system for ILDs. The proposed CNN, which consists of convolutional layers with dilated filters, takes as input a lung CT image of arbitrary size and outputs the corresponding label map. We trained and tested the network on a data set of 172 sparsely annotated CT scans, within a cross-validation scheme. The training was performed in an end-to-end and semisupervised fashion, utilizing both labeled and nonlabeled image regions. The experimental results show significant performance improvement with respect to the state of the art.

Computational biology: deep learning

Deep learning is the trendiest tool in a computational biologist's toolbox. This exciting class of methods, based on artificial neural networks, quickly became popular due to its competitive performance in prediction problems. In pioneering early work, applying simple network architectures to abundant data already provided gains over traditional counterparts in functional genomics, image analysis, and medical diagnostics. Now, ideas for constructing and training networks and even off-the-shelf models have been adapted from the rapidly developing machine learning subfield to improve performance in a range of computational biology tasks. Here, we review some of these advances in the last 2 years.