Automatic detection of subsolid pulmonary nodules in thoracic computed tomography images

A deep learning-based quantitative computed tomography model for predicting the severity of COVID-19: a retrospective study of 196 patients

BACKGROUND

The assessment of the severity of coronavirus disease 2019 (COVID-19) by clinical presentation has not met the urgent clinical need so far. We aimed to establish a deep learning (DL) model based on quantitative computed tomography (CT) and initial clinical features to predict the severity of COVID-19.

METHODS

One hundred ninety-six hospitalized patients with confirmed COVID-19 were enrolled from January 20 to February 10, 2020 in our centre, and were divided into severe and non-severe groups. The clinico-radiological data on admission were retrospectively collected and compared between the two groups. The optimal clinico-radiological features were determined based on least absolute shrinkage and selection operator (LASSO) logistic regression analysis, and a predictive nomogram model was established by five-fold cross-validation. Receiver operating characteristic (ROC) analyses were conducted, and the areas under the receiver operating characteristic curve (AUCs) of the nomogram model, quantitative CT parameters that were significant in univariate analysis, and pneumonia severity index (PSI) were compared.

RESULTS

In comparison with the non-severe group (151 patients), the severe group (45 patients) had a higher PSI (P<0.001). DL-based quantitative CT indicated that the mass of infection (MOI_CT) and the percentage of infection (POI_CT) in the whole lung were higher in the severe group (both P<0.001). The nomogram model was based on MOI_CT and clinical features, including age, cluster of differentiation 4 (CD4)⁺ T cell count, serum lactate dehydrogenase (LDH), and C-reactive protein (CRP). The AUC values of the model, MOI_CT, POI_CT, and PSI scores were 0.900, 0.813, 0.805, and 0.751, respectively. The nomogram model performed significantly better than the other three parameters in predicting severity (P=0.003, P=0.001, and P<0.001, respectively).

CONCLUSIONS

Although quantitative CT parameters and the PSI can well predict the severity of COVID-19, the DL-based quantitative CT model is more efficient.

Multiscale CNN with compound fusions for false positive reduction in lung nodule detection

Pulmonary lung nodules are often benign at the early stage but they could easily become malignant and metastasize to other locations in later stages. Morphological characteristics of these nodule instances vary largely in terms of their size, shape, and texture. There are also other co-existing lung anatomical structures such as lung walls and blood vessels surrounding these nodules resulting in complex contextual information. As a result, their early diagnosis to enable decisive intervention using Computer-Aided Diagnosis (CAD) systems face serious challenges, especially at low false positive rates. In this paper, we propose a new Convolutional Neural Network (CNN) architecture called Multiscale CNN with Compound Fusions (MCNN-CF) for this purpose which uses multiscale 3D patches as inputs and performs a fusion of intermediate features at two different depths of the network in two diverse fashions. The network is trained by a new iterative training procedure adapted to circumvent the class imbalance problem and obtained a Competitive Performance Metric (CPM) score of 0.948 when tested on the LUNA16 dataset. Experimental results illustrate the robustness of the proposed system which has increased the confidence of the prediction probabilities in the detection of the most variety of nodules.

MD-NDNet: a multi-dimensional convolutional neural network for false-positive reduction in pulmonary nodule detection

Pulmonary nodule false-positive reduction is of great significance for automated nodule detection in clinical diagnosis of low-dose computed tomography (LDCT) lung cancer screening. Due to individual intra-nodule variations and visual similarities between true nodules and false positives as soft tissues in LDCT images, the current clinical practices remain subject to shortcomings of potential high-risk and time-consumption issues. In this paper, we propose a multi-dimensional nodule detection network (MD-NDNet) for automatic nodule false-positive reduction using deep convolutional neural network (DCNNs). The underlying method collaboratively integrates multi-dimensional nodule information to complementarily and comprehensively extract nodule inter-plane volumetric correlation features using three-dimensional CNNs (3D CNNs) and spatial nodule correlation features from sagittal, coronal, and axial planes using two-dimensional CNNs (2D CNNs) with attention module. To incorporate different sizes and shapes of nodule candidates, a multi-scale ensemble strategy is employed for probability aggregation with weights. The proposed method is evaluated on the LUNA16 challenge dataset in ISBI 2016 with ten-fold cross-validation. Experiment results show that the proposed framework achieves classification performance with a CPM score of 0.9008. All of these indicate that our method enables an efficient, accurate and reliable pulmonary nodule detection for clinical diagnosis.

Multi-level 3D Densenets for False-positive Reduction in Lung Nodule Detection Based on Chest Computed Tomography

OBJECTIVE

False-positive nodule reduction is a crucial part of a computer-aided detection (CADe) system, which assists radiologists in accurate lung nodule detection. In this research, a novel scheme using multi-level 3D DenseNet framework is proposed to implement false-positive nodule reduction task.

METHODS

Multi-level 3D DenseNet models were extended to differentiate lung nodules from falsepositive nodules. First, different models were fed with 3D cubes with different sizes for encoding multi-level contextual information to meet the challenges of the large variations of lung nodules. In addition, image rotation and flipping were utilized to upsample positive samples which consisted of a positive sample set. Furthermore, the 3D DenseNets were designed to keep low-level information of nodules, as densely connected structures in DenseNet can reuse features of lung nodules and then boost feature propagation. Finally, the optimal weighted linear combination of all model scores obtained the best classification result in this research.

RESULTS

The proposed method was evaluated with LUNA16 dataset which contained 888 thin-slice CT scans. The performance was validated via 10-fold cross-validation. Both the Free-response Receiver Operating Characteristic (FROC) curve and the Competition Performance Metric (CPM) score show that the proposed scheme can achieve a satisfactory detection performance in the falsepositive reduction track of the LUNA16 challenge.

CONCLUSION

The result shows that the proposed scheme can be significant for false-positive nodule reduction task.

2D CNN versus 3D CNN for false-positive reduction in lung cancer screening

Purpose: To clarify whether and to what extent three-dimensional (3D) convolutional neural network (CNN) is superior to 2D CNN when applied to reduce false-positive nodule detections in the scenario of low-dose computed tomography (CT) lung cancer screening. Approach: We established a dataset consisting of 1600 chest CT examinations acquired on different subjects from various sources. There were in total 18,280 candidate nodules in these CT examinations, among which 9185 were nodules and 9095 were not nodules. For each candidate nodule, we extracted a number of cubic subvolumes with a dimension of 72 × 72 × 72    mm 3 by rotating the CT examinations randomly for 25 times prior to the extraction of the axis-aligned subvolumes. These subvolumes were split into three groups in a ratio of 8 ∶ 1 ∶ 1 for training, validation, and independent testing purposes. We developed a multiscale CNN architecture and implemented its 2D and 3D versions to classify pulmonary nodules into two categories, namely true positive and false positive. The performance of the 2D/3D-CNN classification schemes was evaluated using the area under the receiver operating characteristic curves (AUC). The p -values and the 95% confidence intervals (CI) were calculated. Results: The AUC for the optimal 2D-CNN model is 0.9307 (95% CI: 0.9285 to 0.9330) with a sensitivity of 92.70% and a specificity of 76.21%. The 3D-CNN model with the best performance had an AUC of 0.9541 (95% CI: 0.9495 to 0.9583) with a sensitivity of 89.98% and a specificity of 87.30%. The developed multiscale CNN architecture had a better performance than the vanilla architecture did. Conclusions: The 3D-CNN model has a better performance in false-positive reduction compared with its 2D counterpart; however, the improvement is relatively limited and demands more computational resources for training purposes.

Ensemble classification for predicting the malignancy level of pulmonary nodules on chest computed tomography images

Early identification and classification of pulmonary nodules are essential for improving the survival rates of individuals with lung cancer and are considered to be key requirements for computer-assisted diagnosis. To address this topic, the present study proposed a method for predicting the malignant phenotype of pulmonary nodules based on weighted voting rules. This method used the pulmonary nodule regions of interest as the input data and extracted the features of the pulmonary nodules using the Denoising Auto Encoder, ResNet-18. Moreover, the software also modifies texture and shape features to assess the malignant phenotype of the pulmonary nodules. Based on their classification accuracy (Acc), the different classifiers were assigned to different weights. Finally, an integrated classifier was obtained to score the malignant phenotype of the pulmonary nodules. The present study included training and testing experiments conducted by extracting the corresponding lung nodule image data from the Lung Image Database Consortium-Image Database Resource Initiative. The results of the present study indicated a final classification Acc of 93.10±2.4%, demonstrating the feasibility and effectiveness of the proposed method. This method includes the powerful feature extraction ability of deep learning combined with the ability to use traditional features in image representation.

Automated lung cancer diagnosis using three-dimensional convolutional neural networks

Lung cancer is the deadliest cancer worldwide. It has been shown that early detection using low-dose computer tomography (LDCT) scans can reduce deaths caused by this disease. We present a general framework for the detection of lung cancer in chest LDCT images. Our method consists of a nodule detector trained on the LIDC-IDRI dataset followed by a cancer predictor trained on the Kaggle DSB 2017 dataset and evaluated on the IEEE International Symposium on Biomedical Imaging (ISBI) 2018 Lung Nodule Malignancy Prediction test set. Our candidate extraction approach is effective to produce accurate candidates with a recall of 99.6%. In addition, our false positive reduction stage classifies successfully the candidates and increases precision by a factor of 2000. Our cancer predictor obtained a ROC AUC of 0.913 and was ranked 1st place at the ISBI 2018 Lung Nodule Malignancy Prediction challenge. Graphical abstract.

Quantitative computed tomography of the coronavirus disease 2019 (COVID-19) pneumonia

OBJECTIVE

To quantify coronavirus diseases 2019 (COVID-19) pneumonia and to explore whether quantitative computer tomography (CT) could be used to assess severity on admission.

MATERIALS AND METHODS

From January 17 to February 9, 2020, 38 hospitalized patients with COVID-19 pneumonia were consecutively enrolled in our hospitals. All clinical data and the chest CT on admission were retrospectively reviewed and analyzed. Firstly, a quantitative method based on multi-scale convolutional neural networks was used to assess the infected lung segments and this was compared with the semi-quantitative method. Secondly, the quantitative method was tested with laboratory results and the pneumonia severity index (PSI) by correlation analyses. Thirdly, both quantitative and semi-quantitative parameters between patients with different PSI were compared.

RESULTS

Thirty cases were finally enrolled: 16 (53.33%) of them were male, and the mean age was 48 years old. The interval from onset symptoms to first chest CT scan was 8 days. The proportion of ground glass opacity (GGO), consolidation and the total lesion based on the quantitative method was positively correlated with the semi-quantitative CT score (P < 0.001 for all; rs = 0.88, 0.87, 0.90), CRP (P = 0.0278, 0.0168, 0.0078; rs = 0.40, 0.43, 0.48) and ESR (P = 0.0296, 0.0408, 0.0048; rs = 0.46, 0.44, 0.58), respectively, and was negatively correlated with the lymphocyte count (P = 0.0222, 0.0024, 0.0068; rs = -0.42, -0.53, -0.48). There was a positive correlation trend between the proportion of total infection and the pneumonia severity index (P = 0.0994; rs = 0.30) and a tendency that patients with severe COVID-19 pneumonia had higher percentage of consolidation and total infection (P = 0.0903, 0.0989).

CONCLUSIONS

Quantitative CT may have potential in assessing the severity of COVID-19 pneumonia on admission.

3D-MCN: A 3D Multi-scale Capsule Network for Lung Nodule Malignancy Prediction

Despite the advances in automatic lung cancer malignancy prediction, achieving high accuracy remains challenging. Existing solutions are mostly based on Convolutional Neural Networks (CNNs), which require a large amount of training data. Most of the developed CNN models are based only on the main nodule region, without considering the surrounding tissues. Obtaining high sensitivity is challenging with lung nodule malignancy prediction. Moreover, the interpretability of the proposed techniques should be a consideration when the end goal is to utilize the model in a clinical setting. Capsule networks (CapsNets) are new and revolutionary machine learning architectures proposed to overcome shortcomings of CNNs. Capitalizing on the success of CapsNet in biomedical domains, we propose a novel model for lung tumor malignancy prediction. The proposed framework, referred to as the 3D Multi-scale Capsule Network (3D-MCN), is uniquely designed to benefit from: (i) 3D inputs, providing information about the nodule in 3D; (ii) Multi-scale input, capturing the nodule's local features, as well as the characteristics of the surrounding tissues, and; (iii) CapsNet-based design, being capable of dealing with a small number of training samples. The proposed 3D-MCN architecture predicted lung nodule malignancy with a high accuracy of 93.12%, sensitivity of 94.94%, area under the curve (AUC) of 0.9641, and specificity of 90% when tested on the LIDC-IDRI dataset. When classifying patients as having a malignant condition (i.e., at least one malignant nodule is detected) or not, the proposed model achieved an accuracy of 83%, and a sensitivity and specificity of 84% and 81% respectively.

Pulmonary nodule detection using hybrid two-stage 3D CNNs

PURPOSE

Early detection of pulmonary nodules is an effective way to improve patients' chances of survival. In this work, we propose a novel and efficient way to build a computer-aided detection (CAD) system for pulmonary nodules based on computed tomography (CT) scans.

METHODS

The system can be roughly divided into two steps: nodule candidate detection and false positive reduction. Considering the three-dimensional (3D) nature of nodules, the CAD system adopts 3D convolutional neural networks (CNNs) in both stages. Specifically, in the first stage, a segmentation-based 3D CNN with a hybrid loss is designed to segment nodules. According to the probability maps produced by the segmentation network, a threshold method and connected component analysis are applied to generate nodule candidates. In the second stage, we employ three classification-based 3D CNNs with different types of inputs to reduce false positives. In addition to simple raw data input, we also introduce hybrid inputs to make better use of the output of the previous segmentation network. In experiments, we use data augmentation and batch normalization to avoid overfitting.

RESULTS

We evaluate the system on 888 CT scans from the publicly available LIDC-IDRI dataset, and our method achieves the best performance by comparing with the state-of-the-art methods, which has a high detection sensitivity of 97.5% with an average of only one false positive per scan. An additional evaluation on 115 CT scans from local hospitals is also performed.

CONCLUSIONS

Experimental results demonstrate that our method is highly suited for the detection of pulmonary nodules.

Deep Learning for the Classification of Small (≤2 cm) Pulmonary Nodules on CT Imaging: A Preliminary Study

RATIONALE AND OBJECTIVES

We aimed to present a deep learning-based malignancy prediction model (CT-lungNET) that is simpler and faster to use in the diagnosis of small (≤2 cm) pulmonary nodules on nonenhanced chest CT and to preliminarily evaluate its performance and usefulness for human reviewers.

MATERIALS AND METHODS

A total of 173 whole nonenhanced chest CT images containing 208 pulmonary nodules (94 malignant and 11 benign nodules) ranging in size from 5 mm to 20 mm were collected. Pathologically confirmed nodules or nodules that remained unchanged for more than 1 year were included, and 30 benign and 30 malignant nodules were randomly assigned into the test set. We designed CT-lungNET to include three convolutional layers followed by two fully-connected layers and compared its diagnostic performance and processing time with those of AlexNET by using the area under the receiver operating curve (AUROC). An observer performance test was conducted involving eight human reviewers of four different groups (medical students, physicians, radiologic residents, and thoracic radiologists) at test 1 and test 2, referring to the CT-lungNET's malignancy prediction rate with pairwise comparison receiver operating curve analysis.

RESULTS

CT-lungNET showed an improved AUROC (0.85; 95% confidence interval: 0.74-0.93), compared to that of the AlexNET (0.82; 95% confidence interval: 0.71-0.91). The processing speed per one image slice for CT-lungNET was about 10 times faster than that for AlexNET (0.90 vs. 8.79 seconds). During the observer performance test, the classification performance of nonradiologists was increased with the aid of CTlungNET, (mean AUC improvement: 0.13; range: 0.03-0.19) but not significantly so in the radiologists group (mean AUC improvement: 0.02; range: -0.02 to 0.07).

CONCLUSION

CT-lungNET was able to provide better classification results with a significantly shorter amount of processing time as compared to AlexNET in the diagnosis of small pulmonary nodules on nonenhanced chest CT. In this preliminary observer performance test, CT-lungNET may have a role acting as a second reviewer for less experienced reviewers, resulting in enhanced performance in the diagnosis of early lung cancer.

Automatic Pulmonary Nodule Detection in CT Scans Using Convolutional Neural Networks Based on Maximum Intensity Projection

Accurate pulmonary nodule detection is a crucial step in lung cancer screening. Computer-aided detection (CAD) systems are not routinely used by radiologists for pulmonary nodule detection in clinical practice despite their potential benefits. Maximum intensity projection (MIP) images improve the detection of pulmonary nodules in radiological evaluation with computed tomography (CT) scans. Inspired by the clinical methodology of radiologists, we aim to explore the feasibility of applying MIP images to improve the effectiveness of automatic lung nodule detection using convolutional neural networks (CNNs). We propose a CNN-based approach that takes MIP images of different slab thicknesses (5 mm, 10 mm, 15 mm) and 1 mm axial section slices as input. Such an approach augments the two-dimensional (2-D) CT slice images with more representative spatial information that helps discriminate nodules from vessels through their morphologies. Our proposed method achieves sensitivity of 92.7% with 1 false positive per scan and sensitivity of 94.2% with 2 false positives per scan for lung nodule detection on 888 scans in the LIDC-IDRI dataset. The use of thick MIP images helps the detection of small pulmonary nodules (3 mm-10 mm) and results in fewer false positives. Experimental results show that utilizing MIP images can increase the sensitivity and lower the number of false positives, which demonstrates the effectiveness and significance of the proposed MIP-based CNNs framework for automatic pulmonary nodule detection in CT scans. The proposed method also shows the potential that CNNs could gain benefits for nodule detection by combining the clinical procedure.

Spectral analysis for pulmonary nodule detection using the optimal fractional S-Transform

Different frequency components of the lung, which have not been fully considered in traditional computer-aided detection systems for pulmonary nodules, can cause heterogeneous energy distribution. Hence, spectral analysis, which is an important time-frequency representation tool, is utilized to characterize the frequency-dependent energy responses of nodules. In this study, a novel spectral-analysis-based method for nodule candidate detection is presented. The optimal fractional S-transform is applied to transform raw computed tomography images from the spatial to time-frequency domain. Next, a time-frequency cube is decomposed using spectral decomposition to a frequency-dependent energy slice. Subsequently, an energy distribution is obtained by the Teager-Kaiser energy (TKE) to characterize the nodules. Finally, nodule candidates are detected using rule-based and threshold algorithms in the TKE image. The proposed method is validated on a clinical CT data set from Sichuan Provincial People's Hospital. The signal-to-clutter ratio (SCR) increases by 35.5% with respect to raw CT slices. Furthermore, the proposed method exhibits a sensitivity of 97.87%, with only 6.8 false positives per slice. The total number of nodule candidates has an average reduction of 50%. The results indicate that the time-frequency features can effectively characterize solid nodules. Moreover, the proposed method demonstrates accurate detection and can reduce the number of false positive efficiently.

Deep Learning for Lung Cancer Nodules Detection and Classification in CT Scans

Detecting malignant lung nodules from computed tomography (CT) scans is a hard and time-consuming task for radiologists. To alleviate this burden, computer-aided diagnosis (CAD) systems have been proposed. In recent years, deep learning approaches have shown impressive results outperforming classical methods in various fields. Nowadays, researchers are trying different deep learning techniques to increase the performance of CAD systems in lung cancer screening with computed tomography. In this work, we review recent state-of-the-art deep learning algorithms and architectures proposed as CAD systems for lung cancer detection. They are divided into two categories—(1) Nodule detection systems, which from the original CT scan detect candidate nodules; and (2) False positive reduction systems, which from a set of given candidate nodules classify them into benign or malignant tumors. The main characteristics of the different techniques are presented, and their performance is analyzed. The CT lung datasets available for research are also introduced. Comparison between the different techniques is presented and discussed.

Evolving the pulmonary nodules diagnosis from classical approaches to deep learning-aided decision support: three decades' development course and future prospect

PURPOSE

Lung cancer is the commonest cause of cancer deaths worldwide, and its mortality can be reduced significantly by performing early diagnosis and screening. Since the 1960s, driven by the pressing needs to accurately and effectively interpret the massive volume of chest images generated daily, computer-assisted diagnosis of pulmonary nodule has opened up new opportunities to relax the limitation from physicians' subjectivity, experiences and fatigue. And the fair access to the reliable and affordable computer-assisted diagnosis will fight the inequalities in incidence and mortality between populations. It has been witnessed that significant and remarkable advances have been achieved since the 1980s, and consistent endeavors have been exerted to deal with the grand challenges on how to accurately detect the pulmonary nodules with high sensitivity at low false-positive rate as well as on how to precisely differentiate between benign and malignant nodules. There is a lack of comprehensive examination of the techniques' development which is evolving the pulmonary nodules diagnosis from classical approaches to machine learning-assisted decision support. The main goal of this investigation is to provide a comprehensive state-of-the-art review of the computer-assisted nodules detection and benign-malignant classification techniques developed over three decades, which have evolved from the complicated ad hoc analysis pipeline of conventional approaches to the simplified seamlessly integrated deep learning techniques. This review also identifies challenges and highlights opportunities for future work in learning models, learning algorithms and enhancement schemes for bridging current state to future prospect and satisfying future demand.

CONCLUSION

It is the first literature review of the past 30 years' development in computer-assisted diagnosis of lung nodules. The challenges indentified and the research opportunities highlighted in this survey are significant for bridging current state to future prospect and satisfying future demand. The values of multifaceted driving forces and multidisciplinary researches are acknowledged that will make the computer-assisted diagnosis of pulmonary nodules enter into the main stream of clinical medicine and raise the state-of-the-art clinical applications as well as increase both welfares of physicians and patients. We firmly hold the vision that fair access to the reliable, faithful, and affordable computer-assisted diagnosis for early cancer diagnosis would fight the inequalities in incidence and mortality between populations, and save more lives.

An adaptive pulmonary nodule detection algorithm

Recently, lung cancer has been paid more and more attention. People have reached a consensus that early detection and early treatment can improve the survival rate of patients. Among them, pulmonary nodules are the important reference for doctors to determine the lung health. With the continuous improvement of CT image resolution, more suspected pulmonary nodule information appears from the impact of chest CT. How to relatively and accurately locate the suspected nodule location from a large number of CT images has brought challenges to the doctor's daily diagnosis. To solve the problem that the original DBSCAN clustering algorithm needs manual setting of the threshold, this paper proposes a region growing algorithm and an adaptive DBSCAN clustering algorithm to improve the accuracy of pulmonary nodule detection. The image is roughly processed and ROI (Regions of Interest) region is roughly extracted by CLAHE transform. The region growing algorithm is used to roughly process the adjacent region's expansibility and the suspected region in ROI, and mark the center point in the region and the boundary point of its point set. The mean value of region range is taken as the threshold value of DBSCAN clustering algorithm. The center of the point domain is used as the starting point of clustering, and the rough set of points is used as the MinPts threshold. Finally, the clustering results are labeled in the initial CT image. Experiments show that the pulmonary nodule detection method proposed in this paper effectively improves the accuracy of the detection results.

A Computer-Aided Detection System for the Detection of Lung Nodules Based on 3D-ResNet

In recent years, the research into automatic aided detection systems for pulmonary nodules has been extremely active. Most of the existing studies are based on 2D convolution neural networks, which cannot make full use of computed tomography’s (CT) 3D spatial information. To address this problem, a computer-aided detection (CAD) system for lung nodules based on a 3D residual network (3D-ResNet) inspired by cognitive science is proposed in this paper. In this system, we feed the slice information extracted from three different axis planes into the U-NET network set, and make the joint decision to generate a candidate nodule set, which is the input of the proposed 3D residual network after extraction. We extracted 3D samples with 40, 44, 48, 52, and 56 mm sides from each candidate nodule in the candidate set and feed them into the trained residual network to get the probability of positive nodule after re-sampling the 3D sample to 48 × 48 × 48 mm 3 . Finally, a joint judgment is made based on the probabilities of five 3D samples of different sizes to obtain the final result. Random rotation and translation and data amplification technology are used to prevent overfitting during network training. The detection intensity on the largest public data set (i.e., the Lung Image Database Consortium and Image Database Resource Initiative—LIDC-IDRI) reached 86.5% and 92.3% at 1 frame per second (FPs) and 4 FPs respectively using our algorithm, which is better than most CAD systems using 2D convolutional neural networks. In addition, a 3D residual network and a multi-section 2D convolution neural network were tested on the unrelated Tianchi dataset. The results indicate that 3D-ResNet has better feature extraction ability than multi-section 2D-ConvNet and is more suitable for reduction of false positive nodules.

Automatic classification of lung nodule candidates based on a novel 3D convolution network and knowledge transferred from a 2D network

OBJECTIVE

In the automatic lung nodule detection system, the authenticity of a large number of nodule candidates needs to be judged, which is a classification task. However, the variable shapes and sizes of the lung nodules have posed a great challenge to the classification of candidates. To solve this problem, we propose a method for classifying nodule candidates through three-dimensional (3D) convolution neural network (ConvNet) model which is trained by transferring knowledge from a multiresolution two-dimensional (2D) ConvNet model.

METHODS

In this scheme, a novel 3D ConvNet model is preweighted with the weights of the trained 2D ConvNet model, and then the 3D ConvNet model is trained with 3D image volumes. In this way, the knowledge transfer method can make 3D network easier to converge and make full use of the spatial information of nodules with different sizes and shapes to improve the classification accuracy.

RESULTS

The experimental results on 551 065 pulmonary nodule candidates in the LUNA16 dataset show that our method gains a competitive average score in the false-positive reduction track in lung nodule detection, with the sensitivities of 0.619 and 0.642 at 0.125 and 0.25 FPs per scan, respectively.

CONCLUSIONS

The proposed method can maintain satisfactory classification accuracy even when the false-positive rate is extremely small in the face of nodules of different sizes and shapes. Moreover, as a transfer learning idea, the method to transfer knowledge from 2D ConvNet to 3D ConvNet is the first attempt to carry out full migration of parameters of various layers including convolution layers, full connection layers, and classifier between different dimensional models, which is more conducive to utilizing the existing 2D ConvNet resources and generalizing transfer learning schemes.

MR-Forest: A Deep Decision Framework for False Positive Reduction in Pulmonary Nodule Detection

With the development of deep learning methods such as convolutional neural network (CNN), the accuracy of automated pulmonary nodule detection has been greatly improved. However, the high computational and storage costs of the large-scale network have been a potential concern for the future widespread clinical application. In this paper, an alternative Multi-ringed (MR)-Forest framework, against the resource-consuming neural networks (NN)-based architectures, has been proposed for false positive reduction in pulmonary nodule detection, which consists of three steps. First, a novel multi-ringed scanning method is used to extract the order ring facets (ORFs) from the surface voxels of the volumetric nodule models; Second, Mesh-LBP and mapping deformation are employed to estimate the texture and shape features. By sliding and resampling the multi-ringed ORFs, feature volumes with different lengths are generated. Finally, the outputs of multi-level are cascaded to predict the candidate class. On 1034 scans merging the dataset from the Affiliated Hospital of Liaoning University of Traditional Chinese Medicine (AH-LUTCM) and the LUNA16 Challenge dataset, our framework performs enough competitiveness than state-of-the-art in false positive reduction task (CPM score of 0.865). Experimental results demonstrate that MR-Forest is a successful solution to satisfy both resource-consuming and effectiveness for automated pulmonary nodule detection. The proposed MR-forest is a general architecture for 3D target detection, it can be easily extended in many other medical imaging analysis tasks, where the growth trend of the targeting object is approximated as a spheroidal expansion.

Multi-Scale Heterogeneous 3D CNN for False-Positive Reduction in Pulmonary Nodule Detection, Based on Chest CT Images

Currently, lung cancer has one of the highest mortality rates because it is often caught too late. Therefore, early detection is essential to reduce the risk of death. Pulmonary nodules are considered key indicators of primary lung cancer. Developing an efficient and accurate computer-aided diagnosis system for pulmonary nodule detection is an important goal. Typically, a computer-aided diagnosis system for pulmonary nodule detection consists of two parts: candidate nodule extraction and false-positive reduction of candidate nodules. The reduction of false positives (FPs) of candidate nodules remains an important challenge due to morphological characteristics of nodule height changes and similar characteristics to other organs. In this study, we propose a novel multi-scale heterogeneous three-dimensional (3D) convolutional neural network (MSH-CNN) based on chest computed tomography (CT) images. There are three main strategies of the design: (1) using multi-scale 3D nodule blocks with different levels of contextual information as inputs; (2) using two different branches of 3D CNN to extract the expression features; (3) using a set of weights which are determined by back propagation to fuse the expression features produced by step 2. In order to test the performance of the algorithm, we trained and tested on the Lung Nodule Analysis 2016 (LUNA16) dataset, achieving an average competitive performance metric (CPM) score of 0.874 and a sensitivity of 91.7% at two FPs/scan. Moreover, our framework is universal and can be easily extended to other candidate false-positive reduction tasks in 3D object detection, as well as 3D object classification.

DeepLNAnno: a Web-Based Lung Nodules Annotating System for CT Images

Lung cancer is one of the most common and fatal types of cancer, and pulmonary nodule detection plays a crucial role in the screening and diagnosis of this disease. A well-trained deep neural network model can help doctors to find nodules on computed tomography(CT) images while requiring lots of labeled data. However, currently available annotating systems are not suitable for annotating pulmonary nodules in CT images. We propose a web-based lung nodules annotating system named as DeepLNAnno. DeepLNAnno has a unique three-tier working process and loads of features like semi-automatic annotation, which not only make it much easier for doctors to annotate compared to some other annotating systems but also increase the accuracy of the labels. We invited a medical group from West China Hospital to annotate the CT images using our DeepLNAnno system, and collected a large number of labeled data. The results of our experiments demonstrated that a usable nodule-detection system is developed, and good benchmark scores on our evaluation data are obtained.

Multi-scale gradual integration CNN for false positive reduction in pulmonary nodule detection

Lung cancer is a global and dangerous disease, and its early detection is crucial for reducing the risks of mortality. In this regard, it has been of great interest in developing a computer-aided system for pulmonary nodules detection as early as possible on thoracic CT scans. In general, a nodule detection system involves two steps: (i) candidate nodule detection at a high sensitivity, which captures many false positives and (ii) false positive reduction from candidates. However, due to the high variation of nodule morphological characteristics and the possibility of mistaking them for neighboring organs, candidate nodule detection remains a challenge. In this study, we propose a novel Multi-scale Gradual Integration Convolutional Neural Network (MGI-CNN), designed with three main strategies: (1) to use multi-scale inputs with different levels of contextual information, (2) to use abstract information inherent in different input scales with gradual integration, and (3) to learn multi-stream feature integration in an end-to-end manner. To verify the efficacy of the proposed network, we conducted exhaustive experiments on the LUNA16 challenge datasets by comparing the performance of the proposed method with state-of-the-art methods in the literature. On two candidate subsets of the LUNA16 dataset, i.e., V1 and V2, our method achieved an average CPM of 0.908 (V1) and 0.942 (V2), outperforming comparable methods by a large margin. Our MGI-CNN is implemented in Python using TensorFlow and the source code is available from https://github.com/ku-milab/MGICNN.

The Added Value of Computer-aided Detection of Small Pulmonary Nodules and Missed Lung Cancers

Lung cancer at its earliest stage is typically manifested on computed tomography as a pulmonary nodule, which could be detected by low-dose multidetector computed tomography technology and the use of thinner collimation. Within the last 2 decades, computer-aided detection (CAD) of pulmonary nodules has been developed to meet the increasing demand for lung cancer screening computed tomography with a larger set of images per scan. This review introduced the basic techniques and then summarized the up-to-date applications of CAD systems in clinical and research programs and in the low-dose lung cancer screening trials, especially in the detection of small pulmonary nodules and missed lung cancers. Many studies have already shown that the CAD systems could increase the sensitivity and reduce the false-positive rate in the diagnosis of pulmonary nodules, especially for the small and isolated nodules. Further improvements to the current CAD schemes are needed to detect nodules accurately, particularly for subsolid nodules.

Improving Accuracy of Lung Nodule Classification Using Deep Learning with Focal Loss

Early detection and classification of pulmonary nodules using computer-aided diagnosis (CAD) systems is useful in reducing mortality rates of lung cancer. In this paper, we propose a new deep learning method to improve classification accuracy of pulmonary nodules in computed tomography (CT) scans. Our method uses a novel 15-layer 2D deep convolutional neural network architecture for automatic feature extraction and classification of pulmonary candidates as nodule or nonnodule. Focal loss function is then applied to the training process to boost classification accuracy of the model. We evaluated our method on the LIDC/IDRI dataset extracted by the LUNA16 challenge. The experiments showed that our deep learning method with focal loss is a high-quality classifier with an accuracy of 97.2%, sensitivity of 96.0%, and specificity of 97.3%.

Pulmonary nodule segmentation with CT sample synthesis using adversarial networks

PURPOSE

Segmentation of pulmonary nodules is critical for the analysis of nodules and lung cancer diagnosis. We present a novel framework of segmentation for various types of nodules using convolutional neural networks (CNNs).

METHODS

The proposed framework is composed of two major parts. The first part is to increase the variety of samples and build a more balanced dataset. A conditional generative adversarial network (cGAN) is employed to produce synthetic CT images. Semantic labels are generated to impart spatial contextual knowledge to the network. Nine attribute scoring labels are combined as well to preserve nodule features. To refine the realism of synthesized samples, reconstruction error loss is introduced into cGAN. The second part is to train a nodule segmentation network on the extended dataset. We build a three-dimensional (3D) CNN model that exploits heterogeneous maps including edge maps and local binary pattern maps. The incorporation of these maps informs the model of texture patterns and boundary information of nodules, which assists high-level feature learning for segmentation. Residual unit, which learns to reduce residual error, is adopted to accelerate training and improve accuracy.

RESULTS

Validation on LIDC-IDRI dataset demonstrates that the generated samples are realistic. The mean squared error and average cosine similarity between real and synthesized samples are 1.55 × 10 - 2 and 0.9534, respectively. The Dice coefficient, positive predicted value, sensitivity, and accuracy are, respectively, 0.8483, 0.8895, 0.8511, and 0.9904 for the segmentation results.

CONCLUSIONS

The proposed 3D CNN segmentation framework, based on the use of synthesized samples and multiple maps with residual learning, achieves more accurate nodule segmentation compared to existing state-of-the-art methods. The proposed CT image synthesis method can not only output samples close to real images but also allow for stochastic variation in image diversity.

Classification of lung nodules in CT scans using three-dimensional deep convolutional neural networks with a checkpoint ensemble method

BACKGROUND

Accurately detecting and examining lung nodules early is key in diagnosing lung cancers and thus one of the best ways to prevent lung cancer deaths. Radiologists spend countless hours detecting small spherical-shaped nodules in computed tomography (CT) images. In addition, even after detecting nodule candidates, a considerable amount of effort and time is required for them to determine whether they are real nodules. The aim of this paper is to introduce a high performance nodule classification method that uses three dimensional deep convolutional neural networks (DCNNs) and an ensemble method to distinguish nodules between non-nodules.

METHODS

In this paper, we use a three dimensional deep convolutional neural network (3D DCNN) with shortcut connections and a 3D DCNN with dense connections for lung nodule classification. The shortcut connections and dense connections successfully alleviate the gradient vanishing problem by allowing the gradient to pass quickly and directly. Connections help deep structured networks to obtain general as well as distinctive features of lung nodules. Moreover, we increased the dimension of DCNNs from two to three to capture 3D features. Compared with shallow 3D CNNs used in previous studies, deep 3D CNNs more effectively capture the features of spherical-shaped nodules. In addition, we use an alternative ensemble method called the checkpoint ensemble method to boost performance.

RESULTS

The performance of our nodule classification method is compared with that of the state-of-the-art methods which were used in the LUng Nodule Analysis 2016 Challenge. Our method achieves higher competition performance metric (CPM) scores than the state-of-the-art methods using deep learning. In the experimental setup ESB-ALL, the 3D DCNN with shortcut connections and the 3D DCNN with dense connections using the checkpoint ensemble method achieved the highest CPM score of 0.910.

CONCLUSION

The result demonstrates that our method of using a 3D DCNN with shortcut connections, a 3D DCNN with dense connections, and the checkpoint ensemble method is effective for capturing 3D features of nodules and distinguishing nodules between non-nodules.

Automatic nodule detection for lung cancer in CT images: A review

Automatic lung nodule detection has great significance for treating lung cancer and increasing patient survival. This work summarizes a critical review of recent techniques for automatic lung nodule detection in computed tomography images. This review indicates the current tendency and obtained progress as well as future challenges in this field. This research covered the databases including Web of Science, PubMed, and the Press, including IEEE Xplore and Science Direct, up to May 2018. Each part of the paper is summarized carefully in terms of the method and validation results for better comparison. Based on the results, some techniques show better performance for lung nodule detection. However, researchers should pay attention to the existing challenges, such as high sensitivity with a low false positive rate, large and different patient databases, developing or optimizing the detection technique of various types of lung nodules with different sizes, shapes, textures and locations, combining electronic medical records and picture archiving and communication systems, building efficient feature sets for better classification and promoting the cooperation and communication between academic institutions and medical organizations. We believe that automatic computer-aided detection systems will be developed with strong robustness, high efficiency and security assurance. This review will be helpful for professional researchers and radiologists to further learn about the latest techniques in computer-aided detection systems.

A Computer-Aided Pipeline for Automatic Lung Cancer Classification on Computed Tomography Scans

Lung cancer is one of the most common cancer types. For the survival of the patient, early detection of lung cancer with the best treatment method is crucial. In this study, we propose a novel computer-aided pipeline on computed tomography (CT) scans for early diagnosis of lung cancer thanks to the classification of benign and malignant nodules. The proposed pipeline is composed of four stages. In preprocessing steps, CT images are enhanced, and lung volumes are extracted from the image with the help of a novel method called lung volume extraction method (LUVEM). The significance of the proposed pipeline is using LUVEM for extracting lung region. In nodule detection stage, candidate nodules are determined according to the circular Hough transform- (CHT-) based method. Then, lung nodules are segmented with self-organizing maps (SOM). In feature computation stage, intensity, shape, texture, energy, and combined features are used for feature extraction, and principal component analysis (PCA) is used for feature reduction step. In the final stage, probabilistic neural network (PNN) classifies benign and malign nodules. According to the experiments performed on our dataset, the proposed pipeline system can classify benign and malign nodules with 95.91% accuracy, 97.42% sensitivity, and 94.24% specificity. Even in cases of small-sized nodules (3-10 mm), the proposed system can determine the nodule type with 94.68% accuracy.

Predicting Malignancy Risk of Screen-Detected Lung Nodules-Mean Diameter or Volume

OBJECTIVE

In lung cancer screening practice low-dose computed tomography, diameter, and volumetric measurement have been used in the management of screen-detected lung nodules. The aim of this study was to compare the performance of nodule malignancy risk prediction tools using diameter or volume and between computer-aided detection (CAD) and radiologist measurements.

METHODS

Multivariable logistic regression models were prepared by using data from two multicenter lung cancer screening trials. For model development and validation, baseline low-dose computed tomography scans from the Pan-Canadian Early Detection of Lung Cancer Study and a subset of National Lung Screening Trial (NLST) scans with lung nodules 3 mm or more in mean diameter were analyzed by using the CIRRUS Lung Screening Workstation (Radboud University Medical Center, Nijmegen, the Netherlands). In the NLST sample, nodules with cancer had been matched on the basis of size to nodules without cancer.

RESULTS

Both CAD-based mean diameter and volume models showed excellent discrimination and calibration, with similar areas under the receiver operating characteristic curves of 0.947. The two CAD models had predictive performance similar to that of the radiologist-based model. In the NLST validation data, the CAD mean diameter and volume models also demonstrated excellent discrimination: areas under the curve of 0.810 and 0.821, respectively. These performance statistics are similar to those of the Pan-Canadian Early Detection of Lung Cancer Study malignancy probability model with use of these data and radiologist-measured maximum diameter.

CONCLUSION

Either CAD-based nodule diameter or volume can be used to assist in predicting a nodule's malignancy risk.

Single-view 2D CNNs with fully automatic non-nodule categorization for false positive reduction in pulmonary nodule detection

BACKGROUND AND OBJECTIVE

In pulmonary nodule detection, the first stage, candidate detection, aims to detect suspicious pulmonary nodules. However, detected candidates include many false positives and thus in the following stage, false positive reduction, such false positives are reliably reduced. Note that this task is challenging due to 1) the imbalance between the numbers of nodules and non-nodules and 2) the intra-class diversity of non-nodules. Although techniques using 3D convolutional neural networks (CNNs) have shown promising performance, they suffer from high computational complexity which hinders constructing deep networks. To efficiently address these problems, we propose a novel framework using the ensemble of 2D CNNs using single views, which outperforms existing 3D CNN-based methods.

METHODS

Our ensemble of 2D CNNs utilizes single-view 2D patches to improve both computational and memory efficiency compared to previous techniques exploiting 3D CNNs. We first categorize non-nodules on the basis of features encoded by an autoencoder. Then, all 2D CNNs are trained by using the same nodule samples, but with different types of non-nodules. By extending the learning capability, this training scheme resolves difficulties of extracting representative features from non-nodules with large appearance variations. Note that, instead of manual categorization requiring the heavy workload of radiologists, we propose to automatically categorize non-nodules based on the autoencoder and k-means clustering.

RESULTS

We performed extensive experiments to validate the effectiveness of our framework based on the database of the lung nodule analysis 2016 challenge. The superiority of our framework is demonstrated through comparing the performance of five frameworks trained with differently constructed training sets. Our proposed framework achieved state-of-the-art performance (0.922 of the competition performance metric score) with low computational demands (789K of parameters and 1024M of floating point operations per second).

CONCLUSION

We presented a novel false positive reduction framework, the ensemble of single-view 2D CNNs with fully automatic non-nodule categorization, for pulmonary nodule detection. Unlike previous 3D CNN-based frameworks, we utilized 2D CNNs using 2D single views to improve computational efficiency. Also, our training scheme using categorized non-nodules, extends the learning capability of representative features of different non-nodules. Our framework achieved state-of-the-art performance with low computational complexity.

Long-Term Active Surveillance of Screening Detected Subsolid Nodules is a Safe Strategy to Reduce Overtreatment

INTRODUCTION

Lung cancer presenting as subsolid nodule (SSN) can show slow growth, hence treating SSN is controversial. Our aim was to determine the long-term outcome of subjects with unresected SSNs in lung cancer screening.

METHODS

Since 2005, the Multicenter Italian Lung Detection (MILD) screening trial implemented active surveillance for persistent SSN, as opposed to early resection. Presence of SSNs was related to diagnosis of cancer at the site of SSN, elsewhere in the lung, or in the body. The risk of overall mortality and lung cancer mortality was tested by Cox proportional hazards model.

RESULTS

SSNs were found in 16.9% (389 of 2303) of screenees. During 9.3 ± 1.2 years of follow-up, the hazard ratio of lung cancer diagnosis in subjects with SSN was 6.77 (95% confidence interval: 3.39-13.54), with 73% (22 of 30) of cancers not arising from SSN (median time to diagnosis 52 months from SSN). Lung cancer-specific mortality in subjects with SSN was significantly increased (hazard ratio = 3.80; 95% confidence interval: 1.24-11.65) compared to subjects without lung nodules. Lung cancer arising from SSN did not lead to death within the follow-up period.

CONCLUSIONS

Subjects with SSN in the MILD cohort showed a high risk of developing lung cancer elsewhere in the lung, with only a minority of cases arising from SSN, and never representing the cause of death. These results show the safety of active surveillance for conservative management of SSN until signs of solid component growth and the need for prolonged follow-up because of high risk of other cancers.

CNN models discriminating between pulmonary micro-nodules and non-nodules from CT images

Background

Early and automatic detection of pulmonary nodules from CT lung screening is the prerequisite for precise management of lung cancer. However, a large number of false positives appear in order to increase the sensitivity, especially for detecting micro-nodules (diameter < 3 mm), which increases the radiologists’ workload and causes unnecessary anxiety for the patients. To decrease the false positive rate, we propose to use CNN models to discriminate between pulmonary micro-nodules and non-nodules from CT image patches.

Methods

A total of 13,179 micro-nodules and 21,315 non-nodules marked by radiologists are extracted with three different patch sizes (16 × 16, 32 × 32 and 64 × 64) from LIDC/IDRI database and used in the experiments. Three CNN models with different depths (1, 2 or 4 convolutional layers) are designed; their performances are evaluated by the fivefold cross-validation in term of the accuracy, area under the curve (AUC), F-score and sensitivity. The network parameters are also optimized.

Results

It is found that the performance of the CNN models is greatly dependent on the patches size and the number of convolutional layers. The CNN model with two convolutional layers presented the best performance in case of 32 × 32 patches size, achieving an accuracy of 88.28%, an AUC of 0.87, a F-score of 83.45% and a sensitivity of 83.82%.

Conclusions

The CNN models with appropriate depth and size of image patches can effectively discriminate between pulmonary micro-nodules and non-nodules, and reduce the false positives and help manage lung cancer precisely.

Evolutionary image simplification for lung nodule classification with convolutional neural networks

PURPOSE

Understanding decisions of deep learning techniques is important. Especially in the medical field, the reasons for a decision in a classification task are as crucial as the pure classification results. In this article, we propose a new approach to compute relevant parts of a medical image. Knowing the relevant parts makes it easier to understand decisions.

METHODS

In our approach, a convolutional neural network is employed to learn structures of images of lung nodules. Then, an evolutionary algorithm is applied to compute a simplified version of an unknown image based on the learned structures by the convolutional neural network. In the simplified version, irrelevant parts are removed from the original image.

RESULTS

In the results, we show simplified images which allow the observer to focus on the relevant parts. In these images, more than 50% of the pixels are simplified. The simplified pixels do not change the meaning of the images based on the learned structures by the convolutional neural network. An experimental analysis shows the potential of the approach. Besides the examples of simplified images, we analyze the run time development.

CONCLUSIONS

Simplified images make it easier to focus on relevant parts and to find reasons for a decision. The combination of an evolutionary algorithm employing a learned convolutional neural network is well suited for the simplification task. From a research perspective, it is interesting which areas of the images are simplified and which parts are taken as relevant.

A deep 3D residual CNN for false-positive reduction in pulmonary nodule detection

PURPOSE

The automatic detection of pulmonary nodules using CT scans improves the efficiency of lung cancer diagnosis, and false-positive reduction plays a significant role in the detection. In this paper, we focus on the false-positive reduction task and propose an effective method for this task.

METHODS

We construct a deep 3D residual CNN (convolution neural network) to reduce false-positive nodules from candidate nodules. The proposed network is much deeper than the traditional 3D CNNs used in medical image processing. Specifically, in the network, we design a spatial pooling and cropping (SPC) layer to extract multilevel contextual information of CT data. Moreover, we employ an online hard sample selection strategy in the training process to make the network better fit hard samples (e.g., nodules with irregular shapes).

RESULTS

Our method is evaluated on 888 CT scans from the dataset of the LUNA16 Challenge. The free-response receiver operating characteristic (FROC) curve shows that the proposed method achieves a high detection performance.

CONCLUSIONS

Our experiments confirm that our method is robust and that the SPC layer helps increase the prediction accuracy. Additionally, the proposed method can easily be extended to other 3D object detection tasks in medical image processing.

Pulmonary quantitative CT imaging in focal and diffuse disease: current research and clinical applications

The frenetic development of imaging technology-both hardware and software-provides exceptional potential for investigation of the lung. In the last two decades, CT was exploited for detailed characterization of pulmonary structures and description of respiratory disease. The introduction of volumetric acquisition allowed increasingly sophisticated analysis of CT data by means of computerized algorithm, namely quantitative CT (QCT). Hundreds of thousands of CTs have been analysed for characterization of focal and diffuse disease of the lung. Several QCT metrics were developed and tested against clinical, functional and prognostic descriptors. Computer-aided detection of nodules, textural analysis of focal lesions, densitometric analysis and airway segmentation in obstructive pulmonary disease and textural analysis in interstitial lung disease are the major chapters of this discipline. The validation of QCT metrics for specific clinical and investigational needs prompted the translation of such metrics from research field to patient care. The present review summarizes the state of the art of QCT in both focal and diffuse lung disease, including a dedicated discussion about application of QCT metrics as parameters for clinical care and outcomes in clinical trials.

Automatic segmentation of the solid core and enclosed vessels in subsolid pulmonary nodules

Subsolid pulmonary nodules are commonly encountered in lung cancer screening and clinical routine. Compared to other nodule types, subsolid nodules are associated with a higher malignancy probability for which the size and mass of the nodule and solid core are important indicators. However, reliably measuring these characteristics on computed tomography (CT) can be hampered by the presence of vessels encompassed by the nodule, since vessels have similar CT attenuation as solid cores. This can affect treatment decisions and patient management. We present a method based on voxel classification to automatically identify vessels and solid cores in given subsolid nodules on CT. Three experts validated our method on 170 screen-detected subsolid nodules from the Multicentric Italian Lung Disease trial. The agreement between the proposed method and the observers was substantial for vessel detection and moderate for solid core detection, which was similar to the inter-observer agreement. We found a relatively high variability in the inter-observer agreement and low method-observer agreements for delineating the borders of vessels and solid cores, illustrating the difficulty of this task. However, 92.4% of the proposed vessel and 80.6% of the proposed solid core segmentations were labeled as usable in clinical practice by the majority of experts.