Machine Learning in Diagnosis and Prognosis of Lung Cancer by PET-CT

As a disease with high morbidity and high mortality, lung cancer has seriously harmed people's health. Therefore, early diagnosis and treatment are more important. PET/CT is usually used to obtain the early diagnosis, staging, and curative effect evaluation of tumors, especially lung cancer, due to the heterogeneity of tumors and the differences in artificial image interpretation and other reasons, it also fails to entirely reflect the real situation of tumors. Artificial intelligence (AI) has been applied to all aspects of life. Machine learning (ML) is one of the important ways to realize AI. With the help of the ML method used by PET/CT imaging technology, there are many studies in the diagnosis and treatment of lung cancer. This article summarizes the application progress of ML based on PET/CT in lung cancer, in order to better serve the clinical. In this study, we searched PubMed using machine learning, lung cancer, and PET/CT as keywords to find relevant articles in the past 5 years or more. We found that PET/CT-based ML approaches have achieved significant results in the detection, delineation, classification of pathology, molecular subtyping, staging, and response assessment with survival and prognosis of lung cancer, which can provide clinicians a powerful tool to support and assist in critical daily clinical decisions. However, ML has some shortcomings such as slightly poor repeatability and reliability.

Information fusion for fully automated segmentation of head and neck tumors from PET and CT images

BACKGROUND

PET/CT images combining anatomic and metabolic data provide complementary information that can improve clinical task performance. PET image segmentation algorithms exploiting the multi-modal information available are still lacking.

PURPOSE

Our study aimed to assess the performance of PET and CT image fusion for gross tumor volume (GTV) segmentations of head and neck cancers (HNCs) utilizing conventional, deep learning (DL), and output-level voting-based fusions.

METHODS

The current study is based on a total of 328 histologically confirmed HNCs from six different centers. The images were automatically cropped to a 200 × 200 head and neck region box, and CT and PET images were normalized for further processing. Eighteen conventional image-level fusions were implemented. In addition, a modified U2-Net architecture as DL fusion model baseline was used. Three different input, layer, and decision-level information fusions were used. Simultaneous truth and performance level estimation (STAPLE) and majority voting to merge different segmentation outputs (from PET and image-level and network-level fusions), that is, output-level information fusion (voting-based fusions) were employed. Different networks were trained in a 2D manner with a batch size of 64. Twenty percent of the dataset with stratification concerning the centers (20% in each center) were used for final result reporting. Different standard segmentation metrics and conventional PET metrics, such as SUV, were calculated.

RESULTS

In single modalities, PET had a reasonable performance with a Dice score of 0.77 ± 0.09, while CT did not perform acceptably and reached a Dice score of only 0.38 ± 0.22. Conventional fusion algorithms obtained a Dice score range of [0.76-0.81] with guided-filter-based context enhancement (GFCE) at the low-end, and anisotropic diffusion and Karhunen-Loeve transform fusion (ADF), multi-resolution singular value decomposition (MSVD), and multi-level image decomposition based on latent low-rank representation (MDLatLRR) at the high-end. All DL fusion models achieved Dice scores of 0.80. Output-level voting-based models outperformed all other models, achieving superior results with a Dice score of 0.84 for Majority_ImgFus, Majority_All, and Majority_Fast. A mean error of almost zero was achieved for all fusions using SUVpeak , SUVmean and SUVmedian .

CONCLUSION

PET/CT information fusion adds significant value to segmentation tasks, considerably outperforming PET-only and CT-only methods. In addition, both conventional image-level and DL fusions achieve competitive results. Meanwhile, output-level voting-based fusion using majority voting of several algorithms results in statistically significant improvements in the segmentation of HNC.

Efficacy of exponentiation method with a convolutional neural network for classifying lung nodules on CT images by malignancy level

OBJECTIVES

The aim of this study was to examine the performance of a convolutional neural network (CNN) combined with exponentiating each pixel value in classifying benign and malignant lung nodules on computed tomography (CT) images.

MATERIALS AND METHODS

Images in the Lung Image Database Consortium-Image Database Resource Initiative (LIDC-IDRI) were analyzed. Four CNN models were then constructed to classify the lung nodules by malignancy level (malignancy level 1 vs. 2, malignancy level 1 vs. 3, malignancy level 1 vs. 4, and malignancy level 1 vs. 5). The exponentiation method was applied for exponent values of 1.0 to 10.0 in increments of 0.5. Accuracy, sensitivity, specificity, and area under the curve of receiver operating characteristics (AUC-ROC) were calculated. These statistics were compared between an exponent value of 1.0 and all other exponent values in each model by the Mann-Whitney U-test.

RESULTS

In malignancy 1 vs. 4, maximum test accuracy (MTA; exponent value = 2.0, 3.0, 3.5, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5, and 10.0) and specificity (6.5, 7.0, and 9.0) were improved by up to 0.012 and 0.037, respectively. In malignancy 1 vs. 5, MTA (6.5 and 7.0) and sensitivity (1.5) were improved by up to 0.030 and 0.0040, respectively.

CONCLUSIONS

The exponentiation method improved the performance of the CNN in the task of classifying lung nodules on CT images as benign or malignant. The exponentiation method demonstrated two advantages: improved accuracy, and the ability to adjust sensitivity and specificity by selecting an appropriate exponent value.

CLINICAL RELEVANCE STATEMENT

Adjustment of sensitivity and specificity by selecting an exponent value enables the construction of proper CNN models for screening, diagnosis, and treatment processes among patients with lung nodules.

KEY POINTS

• The exponentiation method improved the performance of the convolutional neural network. • Contrast accentuation by the exponentiation method may derive features of lung nodules. • Sensitivity and specificity can be adjusted by selecting an exponent value.

Residual neural network with mixed loss based on batch training technique for identification of EGFR mutation status in lung cancer

Epidermal growth factor receptor (EGFR) is the key to targeted therapy with tyrosine kinase inhibitors in lung cancer. Traditional identification of EGFR mutation status requires biopsy and sequence testing, which may not be suitable for certain groups who cannot perform biopsy. In this paper, using easily accessible and non-invasive CT images, the residual neural network (ResNet) with mixed loss based on batch training technique is proposed for identification of EGFR mutation status in lung cancer. In this model, the ResNet is regarded as the baseline for feature extraction to avoid the gradient disappearance. Besides, a new mixed loss based on the batch similarity and the cross entropy is proposed to guide the network to better learn the model parameters. The proposed mixed loss utilizes the similarity among batch samples to evaluate the distribution of training data, which can reduce the similarity of different classes and the difference of the same classes. In the experiments, VGG16Net, DenseNet, ResNet18, ResNet34 and ResNet50 models with the mixed loss are trained on the public CT dataset with 155 patients including EGFR mutation status from TCIA. The trained networks are employed to the collected preoperative CT dataset with 56 patients from the cooperative hospital for validating the efficiency of the proposed models. Experimental results show that the proposed models are more appropriate and effective on the lung cancer dataset for identifying the EGFR mutation status. In these models, the ResNet34 with mixed loss is optimal (accuracy = 81.58%, AUC = 0.8861, sensitivity = 80.02%, specificity = 82.90%).

Multi-scale dense selective network based on border modeling for lung nodule segmentation

PURPOSE

Accurate quantification of pulmonary nodules helps physicians to accurately diagnose and treat lung cancer. We try to improve the segmentation efficiency of irregular nodules while maintaining the segmentation accuracy of simple types of nodules.

METHODS

In this paper, we obtain the unique edge part of pulmonary nodules and process it as a single branch stream, i.e., border stream, to explicitly model the nodule edge information. We propose a multi-scale dense selective network based on border modeling (BorDenNet). Its overall framework consists of a dual-branch encoder-decoder, which achieves parallel processing of classical image stream and border stream. We design a dense attention module to facilitate a strongly coupled status of feature images to focus on key regions of pulmonary nodules. Then, during the process of model decoding, the multi-scale selective attention module is proposed to establish long-range correlation relationships between different scale features, which further achieves finer feature discrimination and spatial recovery. We introduce border context enhancement module to mutually fuse and enhance the edge-related voxel features contained in the image stream and border stream and finally achieve the accurate segmentation of pulmonary nodules.

RESULTS

We evaluate the BorDenNet rigorously on the lung public dataset LIDC-IDRI. For the segmentation of the target nodules, the average Dice score is 92.78[Formula: see text], the average sensitivity is 91.37[Formula: see text], and the average Hausdorff distance is 3.06 mm. We further test on a private dataset from Shanxi Provincial People's Hospital, which verifies the excellent generalization of BorDenNet. Our BorDenNet relatively improves the segmentation efficiency for multi-type nodules such as adherent pulmonary nodules and ground-glass pulmonary nodules.

CONCLUSION

Accurate segmentation of irregular pulmonary nodules can obtain important clinical parameters, which can be used as a guide for clinicians and improve clinical efficiency.

Automated lung cancer assessment on 18F-PET/CT using Retina U-Net and anatomical region segmentation

OBJECTIVES

To develop and test a Retina U-Net algorithm for the detection of primary lung tumors and associated metastases of all stages on FDG-PET/CT.

METHODS

A data set consisting of 364 FDG-PET/CTs of patients with histologically confirmed lung cancer was used for algorithm development and internal testing. The data set comprised tumors of all stages. All lung tumors (T), lymphatic metastases (N), and distant metastases (M) were manually segmented as 3D volumes using whole-body PET/CT series. The data set was split into a training (n = 216), validation (n = 74), and internal test data set (n = 74). Detection performance for all lesion types at multiple classifier thresholds was evaluated and false-positive-findings-per-case (FP/c) calculated. Next, detected lesions were assigned to categories T, N, or M using an automated anatomical region segmentation. Furthermore, reasons for FPs were visually assessed and analyzed. Finally, performance was tested on 20 PET/CTs from another institution.

RESULTS

Sensitivity for T lesions was 86.2% (95% CI: 77.2-92.7) at a FP/c of 2.0 on the internal test set. The anatomical correlate to most FPs was the physiological activity of bone marrow (16.8%). TNM categorization based on the anatomical region approach was correct in 94.3% of lesions. Performance on the external test set confirmed the good performance of the algorithm (overall detection rate = 88.8% (95% CI: 82.5-93.5%) and FP/c = 2.7).

CONCLUSIONS

Retina U-Nets are a valuable tool for tumor detection tasks on PET/CT and can form the backbone of reading assistance tools in this field. FPs have anatomical correlates that can lead the way to further algorithm improvements. The code is publicly available.

KEY POINTS

• Detection of malignant lesions in PET/CT with Retina U-Net is feasible. • All false-positive findings had anatomical correlates, physiological bone marrow activity being the most prevalent. • Retina U-Nets can build the backbone for tools assisting imaging professionals in lung tumor staging.

The role of artificial intelligence based on PET/CT radiomics in NSCLC: Disease management, opportunities, and challenges

Objectives

Lung cancer has been widely characterized through radiomics and artificial intelligence (AI). This review aims to summarize the published studies of AI based on positron emission tomography/computed tomography (PET/CT) radiomics in non-small-cell lung cancer (NSCLC).

Materials and methods

A comprehensive search of literature published between 2012 and 2022 was conducted on the PubMed database. There were no language or publication status restrictions on the search. About 127 articles in the search results were screened and gradually excluded according to the exclusion criteria. Finally, this review included 39 articles for analysis.

Results

Classification is conducted according to purposes and several studies were identified at each stage of disease:1) Cancer detection (n=8), 2) histology and stage of cancer (n=11), 3) metastases (n=6), 4) genotype (n=6), 5) treatment outcome and survival (n=8). There is a wide range of heterogeneity among studies due to differences in patient sources, evaluation criteria and workflow of radiomics. On the whole, most models show diagnostic performance comparable to or even better than experts, and the common problems are repeatability and clinical transformability.

Conclusion

AI-based PET/CT Radiomics play potential roles in NSCLC clinical management. However, there is still a long way to go before being translated into clinical application. Large-scale, multi-center, prospective research is the direction of future efforts, while we need to face the risk of repeatability of radiomics features and the limitation of access to large databases.

MS-ResNet: disease-specific survival prediction using longitudinal CT images and clinical data

PURPOSE

Medical imaging data of lung cancer in different stages contain a large amount of time information related to its evolution (emergence, development, or extinction). We try to explore the evolution process of lung images in time dimension to improve the prediction of lung cancer survival by using longitudinal CT images and clinical data jointly.

METHODS

In this paper, we propose an innovative multi-branch spatiotemporal residual network (MS-ResNet) for disease-specific survival (DSS) prediction by integrating the longitudinal computed tomography (CT) images at different times and clinical data. Specifically, we first extract the deep features from the multi-period CT images by an improved residual network. Then, the feature selection algorithm is used to select the most relevant feature subset from the clinical data. Finally, we integrate the deep features and feature subsets to take full advantage of the complementarity between the two types of data to generate the final prediction results.

RESULTS

The experimental results demonstrate that our MS-ResNet model is superior to other methods, achieving a promising 86.78% accuracy in the classification of short-survivor, med-survivor, and long-survivor.

CONCLUSION

In computer-aided prognostic analysis of cancer, the time dimension features of the course of disease and the integration of patient clinical data and CT data can effectively improve the prediction accuracy.

A few-shot U-Net deep learning model for lung cancer lesion segmentation via PET/CT imaging

Over the past few years, positron emission tomography/computed tomography (PET/CT) imaging for computer-aided diagnosis has received increasing attention. Supervised deep learning architectures are usually employed for the detection of abnormalities, with anatomical localization, especially in the case of CT scans. However, the main limitations of the supervised learning paradigm include (i) large amounts of data required for model training, and (ii) the assumption of fixed network weights upon training completion, implying that the performance of the model cannot be further improved after training. In order to overcome these limitations, we apply a few-shot learning (FSL) scheme. Contrary to traditional deep learning practices, in FSL the model is provided with less data during training. The model then utilizes end-user feedback after training to constantly improve its performance. We integrate FSL in a U-Net architecture for lung cancer lesion segmentation on PET/CT scans, allowing for dynamic model weight fine-tuning and resulting in an online supervised learning scheme. Constant online readjustments of the model weights according to the user's feedback, increase the detection and classification accuracy, especially in cases where low detection performance is encountered. Our proposed method is validated on the Lung-PET-CT-DX TCIA database. PET/CT scans from 87 patients were included in the dataset and were acquired 60 minutes after intravenous¹⁸F-FDG injection. Experimental results indicate the superiority of our approach compared to other state of the art methods.

Multi-Modal Co-Learning for Liver Lesion Segmentation on PET-CT Images

Liver lesion segmentation is an essential process to assist doctors in hepatocellular carcinoma diagnosis and treatment planning. Multi-modal positron emission tomography and computed tomography (PET-CT) scans are widely utilized due to their complementary feature information for this purpose. However, current methods ignore the interaction of information across the two modalities during feature extraction, omit the co-learning of the feature maps of different resolutions, and do not ensure that shallow and deep features complement each others sufficiently. In this paper, our proposed model can achieve feature interaction across multi-modal channels by sharing the down-sampling blocks between two encoding branches to eliminate misleading features. Furthermore, we combine feature maps of different resolutions to derive spatially varying fusion maps and enhance the lesions information. In addition, we introduce a similarity loss function for consistency constraint in case that predictions of separated refactoring branches for the same regions vary a lot. We evaluate our model for liver tumor segmentation using a PET-CT scans dataset, compare our method with the baseline techniques for multi-modal (multi-branches, multi-channels and cascaded networks) and then demonstrate that our method has a significantly higher accuracy ( ) than the baseline models.

Recurrent feature fusion learning for multi-modality pet-ct tumor segmentation

BACKGROUND AND OBJECTIVE

[18f]-fluorodeoxyglucose (fdg) positron emission tomography - computed tomography (pet-ct) is now the preferred imaging modality for staging many cancers. Pet images characterize tumoral glucose metabolism while ct depicts the complementary anatomical localization of the tumor. Automatic tumor segmentation is an important step in image analysis in computer aided diagnosis systems. Recently, fully convolutional networks (fcns), with their ability to leverage annotated datasets and extract image feature representations, have become the state-of-the-art in tumor segmentation. There are limited fcn based methods that support multi-modality images and current methods have primarily focused on the fusion of multi-modality image features at various stages, i.e., early-fusion where the multi-modality image features are fused prior to fcn, late-fusion with the resultant features fused and hyper-fusion where multi-modality image features are fused across multiple image feature scales. Early- and late-fusion methods, however, have inherent, limited freedom to fuse complementary multi-modality image features. The hyper-fusion methods learn different image features across different image feature scales that can result in inaccurate segmentations, in particular, in situations where the tumors have heterogeneous textures.

METHODS

we propose a recurrent fusion network (rfn), which consists of multiple recurrent fusion phases to progressively fuse the complementary multi-modality image features with intermediary segmentation results derived at individual recurrent fusion phases: (1) the recurrent fusion phases iteratively learn the image features and then refine the subsequent segmentation results; and, (2) the intermediary segmentation results allows our method to focus on learning the multi-modality image features around these intermediary segmentation results, which minimize the risk of inconsistent feature learning.

RESULTS

we evaluated our method on two pathologically proven non-small cell lung cancer pet-ct datasets. We compared our method to the commonly used fusion methods (early-fusion, late-fusion and hyper-fusion) and the state-of-the-art pet-ct tumor segmentation methods on various network backbones (resnet, densenet and 3d-unet). Our results show that the rfn provides more accurate segmentation compared to the existing methods and is generalizable to different datasets.

CONCLUSIONS

we show that learning through multiple recurrent fusion phases allows the iterative re-use of multi-modality image features that refines tumor segmentation results. We also identify that our rfn produces consistent segmentation results across different network architectures.

Global evolution of research on pulmonary nodules: a bibliometric analysis

Aim: To provide a historical and global picture of research concerning lung nodules, compare the contributions of major countries and explore research trends over the past 10 years. Methods: A bibliometric analysis of publications from Scopus (1970-2020) and Web of Science (2011-2020). Results: Publications about pulmonary nodules showed an enormous growth trend from 1970 to 2020. There is a high level of collaboration among the 20 most productive countries and regions, with the USA located at the center of the collaboration network. The keywords 'deep learning', 'artificial intelligence' and 'machine learning' are current hotspots. Conclusions: Abundant research has focused on pulmonary nodules. Deep learning is emerging as a promising tool for lung cancer diagnosis and management.

Medical image analysis based on deep learning approach

Medical imaging plays a significant role in different clinical applications such as medical procedures used for early detection, monitoring, diagnosis, and treatment evaluation of various medical conditions. Basicsof the principles and implementations of artificial neural networks and deep learning are essential for understanding medical image analysis in computer vision. Deep Learning Approach (DLA) in medical image analysis emerges as a fast-growing research field. DLA has been widely used in medical imaging to detect the presence or absence of the disease. This paper presents the development of artificial neural networks, comprehensive analysis of DLA, which delivers promising medical imaging applications. Most of the DLA implementations concentrate on the X-ray images, computerized tomography, mammography images, and digital histopathology images. It provides a systematic review of the articles for classification, detection, and segmentation of medical images based on DLA. This review guides the researchers to think of appropriate changes in medical image analysis based on DLA.

A Systematic Review of PET Textural Analysis and Radiomics in Cancer

Background: Although many works have supported the utility of PET radiomics, several authors have raised concerns over the robustness and replicability of the results. This study aimed to perform a systematic review on the topic of PET radiomics and the used methodologies. Methods: PubMed was searched up to 15 October 2020. Original research articles based on human data specifying at least one tumor type and PET image were included, excluding those that apply only first-order statistics and those including fewer than 20 patients. Each publication, cancer type, objective and several methodological parameters (number of patients and features, validation approach, among other things) were extracted. Results: A total of 290 studies were included. Lung (28%) and head and neck (24%) were the most studied cancers. The most common objective was prognosis/treatment response (46%), followed by diagnosis/staging (21%), tumor characterization (18%) and technical evaluations (15%). The average number of patients included was 114 (median = 71; range 20–1419), and the average number of high-order features calculated per study was 31 (median = 26, range 1–286). Conclusions: PET radiomics is a promising field, but the number of patients in most publications is insufficient, and very few papers perform in-depth validations. The role of standardization initiatives will be crucial in the upcoming years.

Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations

Lung cancer is pointed as a leading cause of cancer death worldwide. Early lung nodule diagnosis has great significance for treating lung cancer and increasing patient survival. In this paper, we present a novel method to classify the malignant from benign lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations (SE-ResNeXt). The state-of-the-art SE-ResNeXt module, which integrates the advantages of SENet for feature recalibration and ResNeXt for feature reuse, has great ability in boosting feature discriminability on imaging pattern recognition. The method is evaluated on the public available LUng Nodule Analysis 2016 (LUNA16) database with 1004 (450 malignant and 554 benign) nodules, achieving an area under the receiver operating characteristic curve (AUC) of 0. 9563 and accuracy of 91.67%. The promising results demonstrate that our method has strong robustness in the classification of nodules. The method has the potential to help radiologists better interpret diagnostic data and differentiate the benign from malignant lung nodules on CT images in clinical practice. To our best knowledge, the effectiveness of SE-ResNeXt on lung nodule classification has not been extensively explored.

The Performance of Deep Learning Algorithms on Automatic Pulmonary Nodule Detection and Classification Tested on Different Datasets That Are Not Derived from LIDC-IDRI: A Systematic Review

The aim of this study was to systematically review the performance of deep learning technology in detecting and classifying pulmonary nodules on computed tomography (CT) scans that were not from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database. Furthermore, we explored the difference in performance when the deep learning technology was applied to test datasets different from the training datasets. Only peer-reviewed, original research articles utilizing deep learning technology were included in this study, and only results from testing on datasets other than the LIDC-IDRI were included. We searched a total of six databases: EMBASE, PubMed, Cochrane Library, the Institute of Electrical and Electronics Engineers, Inc. (IEEE), Scopus, and Web of Science. This resulted in 1782 studies after duplicates were removed, and a total of 26 studies were included in this systematic review. Three studies explored the performance of pulmonary nodule detection only, 16 studies explored the performance of pulmonary nodule classification only, and 7 studies had reports of both pulmonary nodule detection and classification. Three different deep learning architectures were mentioned amongst the included studies: convolutional neural network (CNN), massive training artificial neural network (MTANN), and deep stacked denoising autoencoder extreme learning machine (SDAE-ELM). The studies reached a classification accuracy between 68–99.6% and a detection accuracy between 80.6–94%. Performance of deep learning technology in studies using different test and training datasets was comparable to studies using same type of test and training datasets. In conclusion, deep learning was able to achieve high levels of accuracy, sensitivity, and/or specificity in detecting and/or classifying nodules when applied to pulmonary CT scans not from the LIDC-IDRI database.

Automated Lung Nodule Detection and Classification Using Deep Learning Combined with Multiple Strategies

Lung cancer is one of the major causes of cancer-related deaths due to its aggressive nature and delayed detections at advanced stages. Early detection of lung cancer is very important for the survival of an individual, and is a significant challenging problem. Generally, chest radiographs (X-ray) and computed tomography (CT) scans are used initially for the diagnosis of the malignant nodules; however, the possible existence of benign nodules leads to erroneous decisions. At early stages, the benign and the malignant nodules show very close resemblance to each other. In this paper, a novel deep learning-based model with multiple strategies is proposed for the precise diagnosis of the malignant nodules. Due to the recent achievements of deep convolutional neural networks (CNN) in image analysis, we have used two deep three-dimensional (3D) customized mixed link network (CMixNet) architectures for lung nodule detection and classification, respectively. Nodule detections were performed through faster R-CNN on efficiently-learned features from CMixNet and U-Net like encoder-decoder architecture. Classification of the nodules was performed through a gradient boosting machine (GBM) on the learned features from the designed 3D CMixNet structure. To reduce false positives and misdiagnosis results due to different types of errors, the final decision was performed in connection with physiological symptoms and clinical biomarkers. With the advent of the internet of things (IoT) and electro-medical technology, wireless body area networks (WBANs) provide continuous monitoring of patients, which helps in diagnosis of chronic diseases-especially metastatic cancers. The deep learning model for nodules' detection and classification, combined with clinical factors, helps in the reduction of misdiagnosis and false positive (FP) results in early-stage lung cancer diagnosis. The proposed system was evaluated on LIDC-IDRI datasets in the form of sensitivity (94%) and specificity (91%), and better results were obatined compared to the existing methods.

A CAD system for pulmonary nodule prediction based on deep three-dimensional convolutional neural networks and ensemble learning

Background

Detection of pulmonary nodules is an important aspect of an automatic detection system. Incomputer-aided diagnosis (CAD) systems, the ability to detect pulmonary nodules is highly important, which plays an important role in the diagnosis and early treatment of lung cancer. Currently, the detection of pulmonary nodules depends mainly on doctor experience, which varies. This paper aims to address the challenge of pulmonary nodule detection more effectively.

Methods

A method for detecting pulmonary nodules based on an improved neural network is presented in this paper. Nodules are clusters of tissue with a diameter of 3 mm to 30 mm in the pulmonary parenchyma. Because pulmonary nodules are similar to other lung structures and have a low density, false positive nodules often occur. Thus, our team proposed an improved convolutional neural network (CNN) framework to detect nodules. First, a nonsharpening mask is used to enhance the nodules in computed tomography (CT) images; then, CT images of 512×512 pixels are segmented into smaller images of 96×96 pixels. Second, in the 96×96 pixel images which contain or exclude pulmonary nodules, the plaques corresponding to positive and negative samples are segmented. Third, CT images segmented into 96×96 pixels are down-sampled to 64×64 and 32×32 size respectively. Fourth, an improved fusion neural network structure is constructed that consists of three three-dimensional convolutional neural networks, designated as CNN-1, CNN-2, and CNN-3, to detect false positive pulmonary nodules. The networks’ input sizes are 32×32×32, 64×64×64, and 96×96×96 and include 5, 7, and 9 layers, respectively. Finally, we use the AdaBoost classifier to fuse the results of CNN-1, CNN-2, and CNN-3. We call this new neural network framework the Amalgamated-Convolutional Neural Network (A-CNN) and use it to detect pulmonary nodules.

Findings

Our team trained A-CNN using the LUNA16 and Ali Tianchi datasets and evaluated its performance using the LUNA16 dataset. We discarded nodules less than 5mm in diameter. When the average number of false positives per scan was 0.125 and 0.25, the sensitivity of A-CNN reached as high as 81.7% and 85.1%, respectively.

Evaluation of an AI-Powered Lung Nodule Algorithm for Detection and 3D Segmentation of Primary Lung Tumors

Automated detection and segmentation is a prerequisite for the deployment of image-based secondary analyses, especially for lung tumors. However, currently only applications for lung nodules ≤3 cm exist. Therefore, we tested the performance of a fully automated AI-based lung nodule algorithm for detection and 3D segmentation of primary lung tumors in the context of tumor staging using the CT component of FDG-PET/CT and including all T-categories (T1-T4). FDG-PET/CTs of 320 patients with histologically confirmed lung cancer performed between 01/2010 and 06/2016 were selected. First, the main primary lung tumor within each scan was manually segmented using the CT component of the PET/CTs as reference. Second, the CT series were transferred to a platform with AI-based algorithms trained on chest CTs for detection and segmentation of lung nodules. Detection and segmentation performance were analyzed. Factors influencing detection rates were explored with binominal logistic regression and radiomic analysis. We also processed 94 PET/CTs negative for pulmonary nodules to investigate frequency and reasons of false-positive findings. The ratio of detected tumors was best in the T1-category (90.4%) and decreased continuously: T2 (70.8%), T3 (29.4%), and T4 (8.8%). Tumor contact with the pleura was a strong predictor of misdetection. Segmentation performance was excellent for T1 tumors (r = 0.908, p < 0.001) and tumors without pleural contact (r = 0.971, p < 0.001). Volumes of larger tumors were systematically underestimated. There were 0.41 false-positive findings per exam. The algorithm tested facilitates a reliable detection and 3D segmentation of T1/T2 lung tumors on FDG-PET/CTs. The detection and segmentation of more advanced lung tumors is currently imprecise due to the conception of the algorithm for lung nodules <3 cm. Future efforts should therefore focus on this collective to facilitate segmentation of all tumor types and sizes to bridge the gap between CAD applications for screening and staging of lung cancer.

Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FS), multi-branch (MB) techniques, and multichannel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0:05) than the fusion baselines (FS: 99.00%, MB: 99.08%, TC: 98.92%) and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.

An Appraisal of Nodule Diagnosis for Lung Cancer in CT Images

As "the second eyes" of radiologists, computer-aided diagnosis systems play a significant role in nodule detection and diagnosis for lung cancer. In this paper, we aim to provide a systematic survey of state-of-the-art techniques (both traditional techniques and deep learning techniques) for nodule diagnosis from computed tomography images. This review first introduces the current progress and the popular structure used for nodule diagnosis. In particular, we provide a detailed overview of the five major stages in the computer-aided diagnosis systems: data acquisition, nodule segmentation, feature extraction, feature selection and nodule classification. Second, we provide a detailed report of the selected works and make a comprehensive comparison between selected works. The selected papers are from the IEEE Xplore, Science Direct, PubMed, and Web of Science databases up to December 2018. Third, we discuss and summarize the better techniques used in nodule diagnosis and indicate the existing future challenges in this field, such as improving the area under the receiver operating characteristic curve and accuracy, developing new deep learning-based diagnosis techniques, building efficient feature sets (fusing traditional features and deep features), developing high-quality labeled databases with malignant and benign nodules and promoting the cooperation between medical organizations and academic institutions.

Fluorine 18-FDG PET/CT and Diffusion-weighted MRI for Malignant versus Benign Pulmonary Lesions: A Meta-Analysis

Purpose To perform a meta-analysis of the literature to compare the diagnostic performance of fluorine 18 fluorodeoxyglucose PET/CT and diffusion-weighted (DW) MRI in the differentiation of malignant and benign pulmonary nodules and masses. Materials and Methods Published English-language studies on the diagnostic accuracy of PET/CT and/or DW MRI in the characterization of pulmonary lesions were searched in relevant databases through December 2017. The primary focus was on studies in which joint DW MRI and PET/CT were performed in the entire study population, to reduce interstudy heterogeneity. For DW MRI, lesion-to-spinal cord signal intensity ratio and apparent diffusion coefficient were evaluated; for PET/CT, maximum standard uptake value was evaluated. The pooled sensitivities, specificities, diagnostic odds ratios, and areas under the receiver operating characteristic curve (AUCs) for PET/CT and DW MRI were determined along with 95% confidence intervals (CIs). Results Thirty-seven studies met the inclusion criteria, with a total of 4224 participants and 4463 lesions (3090 malignant lesions [69.2%]). In the primary analysis of joint DW MRI and PET/CT studies (n = 6), DW MRI had a pooled sensitivity and specificity of 83% (95% CI: 75%, 89%) and 91% (95% CI: 80%, 96%), respectively, compared with 78% (95% CI: 70%, 84%) (P = .01 vs DW MRI) and 81% (95% CI: 72%, 88%) (P = .056 vs DW MRI) for PET/CT. DW MRI yielded an AUC of 0.93 (95% CI: 0.90, 0.95), versus 0.86 (95% CI: 0.83, 0.89) for PET/CT (P = .001). The diagnostic odds ratio of DW MRI (50 [95% CI: 19, 132]) was superior to that of PET/CT (15 [95% CI: 7, 32]) (P = .006). Conclusion The diagnostic performance of diffusion-weighted MRI is comparable or superior to that of fluorine 18 fluorodeoxyglucose PET/CT in the differentiation of malignant and benign pulmonary lesions. © RSNA, 2018 Online supplemental material is available for this article. See also the editorial by Schiebler in this issue.

Computed tomography-based volumetric tool for standardized measurement of the maxillary sinus

Volume measurements of maxillary sinus may be useful to identify diseases affecting paranasal sinuses. However, literature shows a lack of consensus in studies measuring the volume. This may be attributable to different computed tomography data acquisition techniques, segmentation methods, focuses of investigation, among other reasons. Furthermore, methods for volumetrically quantifying the maxillary sinus are commonly manual or semiautomated, which require substantial user expertise and are time-consuming. The purpose of the present study was to develop an automated tool for quantifying the total and air-free volume of the maxillary sinus based on computed tomography images. The quantification tool seeks to standardize maxillary sinus volume measurements, thus allowing better comparisons and determinations of factors that influence maxillary sinus size. The automated tool utilized image processing techniques (watershed, threshold, and morphological operators). The maxillary sinus volume was quantified in 30 patients. To evaluate the accuracy of the automated tool, the results were compared with manual segmentation that was performed by an experienced radiologist using a standard procedure. The mean percent differences between the automated and manual methods were 7.19% ± 5.83% and 6.93% ± 4.29% for total and air-free maxillary sinus volume, respectively. Linear regression and Bland-Altman statistics showed good agreement and low dispersion between both methods. The present automated tool for maxillary sinus volume assessment was rapid, reliable, robust, accurate, and reproducible and may be applied in clinical practice. The tool may be used to standardize measurements of maxillary volume. Such standardization is extremely important for allowing comparisons between studies, providing a better understanding of the role of the maxillary sinus, and determining the factors that influence maxillary sinus size under normal and pathological conditions.

Rapid Retrieval of Lung Nodule CT Images Based on Hashing and Pruning Methods

The similarity-based retrieval of lung nodule computed tomography (CT) images is an important task in the computer-aided diagnosis of lung lesions. It can provide similar clinical cases for physicians and help them make reliable clinical diagnostic decisions. However, when handling large-scale lung images with a general-purpose computer, traditional image retrieval methods may not be efficient. In this paper, a new retrieval framework based on a hashing method for lung nodule CT images is proposed. This method can translate high-dimensional image features into a compact hash code, so the retrieval time and required memory space can be reduced greatly. Moreover, a pruning algorithm is presented to further improve the retrieval speed, and a pruning-based decision rule is presented to improve the retrieval precision. Finally, the proposed retrieval method is validated on 2,450 lung nodule CT images selected from the public Lung Image Database Consortium (LIDC) database. The experimental results show that the proposed pruning algorithm effectively reduces the retrieval time of lung nodule CT images and improves the retrieval precision. In addition, the retrieval framework is evaluated by differentiating benign and malignant nodules, and the classification accuracy can reach 86.62%, outperforming other commonly used classification methods.

A Segmentation Method for Lung Parenchyma Image Sequences Based on Superpixels and a Self-Generating Neural Forest

Background

Lung parenchyma segmentation is often performed as an important pre-processing step in the computer-aided diagnosis of lung nodules based on CT image sequences. However, existing lung parenchyma image segmentation methods cannot fully segment all lung parenchyma images and have a slow processing speed, particularly for images in the top and bottom of the lung and the images that contain lung nodules.

Method

Our proposed method first uses the position of the lung parenchyma image features to obtain lung parenchyma ROI image sequences. A gradient and sequential linear iterative clustering algorithm (GSLIC) for sequence image segmentation is then proposed to segment the ROI image sequences and obtain superpixel samples. The SGNF, which is optimized by a genetic algorithm (GA), is then utilized for superpixel clustering. Finally, the grey and geometric features of the superpixel samples are used to identify and segment all of the lung parenchyma image sequences.

Results

Our proposed method achieves higher segmentation precision and greater accuracy in less time. It has an average processing time of 42.21 seconds for each dataset and an average volume pixel overlap ratio of 92.22 ± 4.02% for four types of lung parenchyma image sequences.