Lung nodule classification using deep Local-Global networks

Prior knowledge-based multi-task learning network for pulmonary nodule classification

The morphological characteristics of pulmonary nodule, also known as the attributes, are crucial for classification of benign and malignant nodules. In clinical, radiologists usually conduct a comprehensive analysis of correlations between different attributes, to accurately judge pulmonary nodules are benign or malignant. However, most of pulmonary nodule classification models ignore the inherent correlations between different attributes, leading to unsatisfactory classification performance. To address these problems, we propose a prior knowledge-based multi-task learning (PK-MTL) network for pulmonary nodule classification. To be specific, the correlations between different attributes are treated as prior knowledge, and established through multi-order task transfer learning. Then, the complex correlations between different attributes are encoded into hypergraph structure, and leverage hypergraph neural network for learning the correlation representation. On the other hand, a multi-task learning framework is constructed for joint segmentation, benign-malignant classification and attribute scoring of pulmonary nodules, aiming to improve the classification performance of pulmonary nodules comprehensively. In order to embed prior knowledge into multi-task learning framework, a feature fusion block is designed to organically integrate image-level features with attribute prior knowledge. In addition, a channel-wise cross attention block is constructed to fuse the features of encoder and decoder, to further improve the segmentation performance. Extensive experiments on LIDC-IDRI dataset show that our proposed method can achieve 91.04% accuracy for diagnosing malignant nodules, obtaining the state-of-art results.

Explainable Classification of Benign-Malignant Pulmonary Nodules With Neural Networks and Information Bottleneck

Computerized tomography (CT) is a clinically primary technique to differentiate benign-malignant pulmonary nodules for lung cancer diagnosis. Early classification of pulmonary nodules is essential to slow down the degenerative process and reduce mortality. The interactive paradigm assisted by neural networks is considered to be an effective means for early lung cancer screening in large populations. However, some inherent characteristics of pulmonary nodules in high-resolution CT images, e.g., diverse shapes and sparse distribution over the lung fields, have been inducing inaccurate results. On the other hand, most existing methods with neural networks are dissatisfactory from a lack of transparency. In order to overcome these obstacles, a united framework is proposed, including the classification and feature visualization stages, to learn distinctive features and provide visual results. Specifically, a bilateral scheme is employed to synchronously extract and aggregate global-local features in the classification stage, where the global branch is constructed to perceive deep-level features and the local branch is built to focus on the refined details. Furthermore, an encoder is built to generate some features, and a decoder is constructed to simulate decision behavior, followed by the information bottleneck viewpoint to optimize the objective. Extensive experiments are performed to evaluate our framework on two publicly available datasets, namely, 1) the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and 2) the Lung and Colon Histopathological Image Dataset (LC25000). For instance, our framework achieves 92.98% accuracy and presents additional visualizations on the LIDC. The experiment results show that our framework can obtain outstanding performance and is effective to facilitate explainability. It also demonstrates that this united framework is a serviceable tool and further has the scalability to be introduced into clinical research.

Artificial intelligence in lung cancer screening: Detection, classification, prediction, and prognosis

BACKGROUND

The exceptional capabilities of artificial intelligence (AI) in extracting image information and processing complex models have led to its recognition across various medical fields. With the continuous evolution of AI technologies based on deep learning, particularly the advent of convolutional neural networks (CNNs), AI presents an expanded horizon of applications in lung cancer screening, including lung segmentation, nodule detection, false-positive reduction, nodule classification, and prognosis.

METHODOLOGY

This review initially analyzes the current status of AI technologies. It then explores the applications of AI in lung cancer screening, including lung segmentation, nodule detection, and classification, and assesses the potential of AI in enhancing the sensitivity of nodule detection and reducing false-positive rates. Finally, it addresses the challenges and future directions of AI in lung cancer screening.

RESULTS

AI holds substantial prospects in lung cancer screening. It demonstrates significant potential in improving nodule detection sensitivity, reducing false-positive rates, and classifying nodules, while also showing value in predicting nodule growth and pathological/genetic typing.

CONCLUSIONS

AI offers a promising supportive approach to lung cancer screening, presenting considerable potential in enhancing nodule detection sensitivity, reducing false-positive rates, and classifying nodules. However, the universality and interpretability of AI results need further enhancement. Future research should focus on the large-scale validation of new deep learning-based algorithms and multi-center studies to improve the efficacy of AI in lung cancer screening.

Effective lung nodule detection using deep CNN with dual attention mechanisms

Novel methods are required to enhance lung cancer detection, which has overtaken other cancer-related causes of death as the major cause of cancer-related mortality. Radiologists have long-standing methods for locating lung nodules in patients with lung cancer, such as computed tomography (CT) scans. Radiologists must manually review a significant amount of CT scan pictures, which makes the process time-consuming and prone to human error. Computer-aided diagnosis (CAD) systems have been created to help radiologists with their evaluations in order to overcome these difficulties. These systems make use of cutting-edge deep learning architectures. These CAD systems are designed to improve lung nodule diagnosis efficiency and accuracy. In this study, a bespoke convolutional neural network (CNN) with a dual attention mechanism was created, which was especially crafted to concentrate on the most important elements in images of lung nodules. The CNN model extracts informative features from the images, while the attention module incorporates both channel attention and spatial attention mechanisms to selectively highlight significant features. After the attention module, global average pooling is applied to summarize the spatial information. To evaluate the performance of the proposed model, extensive experiments were conducted using benchmark dataset of lung nodules. The results of these experiments demonstrated that our model surpasses recent models and achieves state-of-the-art accuracy in lung nodule detection and classification tasks.

Pulmonary Nodule Classification Using a Multiview Residual Selective Kernel Network

Lung cancer is one of the leading causes of death worldwide and early detection is crucial to reduce the mortality. A reliable computer-aided diagnosis (CAD) system can help facilitate early detection of malignant nodules. Although existing methods provide adequate classification accuracy, there is still room for further improvement. This study is dedicated to investigating a new CAD scheme for predicting the malignant likelihood of lung nodules in computed tomography (CT) images in light of a deep learning strategy. Conceived from the residual learning and selective kernel, we investigated an efficient residual selective kernel (RSK) block to handle the diversity of lung nodules with various shapes and obscure structures. Founded on this RSK block, we established a multiview RSK network (MRSKNet), to which three anatomical planes in the axial, coronal, and sagittal directions were fed. To reinforce the classification efficiency, seven handcrafted texture features with a filter-like computation strategy were explored, among which the homogeneity (HOM) feature maps are combined with the corresponding intensity CT images for concatenation input, leading to an improved network architecture. Evaluated on the public benchmark Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) challenge database with ten-fold cross validation of binary classification, our experimental results indicated high area under receiver operating characteristic (AUC) and accuracy scores. A better compromise between recall and specificity was struck using the suggested concatenation strategy comparing to many state-of-the-art approaches. The proposed pulmonary nodule classification framework exhibited great efficacy and achieved a higher AUC of 0.9711. The association of handcrafted texture features with deep learning models is promising in advancing the classification performance. The developed pulmonary nodule CAD network architecture is of potential in facilitating the diagnosis of lung cancer for further image processing applications.

Lung nodule malignancy classification with associated pulmonary fibrosis using 3D attention-gated convolutional network with CT scans

BACKGROUND

Chest Computed tomography (CT) scans detect lung nodules and assess pulmonary fibrosis. While pulmonary fibrosis indicates increased lung cancer risk, current clinical practice characterizes nodule risk of malignancy based on nodule size and smoking history; little consideration is given to the fibrotic microenvironment.

PURPOSE

To evaluate the effect of incorporating fibrotic microenvironment into classifying malignancy of lung nodules in chest CT images using deep learning techniques.

MATERIALS AND METHODS

We developed a visualizable 3D classification model trained with in-house CT dataset for the nodule malignancy classification task. Three slightly-modified datasets were created: (1) nodule alone (microenvironment removed); (2) nodule with surrounding lung microenvironment; and (3) nodule in microenvironment with semantic fibrosis metadata. For each of the models, tenfold cross-validation was performed. Results were evaluated using quantitative measures, such as accuracy, sensitivity, specificity, and area-under-curve (AUC), as well as qualitative assessments, such as attention maps and class activation maps (CAM).

RESULTS

The classification model trained with nodule alone achieved 75.61% accuracy, 50.00% sensitivity, 88.46% specificity, and 0.78 AUC; the model trained with nodule and microenvironment achieved 79.03% accuracy, 65.46% sensitivity, 85.86% specificity, and 0.84 AUC. The model trained with additional semantic fibrosis metadata achieved 80.84% accuracy, 74.67% sensitivity, 84.95% specificity, and 0.89 AUC. Our visual evaluation of attention maps and CAM suggested that both the nodules and the microenvironment contributed to the task.

CONCLUSION

The nodule malignancy classification performance was found to be improving with microenvironment data. Further improvement was found when incorporating semantic fibrosis information.

Classification of benign and malignant pulmonary nodule based on local-global hybrid network

BACKGROUND

The accurate classification of pulmonary nodules has great application value in assisting doctors in diagnosing conditions and meeting clinical needs. However, the complexity and heterogeneity of pulmonary nodules make it difficult to extract valuable characteristics of pulmonary nodules, so it is still challenging to achieve high-accuracy classification of pulmonary nodules.

OBJECTIVE

In this paper, we propose a local-global hybrid network (LGHNet) to jointly model local and global information to improve the classification ability of benign and malignant pulmonary nodules.

METHODS

First, we introduce the multi-scale local (MSL) block, which splits the input tensor into multiple channel groups, utilizing dilated convolutions with different dilation rates and efficient channel attention to extract fine-grained local information at different scales. Secondly, we design the hybrid attention (HA) block to capture long-range dependencies in spatial and channel dimensions to enhance the representation of global features.

RESULTS

Experiments are carried out on the publicly available LIDC-IDRI and LUNGx datasets, and the accuracy, sensitivity, precision, specificity, and area under the curve (AUC) of the LIDC-IDRI dataset are 94.42%, 94.25%, 93.05%, 92.87%, and 97.26%, respectively. The AUC on the LUNGx dataset was 79.26%.

CONCLUSION

The above classification results are superior to the state-of-the-art methods, indicating that the network has better classification performance and generalization ability.

DeepXplainer: An interpretable deep learning based approach for lung cancer detection using explainable artificial intelligence

BACKGROUND AND OBJECTIVE

Artificial intelligence (AI) has several uses in the healthcare industry, some of which include healthcare management, medical forecasting, practical making of decisions, and diagnosis. AI technologies have reached human-like performance, but their use is limited since they are still largely viewed as opaque black boxes. This distrust remains the primary factor for their limited real application, particularly in healthcare. As a result, there is a need for interpretable predictors that provide better predictions and also explain their predictions.

METHODS

This study introduces "DeepXplainer", a new interpretable hybrid deep learning-based technique for detecting lung cancer and providing explanations of the predictions. This technique is based on a convolutional neural network and XGBoost. XGBoost is used for class label prediction after "DeepXplainer" has automatically learned the features of the input using its many convolutional layers. For providing explanations or explainability of the predictions, an explainable artificial intelligence method known as "SHAP" is implemented.

RESULTS

The open-source "Survey Lung Cancer" dataset was processed using this method. On multiple parameters, including accuracy, sensitivity, F1-score, etc., the proposed method outperformed the existing methods. The proposed method obtained an accuracy of 97.43%, a sensitivity of 98.71%, and an F1-score of 98.08. After the model has made predictions with this high degree of accuracy, each prediction is explained by implementing an explainable artificial intelligence method at both the local and global levels.

CONCLUSIONS

A deep learning-based classification model for lung cancer is proposed with three primary components: one for feature learning, another for classification, and a third for providing explanations for the predictions made by the proposed hybrid (ConvXGB) model. The proposed "DeepXplainer" has been evaluated using a variety of metrics, and the results demonstrate that it outperforms the current benchmarks. Providing explanations for the predictions, the proposed approach may help doctors in detecting and treating lung cancer patients more effectively.

A novel fusion algorithm for benign-malignant lung nodule classification on CT images

The accurate recognition of malignant lung nodules on CT images is critical in lung cancer screening, which can offer patients the best chance of cure and significant reductions in mortality from lung cancer. Convolutional Neural Network (CNN) has been proven as a powerful method in medical image analysis. Radiomics which is believed to be of interest based on expert opinion can describe high-throughput extraction from CT images. Graph Convolutional Network explores the global context and makes the inference on both graph node features and relational structures. In this paper, we propose a novel fusion algorithm, RGD, for benign-malignant lung nodule classification by incorporating Radiomics study and Graph learning into the multiple Deep CNNs to form a more complete and distinctive feature representation, and ensemble the predictions for robust decision-making. The proposed method was conducted on the publicly available LIDC-IDRI dataset in a 10-fold cross-validation experiment and it obtained an average accuracy of 93.25%, a sensitivity of 89.22%, a specificity of 95.82%, precision of 92.46%, F1 Score of 0.9114 and AUC of 0.9629. Experimental results illustrate that the RGD model achieves superior performance compared with the state-of-the-art methods. Moreover, the effectiveness of the fusion strategy has been confirmed by extensive ablation studies. In the future, the proposed model which performs well on the pulmonary nodule classification on CT images will be applied to increase confidence in the clinical diagnosis of lung cancer.

A novel higher order appearance texture analysis to diagnose lung cancer based on a modified local ternary pattern

BACKGROUND AND OBJECTIVE

Lung cancer is an important cause of death and morbidity around the world. Two of the primary computed tomography (CT) imaging markers that can be used to differentiate malignant and benign lung nodules are the inhomogeneity of the nodules' texture and nodular morphology. The objective of this paper is to present a new model that can capture the inhomogeneity of the detected lung nodules as well as their morphology.

METHODS

We modified the local ternary pattern to use three different levels (instead of two) and a new pattern identification algorithm to capture the nodule's inhomogeneity and morphology in a more accurate and flexible way. This modification aims to address the wide Hounsfield unit value range of the detected nodules which decreases the ability of the traditional local binary/ternary pattern to accurately classify nodules' inhomogeneity. The cut-off values defining these three levels of the novel technique are estimated empirically from the training data. Subsequently, the extracted imaging markers are fed to a hyper-tuned stacked generalization-based classification architecture to classify the nodules as malignant or benign. The proposed system was evaluated on in vivo datasets of 679 CT scans (364 malignant nodules and 315 benign nodules) from the benchmark Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and an external dataset of 100 CT scans (50 malignant and 50 benign). The performance of the classifier was quantitatively assessed using a Leave-one-out cross-validation approach and externally validated using the unseen external dataset based on sensitivity, specificity, and accuracy.

RESULTS

The overall accuracy of the system is 96.17% with 97.14% sensitivity and 95.33% specificity. The area under the receiver-operating characteristic curve was 0.98, which highlights the robustness of the system. Using the unseen external dataset for validating the system led to consistent results showing the generalization abilities of the proposed approach. Moreover, applying the original local binary/ternary pattern or using other classification structures achieved inferior performance when compared against the proposed approach.

CONCLUSIONS

These experimental results demonstrate the feasibility of the proposed model as a novel tool to assist physicians and radiologists for lung nodules' early assessment based on the new comprehensive imaging markers.

An Interpretable Three-Dimensional Artificial Intelligence Model for Computer-Aided Diagnosis of Lung Nodules in Computed Tomography Images

Lung cancer is typically classified into small-cell carcinoma and non-small-cell carcinoma. Non-small-cell carcinoma accounts for approximately 85% of all lung cancers. Low-dose chest computed tomography (CT) can quickly and non-invasively diagnose lung cancer. In the era of deep learning, an artificial intelligence (AI) computer-aided diagnosis system can be developed for the automatic recognition of CT images of patients, creating a new form of intelligent medical service. For many years, lung cancer has been the leading cause of cancer-related deaths in Taiwan, with smoking and air pollution increasing the likelihood of developing the disease. The incidence of lung adenocarcinoma in never-smoking women has also increased significantly in recent years, resulting in an important public health problem. Early detection of lung cancer and prompt treatment can help reduce the mortality rate of patients with lung cancer. In this study, an improved 3D interpretable hierarchical semantic convolutional neural network named HSNet was developed and validated for the automatic diagnosis of lung cancer based on a collection of lung nodule images. The interpretable AI model proposed in this study, with different training strategies and adjustment of model parameters, such as cyclic learning rate and random weight averaging, demonstrated better diagnostic performance than the previous literature, with results of a four-fold cross-validation procedure showing calcification: 0.9873 ± 0.006, margin: 0.9207 ± 0.009, subtlety: 0.9026 ± 0.014, texture: 0.9685 ± 0.006, sphericity: 0.8652 ± 0.021, and malignancy: 0.9685 ± 0.006.

Artificial Intelligence in Lung Cancer Screening: The Future Is Now

Lung cancer has one of the worst morbidity and fatality rates of any malignant tumour. Most lung cancers are discovered in the middle and late stages of the disease, when treatment choices are limited, and patients' survival rate is low. The aim of lung cancer screening is the identification of lung malignancies in the early stage of the disease, when more options for effective treatments are available, to improve the patients' outcomes. The desire to improve the efficacy and efficiency of clinical care continues to drive multiple innovations into practice for better patient management, and in this context, artificial intelligence (AI) plays a key role. AI may have a role in each process of the lung cancer screening workflow. First, in the acquisition of low-dose computed tomography for screening programs, AI-based reconstruction allows a further dose reduction, while still maintaining an optimal image quality. AI can help the personalization of screening programs through risk stratification based on the collection and analysis of a huge amount of imaging and clinical data. A computer-aided detection (CAD) system provides automatic detection of potential lung nodules with high sensitivity, working as a concurrent or second reader and reducing the time needed for image interpretation. Once a nodule has been detected, it should be characterized as benign or malignant. Two AI-based approaches are available to perform this task: the first one is represented by automatic segmentation with a consequent assessment of the lesion size, volume, and densitometric features; the second consists of segmentation first, followed by radiomic features extraction to characterize the whole abnormalities providing the so-called "virtual biopsy". This narrative review aims to provide an overview of all possible AI applications in lung cancer screening.

An integrated convolutional neural network for classifying small pulmonary solid nodules

Achieving accurate classification of benign and malignant pulmonary nodules is essential for treating some diseases. However, traditional typing methods have difficulty obtaining satisfactory results on small pulmonary solid nodules, mainly caused by two aspects: (1) noise interference from other tissue information; (2) missing features of small nodules caused by downsampling in traditional convolutional neural networks. To solve these problems, this paper proposes a new typing method to improve the diagnosis rate of small pulmonary solid nodules in CT images. Specifically, first, we introduce the Otsu thresholding algorithm to preprocess the data and filter the interference information. Then, to acquire more small nodule features, we add parallel radiomics to the 3D convolutional neural network. Radiomics can extract a large number of quantitative features from medical images. Finally, the classifier generated more accurate results by the visual and radiomic features. In the experiments, we tested the proposed method on multiple data sets, and the proposed method outperformed other methods in the small pulmonary solid nodule classification task. In addition, various groups of ablation experiments demonstrated that the Otsu thresholding algorithm and radiomics are helpful for the judgment of small nodules and proved that the Otsu thresholding algorithm is more flexible than the manual thresholding algorithm.

TPFR-Net: U-shaped model for lung nodule segmentation based on transformer pooling and dual-attention feature reorganization

Accurate segmentation of lung nodules is the key to diagnosing the lesion type of lung nodule. The complex boundaries of lung nodules and the visual similarity to surrounding tissues make precise segmentation of lung nodules challenging. Traditional CNN based lung nodule segmentation models focus on extracting local features from neighboring pixels and ignore global contextual information, which is prone to incomplete segmentation of lung nodule boundaries. In the U-shaped encoder-decoder structure, variations of image resolution caused by up-sampling and down-sampling result in the loss of feature information, which reduces the reliability of output features. This paper proposes transformer pooling module and dual-attention feature reorganization module to effectively improve the above two defects. Transformer pooling module innovatively fuses the self-attention layer and pooling layer in the transformer, which compensates for the limitation of convolution operation, reduces the loss of feature information in the pooling process, and decreases the computational complexity of the Transformer significantly. Dual-attention feature reorganization module innovatively employs the dual-attention mechanism of channel and spatial to improve the sub-pixel convolution, minimizing the loss of feature information during up-sampling. In addition, two convolutional modules are proposed in this paper, which together with transformer pooling module form an encoder that can adequately extract local features and global dependencies. We use the fusion loss function and deep supervision strategy in the decoder to train the model. The proposed model has been extensively experimented and evaluated on the LIDC-IDRI dataset, the highest Dice Similarity Coefficient is 91.84 and the highest sensitivity is 92.66, indicating the model's comprehensive capability has surpassed state-of-the-art UTNet. The model proposed in this paper has superior segmentation performance for lung nodules and can provide a more in-depth assessment of lung nodules' shape, size, and other characteristics, which is of important clinical significance and application value to assist physicians in the early diagnosis of lung nodules.

WS-LungNet: A two-stage weakly-supervised lung cancer detection and diagnosis network

Computer-aided lung cancer diagnosis (CAD) system on computed tomography (CT) helps radiologists guide preoperative planning and prognosis assessment. The flexibility and scalability of deep learning methods are limited in lung CAD. In essence, two significant challenges to be solved are (1) Label scarcity due to cost annotations of CT images by experienced domain experts, and (2) Label inconsistency between the observed nodule malignancy and the patients' pathology evaluation. These two issues can be considered weak label problems. We address these issues in this paper by introducing a weakly-supervised lung cancer detection and diagnosis network (WS-LungNet), consisting of a semi-supervised computer-aided detection (Semi-CADe) that can segment 3D pulmonary nodules based on unlabeled data through adversarial learning to reduce label scarcity, as well as a cross-nodule attention computer-aided diagnosis (CNA-CADx) for evaluating malignancy at the patient level by modeling correlations between nodules via cross-attention mechanisms and thereby eliminating label inconsistency. Through extensive evaluations on the LIDC-IDRI public database, we show that our proposed method achieves 82.99% competition performance metric (CPM) on pulmonary nodule detection and 88.63% area under the curve (AUC) on lung cancer diagnosis. Extensive experiments demonstrate the advantage of WS-LungNet on nodule detection and malignancy evaluation tasks. Our promising results demonstrate the benefits and flexibility of the semi-supervised segmentation with adversarial learning and the nodule instance correlation learning with the attention mechanism. The results also suggest that making use of the unlabeled data and taking the relationship among nodules in a case into account are essential for lung cancer detection and diagnosis.

An improved 3-D attention CNN with hybrid loss and feature fusion for pulmonary nodule classification

BACKGROUND AND OBJECTIVE

Lung cancer has the highest cancer-related mortality worldwide, and lung nodule usually presents with no symptom. Low-dose computed tomography (LDCT) was an important tool for lung cancer detection and diagnosis. It provided a complete three-dimensional (3-D) chest image with a high resolution.Recently, convolutional neural network (CNN) had flourished and been proven the CNN-based computer-aided diagnosis (CADx) system could extract the features and help radiologists to make a preliminary diagnosis. Therefore, a 3-D ResNeXt-based CADx system was proposed to assist radiologists for diagnosis in this study.

METHODS

The proposed CADx system consists of image preprocessing and a 3-D CNN-based classification model for pulmonary nodule classification. First, the image preprocessing was executed to generate the normalized volumn of interest (VOI) only including nodule information and a few surrounding tissues. Then, the extracted VOI was forwarded to the 3-D nodule classification model. In the classification model, the RestNext was employed as the backbone and the attention scheme was embedded to focus on the important features. Moreover, a multi-level feature fusion network incorporating feature information of different scales was used to enhance the prediction accuracy of small malignant nodules. Finally, a hybrid loss based on channel optimization which make the network learn more detailed information was empolyed to replace a binary cross-entropy (BCE) loss.

RESULTS

In this research, there were a total of 880 low-dose CT images including 440 benign and 440 malignant nodules from the American National Lung Screening Trial (NLST) for system evaluation. The results showed that our system could achieve the accuracy of 85.3%, the sensitivity of 86.8%, the specificity of 83.9%, and the area-under-curve (AUC) value was 0.9042. It was confirmed that the designed system had a good diagnostic ability.

CONCLUSION

In this study, a CADx composed of the image preprocessing and a 3-D nodule classification model with attention scheme, feature fusion, and hybrid loss was proposed for pulmonary nodule classification in LDCT. The results indicated that the proposed CADx system had potential for achieving high performance in classifying lung nodules as benign and malignant.

A Series-Based Deep Learning Approach to Lung Nodule Image Classification

Although many studies have shown that deep learning approaches yield better results than traditional methods based on manual features, CADs methods still have several limitations. These are due to the diversity in imaging modalities and clinical pathologies. This diversity creates difficulties because of variation and similarities between classes. In this context, the new approach from our study is a hybrid method that performs classifications using both medical image analysis and radial scanning series features. Hence, the areas of interest obtained from images are subjected to a radial scan, with their centers as poles, in order to obtain series. A U-shape convolutional neural network model is then used for the 4D data classification problem. We therefore present a novel approach to the classification of 4D data obtained from lung nodule images. With radial scanning, the eigenvalue of nodule images is captured, and a powerful classification is performed. According to our results, an accuracy of 92.84% was obtained and much more efficient classification scores resulted as compared to recent classifiers.

Open-Source Biomedical Image Analysis Models: A Meta-Analysis and Continuous Survey

Open-source research software has proven indispensable in modern biomedical image analysis. A multitude of open-source platforms drive image analysis pipelines and help disseminate novel analytical approaches and algorithms. Recent advances in machine learning allow for unprecedented improvement in these approaches. However, these novel algorithms come with new requirements in order to remain open source. To understand how these requirements are met, we have collected 50 biomedical image analysis models and performed a meta-analysis of their respective papers, source code, dataset, and trained model parameters. We concluded that while there are many positive trends in openness, only a fraction of all publications makes all necessary elements available to the research community.

Res-trans networks for lung nodule classification

PURPOSE

Lung cancer usually presents as pulmonary nodules on early diagnostic images, and accurately estimating the malignancy of pulmonary nodules is crucial to the prevention and diagnosis of lung cancer. Recently, deep learning algorithms based on convolutional neural networks have shown potential for pulmonary nodules classification. However, the size of the nodules is very diverse, ranging from 3 to 30 mm, which makes classifying them to be a challenging task. In this study, we propose a novel architecture called Res-trans networks to classify nodules in computed tomography (CT) scans.

METHODS

We designed local and global blocks to extract features that capture the long-range dependencies between pixels to adapt to the correct classification of lung nodules of different sizes. Specifically, we designed residual blocks with convolutional operations to extract local features and transformer blocks with self-attention to capture global features. Moreover, the Res-trans network has a sequence fusion block that aggregates and extracts the sequence feature information output by the transformer block that improves classification accuracy.

RESULTS

Our proposed method is extensively evaluated on the public LIDC-IDRI dataset, which contains 1,018 CT scans. A tenfold cross-validation result shows that our method obtains better performance with AUC = 0.9628 and Accuracy = 0.9292 compared with recently leading methods.

CONCLUSION

In this paper, a network that can capture local and global features is proposed to classify nodules in chest CT. Experimental results show that our proposed method has better classification performance and can help radiologists to accurately analyze lung nodules.

Evolving Optimised Convolutional Neural Networks for Lung Cancer Classification

Detecting pulmonary nodules early significantly contributes to the treatment success of lung cancer. Several deep learning models for medical image analysis have been developed to help classify pulmonary nodules. The design of convolutional neural network (CNN) architectures, however, is still heavily reliant on human domain knowledge. Manually designing CNN design solutions has been shown to limit the data’s utility by creating a co-dependency on the creator’s cognitive bias, which urges the development of smart CNN architecture design solutions. In this paper, an evolutionary algorithm is used to optimise the classification of pulmonary nodules with CNNs. The implementation of a genetic algorithm (GA) for CNN architectures design and hyperparameter optimisation is proposed, which approximates optimal solutions by implementing a range of bio-inspired mechanisms of natural selection and Darwinism. For comparison purposes, two manually designed deep learning models, FractalNet and Deep Local-Global Network, were trained. The results show an outstanding classification accuracy of the fittest GA-CNN (91.3%), which outperformed both manually designed models. The findings indicate that GAs pose advantageous solutions for diagnostic challenges, the development of which may to be fully automated in the future using GAs to design and optimise CNN architectures for various clinical applications.

Self-Supervised Transfer Learning Based on Domain Adaptation for Benign-Malignant Lung Nodule Classification on Thoracic CT

The spatial heterogeneity is an important indicator of the malignancy of lung nodules in lung cancer diagnosis. Compared with 2D nodule CT images, the 3D volumes with entire nodule objects hold richer discriminative information. However, for deep learning methods driven by massive data, effectively capturing the 3D discriminative features of nodules in limited labeled samples is a challenging task. Different from previous models that proposed transfer learning models in a 2D pattern or learning from scratch 3D models, we develop a self-supervised transfer learning based on domain adaptation (SSTL-DA) 3D CNN framework for benign-malignant lung nodule classification. At first, a data pre-processing strategy termed adaptive slice selection (ASS) is developed to eliminate the redundant noise of the input samples with lung nodules. Then, the self-supervised learning network is constructed to learn robust image representation from CT images. Finally, a transfer learning method based on domain adaptation is designed to obtain discriminant features for classification. The proposed SSTL-DA method has been assessed on the LIDC-IDRI benchmark dataset, and it obtains an accuracy of 91.07% and an AUC of 95.84%. These results demonstrate that the SSTL-DA model achieves quite a competitive classification performance compared with some state-of-the-art approaches.

Lung Nodule Segmentation and Recognition Algorithm Based on Multiposition U-Net

Lung nodules are the main lesions of the lung, and conditions of the lung can be directly displayed through CT images. Due to the limited pixel number of lung nodules in the lung, doctors have the risk of missed detection and false detection in the detection process. In order to reduce doctors' work intensity and assist doctors to make accurate diagnosis, a lung nodule segmentation and recognition algorithm is proposed by simulating doctors' diagnosis process with computer intelligent methods. Firstly, the attention mechanism model is established to focus on the region of lung parenchyma. Then, a pyramid network of bidirectional enhancement features is established from multiple body positions to extract lung nodules. Finally, the morphological and imaging features of lung nodules are calculated, and then, the signs of lung nodules can be identified. The experiments show that the algorithm conforms to the doctor's diagnosis process, focuses the region of interest step by step, and achieves good results in lung nodule segmentation and recognition.

A multi-task two-path deep learning system for predicting the invasiveness of craniopharyngioma

BACKGROUND AND OBJECTIVE

Craniopharyngioma is a kind of benign brain tumor in histography. However, it might be clinically aggressive and have severe manifestations, such as increased intracranial pressure, hypothalamic-pituitary dysfunction, and visual impairment. It is considered challenging for radiologists to predict the invasiveness of craniopharyngioma through MRI images. Therefore, developing a non-invasive method that can predict the invasiveness and boundary of CP as a reference before surgery is of clinical value for making more appropriate and individualized treatment decisions and reducing the occurrence of inappropriate surgical plan choices.

METHODS

The MT-Brain system has consisted of two pathways, a sub-path based on 2D CNN for capturing the features from each slice of MRI images, and a 3D sub-network for capturing additional context information between slices. By introducing the two-path architecture, our system can make full use of the fusion of the above 2D and 3D features for classification. Furthermore, position encoding and mask-guided attention also have been introduced to improve the segmentation and diagnosis performance. To verify the performance of the MT-Brain system, we have enrolled 1032 patients with craniopharyngioma (302 invasion and 730 non-invasion patients), segmented the tumors on postcontrast coronal T1WI and randomized them into a training dataset and a testing dataset at a ratio of 8:2.

RESULTS

The MT-Brain system achieved a remarkable performance in diagnosing the invasiveness of craniopharyngioma with the AUC of 83.84%, the accuracy of 77.94%, the sensitivity of 70.97%, and the specificity of 80.99%. In the lesion segmentation task, the predicted boundaries of lesions were similar to those labeled by radiologists with the dice of 66.36%. In addition, some explorations also have been made on the interpretability of deep learning models, illustrating the reliability of the model.

CONCLUSIONS

To the best of our knowledge, this study is the first to develop an integrated deep learning model to predict the invasiveness of craniopharyngioma preoperatively and locate the lesion boundary synchronously on MRI. The excellent performances indicate that the MT-Brain system has great potential in real-world clinical applications.

Deep Learning Applications in Computed Tomography Images for Pulmonary Nodule Detection and Diagnosis: A Review

Lung cancer has one of the highest mortality rates of all cancers and poses a severe threat to people’s health. Therefore, diagnosing lung nodules at an early stage is crucial to improving patient survival rates. Numerous computer-aided diagnosis (CAD) systems have been developed to detect and classify such nodules in their early stages. Currently, CAD systems for pulmonary nodules comprise data acquisition, pre-processing, lung segmentation, nodule detection, false-positive reduction, segmentation, and classification. A number of review articles have considered various components of such systems, but this review focuses on segmentation and classification parts. Specifically, categorizing segmentation parts based on lung nodule type and network architectures, i.e., general neural network and multiview convolution neural network (CNN) architecture. Moreover, this work organizes related literature for classification of parts based on nodule or non-nodule and benign or malignant. The essential CT lung datasets and evaluation metrics used in the detection and diagnosis of lung nodules have been systematically summarized as well. Thus, this review provides a baseline understanding of the topic for interested readers.

Artificial intelligence: opportunities in lung cancer

PURPOSE OF REVIEW

In this article, we focus on the role of artificial intelligence in the management of lung cancer. We summarized commonly used algorithms, current applications and challenges of artificial intelligence in lung cancer.

RECENT FINDINGS

Feature engineering for tabular data and computer vision for image data are commonly used algorithms in lung cancer research. Furthermore, the use of artificial intelligence in lung cancer has extended to the entire clinical pathway including screening, diagnosis and treatment. Lung cancer screening mainly focuses on two aspects: identifying high-risk populations and the automatic detection of lung nodules. Artificial intelligence diagnosis of lung cancer covers imaging diagnosis, pathological diagnosis and genetic diagnosis. The artificial intelligence clinical decision-support system is the main application of artificial intelligence in lung cancer treatment. Currently, the challenges of artificial intelligence applications in lung cancer mainly focus on the interpretability of artificial intelligence models and limited annotated datasets; and recent advances in explainable machine learning, transfer learning and federated learning might solve these problems.

SUMMARY

Artificial intelligence shows great potential in many aspects of the management of lung cancer, especially in screening and diagnosis. Future studies on interpretability and privacy are needed for further application of artificial intelligence in lung cancer.

Interpretative computer-aided lung cancer diagnosis: From radiology analysis to malignancy evaluation

BACKGROUND AND OBJECTIVE

Computer-aided diagnosis (CAD) systems promote accurate diagnosis and reduce the burden of radiologists. A CAD system for lung cancer diagnosis includes nodule candidate detection and nodule malignancy evaluation. Recently, deep learning-based pulmonary nodule detection has reached satisfactory performance ready for clinical application. However, deep learning-based nodule malignancy evaluation depends on heuristic inference from low-dose computed tomography (LDCT) volume to malignant probability, and lacks clinical cognition.

METHODS

In this paper, we propose a joint radiology analysis and malignancy evaluation network called R2MNet to evaluate pulmonary nodule malignancy via the analysis of radiological characteristics. Radiological features are extracted as channel descriptor to highlight specific regions of the input volume that are critical for nodule malignancy evaluation. In addition, for model explanations, we propose channel-dependent activation mapping (CDAM) to visualize features and shed light on the decision process of deep neural networks (DNNs).

RESULTS

Experimental results on the lung image database consortium image collection (LIDC-IDRI) dataset demonstrate that the proposed method achieved an area under curve (AUC) of 96.27% and 97.52% on nodule radiology analysis and nodule malignancy evaluation, respectively. In addition, explanations of CDAM features proved that the shape and density of nodule regions are two critical factors that influence a nodule to be inferred as malignant. This process conforms to the diagnosis cognition of experienced radiologists.

CONCLUSION

The network inference process conforms to the diagnostic procedure of radiologists and increases the confidence of evaluation results by incorporating radiology analysis with nodule malignancy evaluation. Besides, model interpretation with CDAM features shed light on the focus regions of DNNs during the estimation of nodule malignancy probabilities.

3D axial-attention for lung nodule classification

PURPOSE

In recent years, Non-Local-based methods have been successfully applied to lung nodule classification. However, these methods offer 2D attention or limited 3D attention to low-resolution feature maps. Moreover, they still depend on a convenient local filter such as convolution as full 3D attention is expensive to compute and requires a big dataset, which might not be available.

METHODS

We propose to use 3D Axial-Attention, which requires a fraction of the computing power of a regular Non-Local network (i.e., self-attention). Unlike a regular Non-Local network, the 3D Axial-Attention network applies the attention operation to each axis separately. Additionally, we solve the invariant position problem of the Non-Local network by proposing to add 3D positional encoding to shared embeddings.

RESULTS

We validated the proposed method on 442 benign nodules and 406 malignant nodules, extracted from the public LIDC-IDRI dataset by following a rigorous experimental setup using only nodules annotated by at least three radiologists. Our results show that the 3D Axial-Attention model achieves state-of-the-art performance on all evaluation metrics, including AUC and Accuracy.

CONCLUSIONS

The proposed model provides full 3D attention, whereby every element (i.e., pixel) in the 3D volume space attends to every other element in the nodule effectively. Thus, the 3D Axial-Attention network can be used in all layers without the need for local filters. The experimental results show the importance of full 3D attention for classifying lung nodules.

Multi-Dimension and Multi-Feature Hybrid Learning Network for Classifying the Sub Pathological Type of Lung Nodules through LDCT

In order to develop appropriate treatment and rehabilitation plans with regard to different subpathological types (PILs and IAs) of lung nodules, it is important to diagnose them through low-dose spiral computed tomography (LDCT) during routine screening before surgery. Based on the characteristics of different subpathological lung nodules expressed from LDCT images, we propose a multi-dimension and multi-feature hybrid learning neural network in this paper. Our network consists of a 2D network part and a 3D network part. The feature vectors extracted from the 2D network and 3D network are further learned by XGBoost. Through this formation, the network can better integrate the feature information from the 2D and 3D networks. The main learning block of the network is a residual block combined with attention mechanism. This learning block enables the network to learn better from multiple features and pay more attention to the key feature map among all the feature maps in different channels. We conduct experiments on our dataset collected from a cooperating hospital. The results show that the accuracy, sensitivity and specificity of our network are 83%, 86%, 80%, respectively It is feasible to use this network to classify the subpathological type of lung nodule through routine screening.

On the robustness of deep learning-based lung-nodule classification for CT images with respect to image noise

Robustness is an important aspect when evaluating a method of medical image analysis. In this study, we investigated the robustness of a deep learning (DL)-based lung-nodule classification model for CT images with respect to noise perturbations. A deep neural network (DNN) was established to classify 3D CT images of lung nodules into malignant or benign groups. The established DNN was able to predict malignancy rate of lung nodules based on CT images, achieving the area under the curve of 0.91 for the testing dataset in a tenfold cross validation as compared to radiologists' prediction. We then evaluated its robustness against noise perturbations. We added to the input CT images noise signals generated randomly or via an optimization scheme using a realistic noise model based on a noise power spectrum for a given mAs level, and monitored the DNN's output. The results showed that the CT noise was able to affect the prediction results of the established DNN model. With random noise perturbations at 100 mAs, DNN's predictions for 11.2% of training data and 17.4% of testing data were successfully altered by at least once. The percentage increased to 23.4% and 34.3%, respectively, for optimization-based perturbations. We further evaluated robustness of models with different architectures, parameters, number of output labels, etc, and robustness concern was found in these models to different degrees. To improve model robustness, we empirically proposed an adaptive training scheme. It fine-tuned the DNN model by including perturbations in the training dataset that successfully altered the DNN's perturbations. The adaptive scheme was repeatedly performed to gradually improve DNN's robustness. The numbers of perturbations at 100 mAs affecting DNN's predictions were reduced to 10.8% for training and 21.1% for testing by the adaptive training scheme after two iterations. Our study illustrated that robustness may potentially be a concern for an exemplary DL-based lung-nodule classification model for CT images, indicating the needs for evaluating and ensuring model robustness when developing similar models. The proposed adaptive training scheme may be able to improve model robustness.

Multi-Level Cross Residual Network for Lung Nodule Classification

Computer-aided algorithm plays an important role in disease diagnosis through medical images. As one of the major cancers, lung cancer is commonly detected by computer tomography. To increase the survival rate of lung cancer patients, an early-stage diagnosis is necessary. In this paper, we propose a new structure, multi-level cross residual convolutional neural network (ML-xResNet), to classify the different types of lung nodule malignancies. ML-xResNet is constructed by three-level parallel ResNets with different convolution kernel sizes to extract multi-scale features of the inputs. Moreover, the residuals are connected not only with the current level but also with other levels in a crossover manner. To illustrate the performance of ML-xResNet, we apply the model to process ternary classification (benign, indeterminate, and malignant lung nodules) and binary classification (benign and malignant lung nodules) of lung nodules, respectively. Based on the experiment results, the proposed ML-xResNet achieves the best results of 85.88% accuracy for ternary classification and 92.19% accuracy for binary classification, without any additional handcrafted preprocessing algorithm.

Bridging the Gap Between AI and Healthcare Sides: Towards Developing Clinically Relevant AI-Powered Diagnosis Systems

Despite the success of Convolutional Neural Network-based Computer-Aided Diagnosis research, its clinical applications remain challenging. Accordingly, developing medical Artificial Intelligence (AI) fitting into a clinical environment requires identifying/bridging the gap between AI and Healthcare sides. Since the biggest problem in Medical Imaging lies in data paucity, confirming the clinical relevance for diagnosis of research-proven image augmentation techniques is essential. Therefore, we hold a clinically valuable AI-envisioning workshop among Japanese Medical Imaging experts, physicians, and generalists in Healthcare/Informatics. Then, a questionnaire survey for physicians evaluates our pathology-aware Generative Adversarial Network (GAN)-based image augmentation projects in terms of Data Augmentation and physician training. The workshop reveals the intrinsic gap between AI/Healthcare sides and solutions on Why (i.e., clinical significance/interpretation) and How (i.e., data acquisition, commercial deployment, and safety/feeling safe). This analysis confirms our pathology-aware GANs’ clinical relevance as a clinical decision support system and non-expert physician training tool. Our findings would play a key role in connecting inter-disciplinary research and clinical applications, not limited to the Japanese medical context and pathology-aware GANs.

Lung Nodule Malignancy Prediction From Longitudinal CT Scans With Siamese Convolutional Attention Networks

Goal: We propose a convolutional attention-based network that allows for use of pre-trained 2-D convolutional feature extractors and is extendable to multi-time-point classification in a Siamese structure. Methods: Our proposed framework is evaluated for single- and multi-time-point classification to explore the value that temporal information, such as nodule growth, adds to malignancy prediction. Results: Our results show that the proposed method outperforms a comparable 3-D network with less than half the parameters on single-time-point classification and further achieves performance gains on multi-time-point classification. Conclusions: Attention-based, Siamese 2-D pre-trained CNNs lead to fast training times and are effective for malignancy prediction from single-time-point or multiple-time-point imaging data.

A manifold learning regularization approach to enhance 3D CT image-based lung nodule classification

PURPOSE

Diagnosis of lung cancer requires radiologists to review every lung nodule in CT images. Such a process can be very time-consuming, and the accuracy is affected by many factors, such as experience of radiologists and available diagnosis time. To address this problem, we proposed to develop a deep learning-based system to automatically classify benign and malignant lung nodules.

METHODS

The proposed method automatically determines benignity or malignancy given the 3D CT image patch of a lung nodule to assist diagnosis process. Motivated by the fact that real structure among data is often embedded on a low-dimensional manifold, we developed a novel manifold regularized classification deep neural network (MRC-DNN) to perform classification directly based on the manifold representation of lung nodule images. The concise manifold representation revealing important data structure is expected to benefit the classification, while the manifold regularization enforces strong, but natural constraints on network training, preventing over-fitting.

RESULTS

The proposed method achieves accurate manifold learning with reconstruction error of ~ 30 HU on real lung nodule CT image data. In addition, the classification accuracy on testing data is 0.90 with sensitivity of 0.81 and specificity of 0.95, which outperforms state-of-the-art deep learning methods.

CONCLUSION

The proposed MRC-DNN facilitates an accurate manifold learning approach for lung nodule classification based on 3D CT images. More importantly, MRC-DNN suggests a new and effective idea of enforcing regularization for network training, possessing the potential impact to a board range of applications.