1
|
Liu H, Cao H, Song E, Ma G, Xu X, Jin R, Liu C, Hung CC. Multi-model Ensemble Learning Architecture Based on 3D CNN for Lung Nodule Malignancy Suspiciousness Classification. J Digit Imaging 2020; 33:1242-1256. [PMID: 32607905 PMCID: PMC7649841 DOI: 10.1007/s10278-020-00372-8] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022] Open
Abstract
Classification of benign and malignant in lung nodules using chest CT images is a key step in the diagnosis of early-stage lung cancer, as well as an effective way to improve the patients' survival rate. However, due to the diversity of lung nodules and the visual similarity of lung nodules to their surrounding tissues, it is difficult to construct a robust classification model with conventional deep learning-based diagnostic methods. To address this problem, we propose a multi-model ensemble learning architecture based on 3D convolutional neural network (MMEL-3DCNN). This approach incorporates three key ideas: (1) Constructed multi-model network architecture can be well adapted to the heterogeneity of lung nodules. (2) The input that concatenated of the intensity image corresponding to the nodule mask, the original image, and the enhanced image corresponding to which can help training model to extract advanced feature with more discriminative capacity. (3) Select the corresponding model to different nodule size dynamically for prediction, which can improve the generalization ability of the model effectively. In addition, ensemble learning is applied in this paper to further improve the robustness of the nodule classification model. The proposed method has been experimentally verified on the public dataset, LIDC-IDRI. The experimental results show that the proposed MMEL-3DCNN architecture can obtain satisfactory classification results.
Collapse
Affiliation(s)
- Hong Liu
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Haichao Cao
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Enmin Song
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China.
| | - Guangzhi Ma
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Xiangyang Xu
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Renchao Jin
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Chuhua Liu
- School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China
| | - Chih-Cheng Hung
- Laboratory for Machine Vision and Security Research, Kennesaw State University, Kennesaw, GA, USA
| |
Collapse
|
2
|
Farhat H, Sakr GE, Kilany R. Deep learning applications in pulmonary medical imaging: recent updates and insights on COVID-19. MACHINE VISION AND APPLICATIONS 2020; 31:53. [PMID: 32834523 PMCID: PMC7386599 DOI: 10.1007/s00138-020-01101-5] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 06/21/2020] [Accepted: 07/07/2020] [Indexed: 05/07/2023]
Abstract
Shortly after deep learning algorithms were applied to Image Analysis, and more importantly to medical imaging, their applications increased significantly to become a trend. Likewise, deep learning applications (DL) on pulmonary medical images emerged to achieve remarkable advances leading to promising clinical trials. Yet, coronavirus can be the real trigger to open the route for fast integration of DL in hospitals and medical centers. This paper reviews the development of deep learning applications in medical image analysis targeting pulmonary imaging and giving insights of contributions to COVID-19. It covers more than 160 contributions and surveys in this field, all issued between February 2017 and May 2020 inclusively, highlighting various deep learning tasks such as classification, segmentation, and detection, as well as different pulmonary pathologies like airway diseases, lung cancer, COVID-19 and other infections. It summarizes and discusses the current state-of-the-art approaches in this research domain, highlighting the challenges, especially with COVID-19 pandemic current situation.
Collapse
Affiliation(s)
- Hanan Farhat
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| | - George E. Sakr
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| | - Rima Kilany
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| |
Collapse
|
3
|
Zhang G, Yang Z, Gong L, Jiang S, Wang L, Zhang H. Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations. Radiol Med 2020; 125:374-383. [PMID: 31916105 DOI: 10.1007/s11547-019-01130-9] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2019] [Accepted: 12/27/2019] [Indexed: 12/19/2022]
Abstract
Lung cancer is pointed as a leading cause of cancer death worldwide. Early lung nodule diagnosis has great significance for treating lung cancer and increasing patient survival. In this paper, we present a novel method to classify the malignant from benign lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations (SE-ResNeXt). The state-of-the-art SE-ResNeXt module, which integrates the advantages of SENet for feature recalibration and ResNeXt for feature reuse, has great ability in boosting feature discriminability on imaging pattern recognition. The method is evaluated on the public available LUng Nodule Analysis 2016 (LUNA16) database with 1004 (450 malignant and 554 benign) nodules, achieving an area under the receiver operating characteristic curve (AUC) of 0. 9563 and accuracy of 91.67%. The promising results demonstrate that our method has strong robustness in the classification of nodules. The method has the potential to help radiologists better interpret diagnostic data and differentiate the benign from malignant lung nodules on CT images in clinical practice. To our best knowledge, the effectiveness of SE-ResNeXt on lung nodule classification has not been extensively explored.
Collapse
Affiliation(s)
- Guobin Zhang
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China
| | - Zhiyong Yang
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China
| | - Li Gong
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China
| | - Shan Jiang
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China. .,Centre for Advanced Mechanisms and Robotics, Tianjin University, 135 Yaguan Road, Jinnan District, Tianjin, 300350, China.
| | - Lu Wang
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China
| | - Hongyun Zhang
- School of Mechanical Engineering, Tianjin University, Tianjin, 300350, China
| |
Collapse
|
4
|
Fujiwara K, Fang W, Okino T, Sutherland K, Furusaki A, Sagawa A, Kamishima T. Quick and accurate selection of hand images among radiographs from various body parts using deep learning. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2020; 28:1199-1206. [PMID: 32925161 DOI: 10.3233/xst-200694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
BACKGROUND Although rheumatoid arthritis (RA) causes destruction of articular cartilage, early treatment significantly improves symptoms and delays progression. It is important to detect subtle damage for an early diagnosis. Recent software programs are comparable with the conventional human scoring method regarding detectability of the radiographic progression of RA. Thus, automatic and accurate selection of relevant images (e.g. hand images) among radiographic images of various body parts is necessary for serial analysis on a large scale. OBJECTIVE In this study we examined whether deep learning can select target images from a large number of stored images retrieved from a picture archiving and communication system (PACS) including miscellaneous body parts of patients. METHODS We selected 1,047 X-ray images including various body parts and divided them into two groups: 841 images for training and 206 images for testing. The training images were augmented and used to train a convolutional neural network (CNN) consisting of 4 convolution layers, 2 pooling layers and 2 fully connected layers. After training, we created software to classify the test images and examined the accuracy. RESULTS The image extraction accuracy was 0.952 and 0.979 for unilateral hand and both hands, respectively. In addition, all 206 test images were perfectly classified into unilateral hand, both hands, and the others. CONCLUSIONS Deep learning showed promise to enable efficiently automatic selection of target X-ray images of RA patients.
Collapse
Affiliation(s)
- Kohei Fujiwara
- Department of Health Sciences, Hokkaido University, Kita-ku, Sapporo, Japan
| | - Wanxuan Fang
- Faculty of Health Sciences, Hokkaido University, Kita-ku, Sapporo, Japan
| | - Taichi Okino
- Graduate School of Health Sciences, Hokkaido University, Kita-ku, Sapporo, Japan
| | - Kenneth Sutherland
- Global Center for Biomedical Science and Engineering, Hokkaido University, Kita-ku, Sapporo, Japan
| | - Akira Furusaki
- Sagawa Akira Rheumatology Clinic, Chuo-ku, Sapporo, Japan
| | - Akira Sagawa
- Sagawa Akira Rheumatology Clinic, Chuo-ku, Sapporo, Japan
| | - Tamotsu Kamishima
- Faculty of Health Sciences, Hokkaido University, Kita-ku, Sapporo, Japan
| |
Collapse
|
5
|
Al-Shabi M, Lan BL, Chan WY, Ng KH, Tan M. Lung nodule classification using deep Local-Global networks. Int J Comput Assist Radiol Surg 2019; 14:1815-1819. [PMID: 31020576 DOI: 10.1007/s11548-019-01981-7] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2019] [Accepted: 04/16/2019] [Indexed: 11/28/2022]
Abstract
PURPOSE Lung nodules have very diverse shapes and sizes, which makes classifying them as benign/malignant a challenging problem. In this paper, we propose a novel method to predict the malignancy of nodules that have the capability to analyze the shape and size of a nodule using a global feature extractor, as well as the density and structure of the nodule using a local feature extractor. METHODS We propose to use Residual Blocks with a 3 × 3 kernel size for local feature extraction and Non-Local Blocks to extract the global features. The Non-Local Block has the ability to extract global features without using a huge number of parameters. The key idea behind the Non-Local Block is to apply matrix multiplications between features on the same feature maps. RESULTS We trained and validated the proposed method on the LIDC-IDRI dataset which contains 1018 computed tomography scans. We followed a rigorous procedure for experimental setup, namely tenfold cross-validation, and ignored the nodules that had been annotated by < 3 radiologists. The proposed method achieved state-of-the-art results with AUC = 95.62%, while significantly outperforming other baseline methods. CONCLUSIONS Our proposed deep Local-Global network has the capability to accurately extract both local and global features. Our new method outperforms state-of-the-art architecture including Densenet and Resnet with transfer learning.
Collapse
Affiliation(s)
- Mundher Al-Shabi
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia.
| | - Boon Leong Lan
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia
| | - Wai Yee Chan
- Department of Biomedical Imaging, University of Malaya, 50603, Kuala Lumpur, Malaysia
| | - Kwan-Hoong Ng
- Department of Biomedical Imaging, University of Malaya, 50603, Kuala Lumpur, Malaysia
| | - Maxine Tan
- Electrical and Computer Systems Engineering Discipline, School of Engineering, Monash University Malaysia, 47500, Bandar Sunway, Selangor, Malaysia.,School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK, 73019, USA
| |
Collapse
|