Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network

Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging

BACKGROUND AND OBJECTIVE

Efficient segmentation of skin lesion in dermoscopy images can improve the classification accuracy of skin diseases, which provides a powerful approach for the dermatologists in examining pigmented skin lesions. However, the segmentation is challenging due to the low contrast of skin lesions from a captured image, fuzzy and indistinct lesion boundaries, huge variety of interclass variation of melanomas, the existence of artifacts, etc. In this work, an efficient and accurate melanoma region segmentation method is proposed for computer-aided diagnostic systems.

METHOD

A skin lesion segmentation (SLS) method based on the separable-Unet with stochastic weight averaging is proposed in this work. Specifically, the proposed Separable-Unet framework takes advantage of the separable convolutional block and U-Net architectures, which can extremely capture the context feature channel correlation and higher semantic feature information to enhance the pixel-level discriminative representation capability of fully convolutional networks (FCN). Further, considering that the over-fitting is a local optimum (or sub-optimum) problem, a scheme based on stochastic weight averaging is introduced, which can obtain much broader optimum and better generalization.

RESULTS

The proposed method is evaluated in three publicly available datasets. The experimental results showed that the proposed approach segmented the skin lesions with an average Dice coefficient of 93.03% and Jaccard index of 89.25% for the International Skin Imaging Collaboration (ISIC) 2016 Skin Lesion Challenge (SLC) dataset, 86.93% and 79.26% for the ISIC 2017 SLC, and 94.13% and 89.40% for the PH2 dataset, respectively. The proposed approach is compared with other state-of-the-art methods, and the results demonstrate that the proposed approach outperforms them for SLS on both melanoma and non-melanoma cases. Segmentation of a potential lesion with the proposed approach in a dermoscopy image requires less than 0.05 s of processing time, which is roughly 30 times faster than the second best method (regarding the value of Jaccard index) for the ISIC 2017 dataset with the same hardware configuration.

CONCLUSIONS

We concluded that using the separable convolutional block and U-Net architectures with stochastic weight averaging strategy could enable to obtain better pixel-level discriminative representation capability. Moreover, the considerably decreased computation time suggests that the proposed approach has potential for practical computer-aided diagnose systems, besides provides a segmentation for the specific analysis with improved segmentation performance.

Topological correction of infant white matter surfaces using anatomically constrained convolutional neural network

Reconstruction of accurate cortical surfaces without topological errors (i.e., handles and holes) from infant brain MR images is very important in early brain development studies. However, infant brain MR images typically suffer extremely low tissue contrast and dynamic imaging appearance patterns. Thus, it is inevitable to have large amounts of topological errors in the segmented infant brain tissue images, which lead to inaccurately reconstructed cortical surfaces with topological errors. To address this issue, inspired by recent advances in deep learning, we propose an anatomically constrained network for topological correction on infant cortical surfaces. Specifically, in our method, we first locate regions of potential topological defects by leveraging a topology-preserving level set method. Then, we propose an anatomically constrained network to correct those candidate voxels in the located regions. Since infant cortical surfaces often contain large and complex handles or holes, it is difficult to completely correct all errors using one-shot correction. Therefore, we further enroll these two steps into an iterative framework to gradually correct large topological errors. To the best of our knowledge, this is the first work to introduce deep learning approach for topological correction of infant cortical surfaces. We compare our method with the state-of-the-art methods on both simulated topological errors and real topological errors in human infant brain MR images. Moreover, we also validate our method on the infant brain MR images of macaques. All experimental results show the superior performance of the proposed method.

Using deep convolutional neural networks for multi-classification of thyroid tumor by histopathology: a large-scale pilot study

Background

To explore whether deep convolutional neural networks (DCNNs) have the potential to improve diagnostic efficiency and increase the level of interobserver agreement in the classification of thyroid nodules in histopathological slides.

Methods

A total of 11,715 fragmented images from 806 patients' original histological images were divided into a training dataset and a test dataset. Inception-ResNet-v2 and VGG-19 were trained using the training dataset and tested using the test dataset to determine the diagnostic efficiencies of different histologic types of thyroid nodules, including normal tissue, adenoma, nodular goiter, papillary thyroid carcinoma (PTC), follicular thyroid carcinoma (FTC), medullary thyroid carcinoma (MTC) and anaplastic thyroid carcinoma (ATC). Misdiagnoses were further analyzed.

Results

The total 11,715 fragmented images were divided into a training dataset and a test dataset for each pathology type at a ratio of 5:1. Using the test set, VGG-19 yielded a better average diagnostic accuracy than did Inception-ResNet-v2 (97.34% vs. 94.42%, respectively). The VGG-19 model applied to 7 pathology types showed a fragmentation accuracy of 88.33% for normal tissue, 98.57% for ATC, 98.89% for FTC, 100% for MTC, 97.77% for PTC, 100% for nodular goiter and 92.44% for adenoma. It achieved excellent diagnostic efficiencies for all the malignant types. Normal tissue and adenoma were the most challenging histological types to classify.

Conclusions

The DCNN models, especially VGG-19, achieved satisfactory accuracies on the task of differentiating thyroid tumors by histopathology. Analysis of the misdiagnosed cases revealed that normal tissue and adenoma were the most challenging histological types for the DCNN to differentiate, while all the malignant classifications achieved excellent diagnostic efficiencies. The results indicate that DCNN models may have potential for facilitating histopathologic thyroid disease diagnosis.

Deep convolutional neural network VGG-16 model for differential diagnosing of papillary thyroid carcinomas in cytological images: a pilot study

Objective: In this study, we exploited a VGG-16 deep convolutional neural network (DCNN) model to differentiate papillary thyroid carcinoma (PTC) from benign thyroid nodules using cytological images. Methods: A pathology-proven dataset was built from 279 cytological images of thyroid nodules. The images were cropped into fragmented images and divided into a training dataset and a test dataset. VGG-16 and Inception-v3 DCNNs were trained and tested to make differential diagnoses. The characteristics of tumor cell nucleus were quantified as contours, perimeter, area and mean of pixel intensity and compared using independent Student's t-tests. Results: In the test group, the accuracy rates of the VGG-16 model and Inception-v3 on fragmented images were 97.66% and 92.75%, respectively, and the accuracy rates of VGG-16 and Inception-v3 in patients were 95% and 87.5%, respectively. The contours, perimeter, area and mean of pixel intensity of PTC in fragmented images were more than the benign nodules, which were 61.01±17.10 vs 47.00±24.08, p=0.000, 134.99±21.42 vs 62.40±29.15, p=0.000, 1770.89±627.22 vs 1157.27±722.23, p=0.013, 165.84±26.33 vs 132.94±28.73, p=0.000), respectively. Conclusion: In summary, after training with a large dataset, the DCNN VGG-16 model showed great potential in facilitating PTC diagnosis from cytological images. The contours, perimeter, area and mean of pixel intensity of PTC in fragmented images were more than the benign nodules.

Attention to Lesion: Lesion-Aware Convolutional Neural Network for Retinal Optical Coherence Tomography Image Classification

Automatic and accurate classification of retinal optical coherence tomography (OCT) images is essential to assist ophthalmologist in the diagnosis and grading of macular diseases. Clinically, ophthalmologists usually diagnose macular diseases according to the structures of macular lesions, whose morphologies, size, and numbers are important criteria. In this paper, we propose a novel lesion-aware convolutional neural network (LACNN) method for retinal OCT image classification, in which retinal lesions within OCT images are utilized to guide the CNN to achieve more accurate classification. The LACNN simulates the ophthalmologists' diagnosis that focuses on local lesion-related regions when analyzing the OCT image. Specifically, we first design a lesion detection network to generate a soft attention map from the whole OCT image. The attention map is then incorporated into a classification network to weight the contributions of local convolutional representations. Guided by the lesion attention map, the classification network can utilize the information from local lesion-related regions to further accelerate the network training process and improve the OCT classification. Our experimental results on two clinically acquired OCT datasets demonstrate the effectiveness and efficiency of the proposed LACNN method for retinal OCT image classification.

Deep-learning-powered photonic analog-to-digital conversion

Analog-to-digital converters (ADCs) must be high speed, broadband, and accurate for the development of modern information systems, such as radar, imaging, and communications systems; photonic technologies are regarded as promising technologies for realizing these advanced requirements. Here, we present a deep-learning-powered photonic ADC architecture that simultaneously exploits the advantages of electronics and photonics and overcomes the bottlenecks of the two technologies, thereby overcoming the ADC tradeoff among speed, bandwidth, and accuracy. Via supervised training, the adopted deep neural networks learn the patterns of photonic system defects and recover the distorted data, thereby maintaining the high quality of the electronic quantized data succinctly and adaptively. The numerical and experimental results demonstrate that the proposed architecture outperforms state-of-the-art ADCs with developable high throughput; hence, deep learning performs well in photonic ADC systems. We anticipate that the proposed architecture will inspire future high-performance photonic ADC design and provide opportunities for substantial performance enhancement for the next-generation information systems.

Prognostic Value of Deep Learning PET/CT-Based Radiomics: Potential Role for Future Individual Induction Chemotherapy in Advanced Nasopharyngeal Carcinoma

PURPOSE

We aimed to evaluate the value of deep learning on positron emission tomography with computed tomography (PET/CT)-based radiomics for individual induction chemotherapy (IC) in advanced nasopharyngeal carcinoma (NPC).

EXPERIMENTAL DESIGN

We constructed radiomics signatures and nomogram for predicting disease-free survival (DFS) based on the extracted features from PET and CT images in a training set (n = 470), and then validated it on a test set (n = 237). Harrell's concordance indices (C-index) and time-independent receiver operating characteristic (ROC) analysis were applied to evaluate the discriminatory ability of radiomics nomogram, and compare radiomics signatures with plasma Epstein-Barr virus (EBV) DNA.

RESULTS

A total of 18 features were selected to construct CT-based and PET-based signatures, which were significantly associated with DFS (P < 0.001). Using these signatures, we proposed a radiomics nomogram with a C-index of 0.754 [95% confidence interval (95% CI), 0.709-0.800] in the training set and 0.722 (95% CI, 0.652-0.792) in the test set. Consequently, 206 (29.1%) patients were stratified as high-risk group and the other 501 (70.9%) as low-risk group by the radiomics nomogram, and the corresponding 5-year DFS rates were 50.1% and 87.6%, respectively (P < 0.0001). High-risk patients could benefit from IC while the low-risk could not. Moreover, radiomics nomogram performed significantly better than the EBV DNA-based model (C-index: 0.754 vs. 0.675 in the training set and 0.722 vs. 0.671 in the test set) in risk stratification and guiding IC.

CONCLUSIONS

Deep learning PET/CT-based radiomics could serve as a reliable and powerful tool for prognosis prediction and may act as a potential indicator for individual IC in advanced NPC.

High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays

Optical neural networks (ONNs) have become competitive candidates for the next generation of high-performance neural network accelerators because of their low power consumption and high-speed nature. Beyond fully-connected neural networks demonstrated in pioneer works, optical computing hardwares can also conduct convolutional neural networks (CNNs) by hardware reusing. Following this concept, we propose an optical convolution unit (OCU) architecture. By reusing the OCU architecture with different inputs and weights, convolutions with arbitrary input sizes can be done. A proof-of-concept experiment is carried out by cascaded acousto-optical modulator arrays. When the neural network parameters are ex-situ trained, the OCU conducts convolutions with SDR up to 28.22 dBc and performs well on inferences of typical CNN tasks. Furthermore, we conduct in situ training and get higher SDR at 36.27 dBc, verifying the OCU could be further refined by in situ training. Besides the effectiveness and high accuracy, the simplified OCU architecture served as a building block could be easily duplicated and integrated to future chip-scale optical CNNs.

Quantitative CT texture analysis for diagnosing systemic sclerosis: Effect of iterative reconstructions and radiation doses

To test whether texture analysis (TA) can discriminate between Systemic Sclerosis (SSc) and non-SSc patients in computed tomography (CT) with different radiation doses and reconstruction algorithms.In this IRB-approved retrospective study, 85 CT scans at different radiation doses [49 standard dose CT (SDCT) with a volume CT dose index (CTDIvol) of 4.86 ± 2.1 mGy and 36 low-dose (LDCT) with a CTDIvol of 2.5 ± 1.5 mGy] were selected; 61 patients had Ssc ("cases"), and 24 patients had no SSc ("controls"). CT scans were reconstructed with filtered-back projection (FBP) and with sinogram-affirmed iterative reconstruction (SAFIRE) algorithms. 304 TA features were extracted from each manually drawn region-of-interest at 6 pre-defined levels: at the midpoint between lung apices and tracheal carina, at the level of the tracheal carina, and 4 between the carina and pleural recesses. Each TA feature was averaged between these 6 pre-defined levels and was used as input in the machine learning algorithm artificial neural network (ANN) with backpropagation (MultilayerPerceptron) for differentiating between SSc and non-SSc patients.Results were compared regarding correctly/incorrectly classified instances and ROC-AUCs.ANN correctly classified individuals in 93.8% (AUC = 0.981) of FBP-LDCT, in 78.5% (AUC = 0.859) of FBP-SDCT, in 91.1% (AUC = 0.922) of SAFIRE3-LDCT and 75.7% (AUC = 0.815) of SAFIRE3-SDCT, in 88.1% (AUC = 0.929) of SAFIRE5-LDCT and 74% (AUC = 0.815) of SAFIRE5-SDCT.Quantitative TA-based discrimination of CT of SSc patients is possible showing highest discriminatory power in FBP-LDCT images.

Analysis of Collagen Spatial Structure Using Multiphoton Microscopy and Machine Learning Methods

Pathogenesis of many diseases is associated with changes in the collagen spatial structure. Traditionally, the 3D structure of collagen in biological tissues is analyzed using histochemistry, immunohistochemistry, magnetic resonance imaging, and X-radiography. At present, multiphoton microscopy (MPM) is commonly used to study the structure of biological tissues. MPM has a high spatial resolution comparable to histological analysis and can be used for direct visualization of collagen spatial structure. Because of a large volume of data accumulated due to the high spatial resolution of MPM, special analytical methods should be used for identification of informative features in the images and quantitative evaluation of relationship between these features and pathological processes resulting in the destruction of collagen structure. Here, we describe current approaches and achievements in the identification of informative features in the MPM images of collagen in biological tissues, as well as the development on this basis of algorithms for computer-aided classification of collagen structures using machine learning as a type of artificial intelligence methods.

Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FS), multi-branch (MB) techniques, and multichannel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0:05) than the fusion baselines (FS: 99.00%, MB: 99.08%, TC: 98.92%) and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.

SDFN: Segmentation-based deep fusion network for thoracic disease classification in chest X-ray images

This study aims to automatically diagnose thoracic diseases depicted on the chest x-ray (CXR) images using deep convolutional neural networks. The existing methods generally used the entire CXR images for training purposes, but this strategy may suffer from two drawbacks. First, potential misalignment or the existence of irrelevant objects in the entire CXR images may cause unnecessary noise and thus limit the network performance. Second, the relatively low image resolution caused by the resizing operation, which is a common pre-processing procedure for training neural networks, may lead to the loss of image details, making it difficult to detect pathologies with small lesion regions. To address these issues, we present a novel method termed as segmentation-based deep fusion network (SDFN), which leverages the domain knowledge and the higher-resolution information of local lung regions. Specifically, the local lung regions were identified and cropped by the Lung Region Generator (LRG). Two CNN-based classification models were then used as feature extractors to obtain the discriminative features of the entire CXR images and the cropped lung region images. Lastly, the obtained features were fused by the feature fusion module for disease classification. Evaluated by the NIH benchmark split on the Chest X-ray 14 Dataset, our experimental result demonstrated that the developed method achieved more accurate disease classification compared with the available approaches via the receiver operating characteristic (ROC) analyses. It was also found that the SDFN could localize the lesion regions more precisely as compared to the traditional method.

The Use of Deep Learning to Predict Stroke Patient Mortality

The increase in stroke incidence with the aging of the Korean population will rapidly impose an economic burden on society. Timely treatment can improve stroke prognosis. Awareness of stroke warning signs and appropriate actions in the event of a stroke improve outcomes. Medical service use and health behavior data are easier to collect than medical imaging data. Here, we used a deep neural network to detect stroke using medical service use and health behavior data; we identified 15,099 patients with stroke. Principal component analysis (PCA) featuring quantile scaling was used to extract relevant background features from medical records; we used these to predict stroke. We compared our method (a scaled PCA/deep neural network [DNN] approach) to five other machine-learning methods. The area under the curve (AUC) value of our method was 83.48%; hence; it can be used by both patients and doctors to prescreen for possible stroke.

Prediction of local relapse and distant metastasis in patients with definitive chemoradiotherapy-treated cervical cancer by deep learning from [18F]-fluorodeoxyglucose positron emission tomography/computed tomography

BACKGROUND

We designed a deep learning model for assessing ¹⁸F-FDG PET/CT for early prediction of local and distant failures for patients with locally advanced cervical cancer.

METHODS

All 142 patients with cervical cancer underwent ¹⁸F-FDG PET/CT for pretreatment staging and received allocated treatment. To augment the amount of image data, each tumor was represented as 11 slice sets each of which contains 3 2D orthogonal slices to acquire a total of 1562 slice sets. In each round of k-fold cross-validation, a well-trained proposed model and a slice-based optimal threshold were derived from a training set and used to classify each slice set in the test set into the categories of with or without local or distant failure. The classification results of each tumor were aggregated to summarize a tumor-based prediction result.

RESULTS

In total, 21 and 26 patients experienced local and distant failures, respectively. Regarding local recurrence, the tumor-based prediction result summarized from all test sets demonstrated that the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 71%, 93%, 63%, 95%, and 89%, respectively. The corresponding values for distant metastasis were 77%, 90%, 63%, 95%, and 87%, respectively.

CONCLUSION

This is the first study to use deep learning model for assessing ¹⁸F-FDG PET/CT images which is capable of predicting treatment outcomes in cervical cancer patients.

KEY POINTS

• This is the first study to use deep learning model for assessing ¹⁸ F-FDG PET/CT images which is capable of predicting treatment outcomes in cervical cancer patients. • All 142 patients with cervical cancer underwent ¹⁸ F-FDG PET/CT for pretreatment staging and received allocated treatment. To augment the amount of image data, each tumor was represented as 11 slice sets each of which contains 3 2D orthogonal slices to acquire a total of 1562 slice sets. • For local recurrence, all test sets demonstrated that the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 71%, 93%, 63%, 95%, and 89%, respectively. The corresponding values for distant metastasis were 77%, 90%, 63%, 95%, and 87%, respectively.

Comparison of convolutional neural networks for landslide susceptibility mapping in Yanshan County, China

Assessments of landslide disasters are becoming increasingly urgent. The aim of this study is to investigate a convolutional neural network (CNN) framework for landslide susceptibility mapping (LSM) in Yanshan County, China. The two primary contributions of this study are summarized as follows. First, to the best of our knowledge, this report describes the first time that the CNN framework is used for LSM. Second, different data representation algorithms are developed to construct three novel CNN architectures. In this work, sixteen influencing factors associated with landslide occurrence were considered and historical landslide locations were randomly divided into training (70% of the total) and validation (30%) sets. Validation of these CNNs was performed using different commonly used measures in comparison to several of the most popular machine learning and deep learning methods. The experimental results demonstrated that the proportions of highly susceptible zones in all of the CNN landslide susceptibility maps are highly similar and lower than 30%, which indicates that these CNNs are more practical for landslide prevention and management than conventional methods. Furthermore, the proposed CNN framework achieved higher or comparable prediction accuracy. Specifically, the proposed CNNs were 3.94%-7.45% and 0.079-0.151 higher than those of the optimized support vector machine (SVM) in terms of overall accuracy (OA) and Matthews correlation coefficient (MCC), respectively.

Deep learning for liver tumor diagnosis part II: convolutional neural network interpretation using radiologic imaging features

OBJECTIVES

To develop a proof-of-concept "interpretable" deep learning prototype that justifies aspects of its predictions from a pre-trained hepatic lesion classifier.

METHODS

A convolutional neural network (CNN) was engineered and trained to classify six hepatic tumor entities using 494 lesions on multi-phasic MRI, described in Part 1. A subset of each lesion class was labeled with up to four key imaging features per lesion. A post hoc algorithm inferred the presence of these features in a test set of 60 lesions by analyzing activation patterns of the pre-trained CNN model. Feature maps were generated that highlight regions in the original image that correspond to particular features. Additionally, relevance scores were assigned to each identified feature, denoting the relative contribution of a feature to the predicted lesion classification.

RESULTS

The interpretable deep learning system achieved 76.5% positive predictive value and 82.9% sensitivity in identifying the correct radiological features present in each test lesion. The model misclassified 12% of lesions. Incorrect features were found more often in misclassified lesions than correctly identified lesions (60.4% vs. 85.6%). Feature maps were consistent with original image voxels contributing to each imaging feature. Feature relevance scores tended to reflect the most prominent imaging criteria for each class.

CONCLUSIONS

This interpretable deep learning system demonstrates proof of principle for illuminating portions of a pre-trained deep neural network's decision-making, by analyzing inner layers and automatically describing features contributing to predictions.

KEY POINTS

• An interpretable deep learning system prototype can explain aspects of its decision-making by identifying relevant imaging features and showing where these features are found on an image, facilitating clinical translation. • By providing feedback on the importance of various radiological features in performing differential diagnosis, interpretable deep learning systems have the potential to interface with standardized reporting systems such as LI-RADS, validating ancillary features and improving clinical practicality. • An interpretable deep learning system could potentially add quantitative data to radiologic reports and serve radiologists with evidence-based decision support.

Label-free optical hemogram of granulocytes enhanced by artificial neural networks

An outstanding challenge for immunology is the classification of immune cells in a label-free fashion with high speed. For this purpose, optical techniques such as Raman spectroscopy or digital holographic microscopy have been used successfully to identify immune cell subsets. To achieve high accuracy, these techniques require a post-processing step using linear methods of multivariate processing, such as principal component analysis. Here we demonstrate for the first time a comparison between artificial neural networks and principal component analysis (PCA) to classify the key granulocyte cell lineages of neutrophils and eosinophils using both digital holographic microscopy and Raman spectroscopy. Artificial neural networks can offer advantages in terms of classification accuracy and speed over a PCA approach. We conclude that digital holographic microscopy with convolutional neural networks based analysis provides a route to a robust, stand-alone and high-throughput hemogram with a classification accuracy of 91.3 % at a throughput rate of greater than 100 cells per second.

Automated segmentation of macular edema in OCT using deep neural networks

Macular edema is an eye disease that can affect visual acuity. Typical disease symptoms include subretinal fluid (SRF) and pigment epithelium detachment (PED). Optical coherence tomography (OCT) has been widely used for diagnosing macular edema because of its non-invasive and high resolution properties. Segmentation for macular edema lesions from OCT images plays an important role in clinical diagnosis. Many computer-aided systems have been proposed for the segmentation. Most traditional segmentation methods used in these systems are based on low-level hand-crafted features, which require significant domain knowledge and are sensitive to the variations of lesions. To overcome these shortcomings, this paper proposes to use deep neural networks (DNNs) together with atrous spatial pyramid pooling (ASPP) to automatically segment the SRF and PED lesions. Lesions-related features are first extracted by DNNs, then processed by ASPP which is composed of multiple atrous convolutions with different fields of view to accommodate the various scales of the lesions. Based on ASPP, a novel module called stochastic ASPP (sASPP) is proposed to combat the co-adaptation of multiple atrous convolutions. A large OCT dataset provided by a competition platform called "AI Challenger" are used to train and evaluate the proposed model. Experimental results demonstrate that the DNNs together with ASPP achieve higher segmentation accuracy compared with the state-of-the-art method. The stochastic operation added in sASPP is empirically verified as an effective regularization method that can alleviate the overfitting problem and significantly reduce the validation error.

Improving the Cognitive Agent Intelligence by Deep Knowledge Classification

In this paper, a new idea is developed for improving the agent intelligence. In fact with the presented convolutional neural network (CNN) approach for knowledge classification, the agent will be able to manage its knowledge. This new concept allows the agent to select only the actionable rule class, instead of trying to infer its whole rule base exhaustively. In addition, through this research, we developed a comparative study between the proposed CNN approach and the classical classification approaches. As foreseeable the deep learning method outperforms the others in term of classification accuracy.

Fully Automated Diagnosis of Anterior Cruciate Ligament Tears on Knee MR Images by Using Deep Learning

Purpose

To investigate the feasibility of using a deep learning-based approach to detect an anterior cruciate ligament (ACL) tear within the knee joint at MRI by using arthroscopy as the reference standard.

Materials and Methods

A fully automated deep learning-based diagnosis system was developed by using two deep convolutional neural networks (CNNs) to isolate the ACL on MR images followed by a classification CNN to detect structural abnormalities within the isolated ligament. With institutional review board approval, sagittal proton density-weighted and fat-suppressed T2-weighted fast spin-echo MR images of the knee in 175 subjects with a full-thickness ACL tear (98 male subjects and 77 female subjects; average age, 27.5 years) and 175 subjects with an intact ACL (100 male subjects and 75 female subjects; average age, 39.4 years) were retrospectively analyzed by using the deep learning approach. Sensitivity and specificity of the ACL tear detection system and five clinical radiologists for detecting an ACL tear were determined by using arthroscopic results as the reference standard. Receiver operating characteristic (ROC) analysis and two-sided exact binomial tests were used to further assess diagnostic performance.

Results

The sensitivity and specificity of the ACL tear detection system at the optimal threshold were 0.96 and 0.96, respectively. In comparison, the sensitivity of the clinical radiologists ranged between 0.96 and 0.98, while the specificity ranged between 0.90 and 0.98. There was no statistically significant difference in diagnostic performance between the ACL tear detection system and clinical radiologists at P < .05. The area under the ROC curve for the ACL tear detection system was 0.98, indicating high overall diagnostic accuracy.

Conclusion

There was no significant difference between the diagnostic performance of the ACL tear detection system and clinical radiologists for determining the presence or absence of an ACL tear at MRI.© RSNA, 2019Supplemental material is available for this article.

Pattern Classification for Gastrointestinal Stromal Tumors by Integration of Radiomics and Deep Convolutional Features

Predicting malignant potential is one of the most critical components of a computer-aided diagnosis system for gastrointestinal stromal tumors (GISTs). These tumors have been studied only on the basis of subjective computed tomography findings. Among various methodologies, radiomics, and deep learning algorithms, specifically convolutional neural networks (CNNs), have recently been confirmed to achieve significant success by outperforming the state-of-the-art performance in medical image pattern classification and have rapidly become leading methodologies in this field. However, the existing methods generally use radiomics or deep convolutional features independently for pattern classification, which tend to take into account only global or local features, respectively. In this paper, we introduce and evaluate a hybrid structure that includes different features selected with radiomics model and CNNs and integrates these features to deal with GISTs classification. The Radiomics model and CNNs are constructed for global radiomics and local convolutional feature selection, respectively. Subsequently, we utilize distinct radiomics and deep convolutional features to perform pattern classification for GISTs. Specifically, we propose a new pooling strategy to assemble the deep convolutional features of 54 three-dimensional patches from the same case and integrate these features with the radiomics features for independent case, followed by random forest classifier. Our method can be extensively evaluated using multiple clinical datasets. The classification performance (area under the curve (AUC): 0.882; 95% confidence interval (CI): 0.816-0.947) consistently outperforms those of independent radiomics (AUC: 0.807; 95% CI: 0.724-0.892) and CNNs (AUC: 0.826; 95% CI: 0.795-0.856) approaches.

Artificial Intelligence in Cardiothoracic Radiology

OBJECTIVE. The goal of this article is to examine some of the current cardiothoracic radiology applications of artificial intelligence in general and deep learning in particular. CONCLUSION. Artificial intelligence has been used for the analysis of medical images for decades. Recent advances in computer algorithms and hardware, coupled with the availability of larger labeled datasets, have brought about rapid advances in this field. Many of the more notable recent advances have been in the artificial intelligence subfield of deep learning.

Radiological images and machine learning: Trends, perspectives, and prospects

The application of machine learning to radiological images is an increasingly active research area that is expected to grow in the next five to ten years. Recent advances in machine learning have the potential to recognize and classify complex patterns from different radiological imaging modalities such as x-rays, computed tomography, magnetic resonance imaging and positron emission tomography imaging. In many applications, machine learning based systems have shown comparable performance to human decision-making. The applications of machine learning are the key ingredients of future clinical decision making and monitoring systems. This review covers the fundamental concepts behind various machine learning techniques and their applications in several radiological imaging areas, such as medical image segmentation, brain function studies and neurological disease diagnosis, as well as computer-aided systems, image registration, and content-based image retrieval systems. Synchronistically, we will briefly discuss current challenges and future directions regarding the application of machine learning in radiological imaging. By giving insight on how take advantage of machine learning powered applications, we expect that clinicians can prevent and diagnose diseases more accurately and efficiently.

Decision Fusion-Based Fetal Ultrasound Image Plane Classification Using Convolutional Neural Networks

Machine learning for ultrasound image analysis and interpretation can be helpful in automated image classification in large-scale retrospective analyses to objectively derive new indicators of abnormal fetal development that are embedded in ultrasound images. Current approaches to automatic classification are limited to the use of either image patches (cropped images) or the global (whole) image. As many fetal organs have similar visual features, cropped images can misclassify certain structures such as the kidneys and abdomen. Also, the whole image does not encode sufficient local information about structures to identify different structures in different locations. Here we propose a method to automatically classify 14 different fetal structures in 2-D fetal ultrasound images by fusing information from both cropped regions of fetal structures and the whole image. Our method trains two feature extractors by fine-tuning pre-trained convolutional neural networks with the whole ultrasound fetal images and the discriminant regions of the fetal structures found in the whole image. The novelty of our method is in integrating the classification decisions made from the global and local features without relying on priors. In addition, our method can use the classification outcome to localize the fetal structures in the image. Our experiments on a data set of 4074 2-D ultrasound images (training: 3109, test: 965) achieved a mean accuracy of 97.05%, mean precision of 76.47% and mean recall of 75.41%. The Cohen κ of 0.72 revealed the highest agreement between the ground truth and the proposed method. The superiority of the proposed method over the other non-fusion-based methods is statistically significant (p < 0.05). We found that our method is capable of predicting images without ultrasound scanner overlays with a mean accuracy of 92%. The proposed method can be leveraged to retrospectively classify any ultrasound images in clinical research.

M-ary Rank Classifier Combination: A Binary Linear Programming Problem

The goal of classifier combination can be briefly stated as combining the decisions of individual classifiers to obtain a better classifier. In this paper, we propose a method based on the combination of weak rank classifiers because rankings contain more information than unique choices for a many-class problem. The problem of combining the decisions of more than one classifier with raw outputs in the form of candidate class rankings is considered and formulated as a general discrete optimization problem with an objective function based on the distance between the data and the consensus decision. This formulation uses certain performance statistics about the joint behavior of the ensemble of classifiers. Assuming that each classifier produces a ranking list of classes, an initial approach leads to a binary linear programming problem with a simple and global optimum solution. The consensus function can be considered as a mapping from a set of individual rankings to a combined ranking, leading to the most relevant decision. We also propose an information measure that quantifies the degree of consensus between the classifiers to assess the strength of the combination rule that is used. It is easy to implement and does not require any training. The main conclusion is that the classification rate is strongly improved by combining rank classifiers globally. The proposed algorithm is tested on real cytology image data to detect cervical cancer.

Robust and accurate quantification of biomarkers of immune cells in lung cancer micro-environment using deep convolutional neural networks

Recent years have seen a growing awareness of the role the immune system plays in successful cancer treatment, especially in novel therapies like immunotherapy. The characterization of the immunological composition of tumors and their micro-environment is thus becoming a necessity. In this paper we introduce a deep learning-based immune cell detection and quantification method, which is based on supervised learning, i.e., the input data for training comprises labeled images. Our approach objectively deals with staining variation and staining artifacts in immunohistochemically stained lung cancer tissue and is as precise as humans. This is evidenced by the low cell count difference to humans of 0.033 cells on average. This method, which is based on convolutional neural networks, has the potential to provide a new quantitative basis for research on immunotherapy.

Image-based biomarkers for solid tumor quantification

The last few decades have witnessed tremendous technological developments in image-based biomarkers for tumor quantification and characterization. Initially limited to manual one- and two-dimensional size measurements, image biomarkers have evolved to harness developments not only in image acquisition technology but also in image processing and analysis algorithms. At the same time, clinical validation remains a major challenge for the vast majority of these novel techniques, and there is still a major gap between the latest technological developments and image biomarkers used in everyday clinical practice. Currently, the imaging biomarker field is attracting increasing attention not only because of the tremendous interest in cutting-edge therapeutic developments and personalized medicine but also because of the recent progress in the application of artificial intelligence (AI) algorithms to large-scale datasets. Thus, the goal of the present article is to review the current state of the art for image biomarkers and their use for characterization and predictive quantification of solid tumors. Beginning with an overview of validated imaging biomarkers in current clinical practice, we proceed to a review of AI-based methods for tumor characterization, such as radiomics-based approaches and deep learning.Key Points• Recent years have seen tremendous technological developments in image-based biomarkers for tumor quantification and characterization.• Image-based biomarkers can be used on an ongoing basis, in a non-invasive (or mildly invasive) way, to monitor the development and progression of the disease or its response to therapy.• We review the current state of the art for image biomarkers, as well as the recent developments in artificial intelligence (AI) algorithms for image processing and analysis.

Deep learning-based image restoration algorithm for coronary CT angiography

OBJECTIVES

The purpose of this study was to compare the image quality of coronary computed tomography angiography (CTA) subjected to deep learning-based image restoration (DLR) method with images subjected to hybrid iterative reconstruction (IR).

METHODS

We enrolled 30 patients (22 men, 8 women) who underwent coronary CTA on a 320-slice CT scanner. The images were reconstructed with hybrid IR and with DLR. The image noise in the ascending aorta, left atrium, and septal wall of the ventricle was measured on all images and the contrast-to-noise ratio (CNR) in the proximal coronary arteries was calculated. We also generated CT attenuation profiles across the proximal coronary arteries and measured the width of the edge rise distance (ERD) and the edge rise slope (ERS). Two observers visually evaluated the overall image quality using a 4-point scale (1 = poor, 4 = excellent).

RESULTS

On DLR images, the mean image noise was lower than that on hybrid IR images (18.5 ± 2.8 HU vs. 23.0 ± 4.6 HU, p < 0.01) and the CNR was significantly higher (p < 0.01). The mean ERD was significantly shorter on DLR than on hybrid IR images, whereas the mean ERS was steeper on DLR than on hybrid IR images. The mean image quality score for hybrid IR and DLR images was 2.96 and 3.58, respectively (p < 0.01).

CONCLUSIONS

DLR reduces the image noise and improves the image quality at coronary CTA.

KEY POINTS

• Deep learning-based image restoration is a new technique that employs the deep convolutional neural network for image quality improvement. • Deep learning-based restoration reduces the image noise and improves image quality at coronary CT angiography. • This method may allow for a reduction in radiation exposure.

Effective Diagnosis and Treatment through Content-Based Medical Image Retrieval (CBMIR) by Using Artificial Intelligence

Medical-image-based diagnosis is a tedious task‚ and small lesions in various medical images can be overlooked by medical experts due to the limited attention span of the human visual system, which can adversely affect medical treatment. However, this problem can be resolved by exploring similar cases in the previous medical database through an efficient content-based medical image retrieval (CBMIR) system. In the past few years, heterogeneous medical imaging databases have been growing rapidly with the advent of different types of medical imaging modalities. Recently, a medical doctor usually refers to various types of imaging modalities all together such as computed tomography (CT), magnetic resonance imaging (MRI), X-ray, and ultrasound, etc of various organs in order for the diagnosis and treatment of specific disease. Accurate classification and retrieval of multimodal medical imaging data is the key challenge for the CBMIR system. Most previous attempts use handcrafted features for medical image classification and retrieval, which show low performance for a massive collection of multimodal databases. Although there are a few previous studies on the use of deep features for classification, the number of classes is very small. To solve this problem, we propose the classification-based retrieval system of the multimodal medical images from various types of imaging modalities by using the technique of artificial intelligence, named as an enhanced residual network (ResNet). Experimental results with 12 databases including 50 classes demonstrate that the accuracy and F1.score by our method are respectively 81.51% and 82.42% which are higher than those by the previous method of CBMIR (the accuracy of 69.71% and F1.score of 69.63%).

Identification of Cherry Leaf Disease Infected by Podosphaera Pannosa via Convolutional Neural Network

The cherry leaves infected by Podosphaera pannosa will suffer powdery mildew, which is a serious disease threatening the cherry production industry. In order to identify the diseased cherry leaves in early stage, the authors formulate the cherry leaf disease infected identification as a classification problem and propose a fully automatic identification method based on convolutional neural network (CNN). The GoogLeNet is used as backbone of the CNN. Then, transferred learning techniques are applied to fine-tune the CNN from pre-trained GoogLeNet on ImageNet dataset. This article compares the proposed method against three traditional machine learning methods i.e., support vector machine (SVM), k-nearest neighbor (KNN) and back propagation (BP) neural network. Quantitative evaluations conducted on a data set of 1,200 images collected by smart phones, demonstrates that the CNN achieves best precise performance in identifying diseased cherry leaves, with the testing accuracy of 99.6%. Thus, a CNN can be used effectively in identifying the diseased cherry leaves.

Recent Advances of Deep Learning in Bioinformatics and Computational Biology

Extracting inherent valuable knowledge from omics big data remains as a daunting problem in bioinformatics and computational biology. Deep learning, as an emerging branch from machine learning, has exhibited unprecedented performance in quite a few applications from academia and industry. We highlight the difference and similarity in widely utilized models in deep learning studies, through discussing their basic structures, and reviewing diverse applications and disadvantages. We anticipate the work can serve as a meaningful perspective for further development of its theory, algorithm and application in bioinformatic and computational biology.

Classification of Pulmonary CT Images by Using Hybrid 3D-Deep Convolutional Neural Network Architecture

Lung cancer is the most common cause of cancer-related deaths worldwide. Hence, the survival rate of patients can be increased by early diagnosis. Recently, machine learning methods on Computed Tomography (CT) images have been used in the diagnosis of lung cancer to accelerate the diagnosis process and assist physicians. However, in conventional machine learning techniques, using handcrafted feature extraction methods on CT images are complicated processes. Hence, deep learning as an effective area of machine learning methods by using automatic feature extraction methods could minimize the process of feature extraction. In this study, two Convolutional Neural Network (CNN)-based models were proposed as deep learning methods to diagnose lung cancer on lung CT images. To investigate the performance of the two proposed models (Straight 3D-CNN with conventional softmax and hybrid 3D-CNN with Radial Basis Function (RBF)-based SVM), the altered models of two-well known CNN architectures (3D-AlexNet and 3D-GoogleNet) were considered. Experimental results showed that the performance of the two proposed models surpassed 3D-AlexNet and 3D-GoogleNet. Furthermore, the proposed hybrid 3D-CNN with SVM achieved more satisfying results (91.81%, 88.53% and 91.91% for accuracy rate, sensitivity and precision respectively) compared to straight 3D-CNN with softmax in the diagnosis of lung cancer.

Immunotherapy Associated Pulmonary Toxicity: Biology Behind Clinical and Radiological Features

The broader use of immune checkpoint blockade in clinical routine challenges clinicians in the diagnosis and management of side effects which are caused by inflammation generated by the activation of the immune response. Nearly all organs can be affected by immune-related toxicities. However, the most frequently reported are: fatigue, rash, pruritus, diarrhea, nausea/vomiting, arthralgia, decreased appetite and abdominal pain. Although these adverse events are usually mild, reversible and not frequent, an early diagnosis is crucial. Immune-related pulmonary toxicity was most frequently observed in trials of lung cancer and of melanoma patients treated with the combination of the anti-cytotoxic T lymphocyte antigen (CTLA)-4 and the anti-programmed cell death-1 (PD-1) antibodies. The most frequent immune-related adverse event in the lung is represented by pneumonitis due to the development of infiltrates in the interstitium and in the alveoli. Clinical symptoms and radiological patterns are the key elements to be considered for an early diagnosis, rendering the differential diagnosis crucial. Diagnosis of immune-related pneumonitis may imply the temporary or definitive suspension of immunotherapy, along with the start of immuno-suppressive treatments. The aim of this work is to summarize the biological bases, clinical and radiological findings of lung toxicity under immune checkpoint blockade, underlining the importance of multidisciplinary teams for an optimal early diagnosis of this side effect, with the aim to reach an improved patient care.

The present and future of deep learning in radiology

The advent of Deep Learning (DL) is poised to dramatically change the delivery of healthcare in the near future. Not only has DL profoundly affected the healthcare industry it has also influenced global businesses. Within a span of very few years, advances such as self-driving cars, robots performing jobs that are hazardous to human, and chat bots talking with human operators have proved that DL has already made large impact on our lives. The open source nature of DL and decreasing prices of computer hardware will further propel such changes. In healthcare, the potential is immense due to the need to automate the processes and evolve error free paradigms. The sheer quantum of DL publications in healthcare has surpassed other domains growing at a very fast pace, particular in radiology. It is therefore imperative for the radiologists to learn about DL and how it differs from other approaches of Artificial Intelligence (AI). The next generation of radiology will see a significant role of DL and will likely serve as the base for augmented radiology (AR). Better clinical judgement by AR will help in improving the quality of life and help in life saving decisions, while lowering healthcare costs. A comprehensive review of DL as well as its implications upon the healthcare is presented in this review. We had analysed 150 articles of DL in healthcare domain from PubMed, Google Scholar, and IEEE EXPLORE focused in medical imagery only. We have further examined the ethic, moral and legal issues surrounding the use of DL in medical imaging.

Sensitivity and specificity of computer vision classification of eyelid photographs for programmatic trachoma assessment

Background/aims

Trachoma programs base treatment decisions on the community prevalence of the clinical signs of trachoma, assessed by direct examination of the conjunctiva. Automated assessment could be more standardized and more cost-effective. We tested the hypothesis that an automated algorithm could classify eyelid photographs better than chance.

Methods

A total of 1,656 field-collected conjunctival images were obtained from clinical trial participants in Niger and Ethiopia. Images were scored for trachomatous inflammation—follicular (TF) and trachomatous inflammation—intense (TI) according to the simplified World Health Organization grading system by expert raters. We developed an automated procedure for image enhancement followed by application of a convolutional neural net classifier for TF and separately for TI. One hundred images were selected for testing TF and TI, and these images were not used for training.

Results

The agreement score for TF and TI tasks for the automated algorithm relative to expert graders was κ = 0.44 (95% CI: 0.26 to 0.62, P < 0.001) and κ = 0.69 (95% CI: 0.55 to 0.84, P < 0.001), respectively.

Discussion

For assessing the clinical signs of trachoma, a convolutional neural net performed well above chance when tested against expert consensus. Further improvements in specificity may render this method suitable for field use.

Texture Analysis for Muscular Dystrophy Classification in MRI with Improved Class Activation Mapping

The muscular dystrophies are made up of a diverse group of rare genetic diseases characterized by progressive loss of muscle strength and muscle damage. Since there is no cure for muscular dystrophy and clinical outcome measures are limited, it is critical to assess the progression of MD objectively. Imaging muscle replacement by fibrofatty tissue has been shown to be a robust biomarker to monitor disease progression in DMD. In magnetic resonance imaging (MRI) data, specific texture patterns are found to correlate to certain MD subtypes and thus present a potential way for automatic assessment. In this paper, we first apply state-of-the-art convolutional neural networks (CNNs) to perform accurate MD image classification and then propose an effective visualization method to highlight the important image textures. With a dystrophic MRI dataset, we found that the best CNN model delivers an 91.7% classification accuracy, which significantly outperforms non-deep learning methods, e.g., >40% improvement has been found over the traditional mean fat fraction (MFF) criterion for DMD and CMD classification. After investigating every single neuron at the top layer of CNN model, we found the superior classification ability of CNN can be explained by its 91 and 118 neurons were performing better than the MFF criterion under the measurements of Euclidean and Chi-square distance, respectively. In order to further interpret CNNs predictions, we tested an improved class activation mapping (ICAM) method to visualize the important regions in the MRI images. With this ICAM, CNNs are able to locate the most discriminative texture patterns of DMD in soleus, lateral gastrocnemius, and medial gastrocnemius; for CMD, the critical texture patterns are highlighted in soleus, tibialis posterior, and peroneus.

Convolutional Neural Networks for Radiologic Images: A Radiologist's Guide

Deep learning has rapidly advanced in various fields within the past few years and has recently gained particular attention in the radiology community. This article provides an introduction to deep learning technology and presents the stages that are entailed in the design process of deep learning radiology research. In addition, the article details the results of a survey of the application of deep learning-specifically, the application of convolutional neural networks-to radiologic imaging that was focused on the following five major system organs: chest, breast, brain, musculoskeletal system, and abdomen and pelvis. The survey of the studies is followed by a discussion about current challenges and future trends and their potential implications for radiology. This article may be used as a guide for radiologists planning research in the field of radiologic image analysis using convolutional neural networks.

Automated CT quantification methods for the assessment of interstitial lung disease in collagen vascular diseases: A systematic review

Interstitial lung disease (ILD) is highly prevalent in collagen vascular diseases and reduction of ILD is an important therapeutic target. To that end, reliable quantification of pulmonary disease severity is of great significance. This study systematically reviewed the literature on automated computed tomography (CT) quantification methods for assessing ILD in collagen vascular diseases. PRISMA-DTA guidelines for systematic reviews were used and 19 original research articles up to January 2018 were included based on a MEDLINE/Pubmed and Embase search. Quantitative CT methods were categorized as histogram assessment (12 studies) or pattern/texture recognition (7 studies). R² for correlation with visual ILD scoring ranged from 0.143 (p < 0.01) to 0.687 (p < 0.0001), for FVC from 0.048 (p < 0.0001) to 0.504 (p < 0.0001) and for DLCO from 0.015 (p = 0.61) to 0.449 (p < 0.0001). Automated CT methods are independent of reader's expertise and are a promising tool in the quantification of ILD in collagen vascular disease patients.

Acute Lymphoblastic Leukemia Detection and Classification of Its Subtypes Using Pretrained Deep Convolutional Neural Networks

Leukemia is a fatal disease of white blood cells which affects the blood and bone marrow in human body. We deployed deep convolutional neural network for automated detection of acute lymphoblastic leukemia and classification of its subtypes into 4 classes, that is, L1, L2, L3, and Normal which were mostly neglected in previous literature. In contrary to the training from scratch, we deployed pretrained AlexNet which was fine-tuned on our data set. Last layers of the pretrained network were replaced with new layers which can classify the input images into 4 classes. To reduce overtraining, data augmentation technique was used. We also compared the data sets with different color models to check the performance over different color images. For acute lymphoblastic leukemia detection, we achieved a sensitivity of 100%, specificity of 98.11%, and accuracy of 99.50%; and for acute lymphoblastic leukemia subtype classification the sensitivity was 96.74%, specificity was 99.03%, and accuracy was 96.06%. Unlike the standard methods, our proposed method was able to achieve high accuracy without any need of microscopic image segmentation.

Segmentation of lung parenchyma in CT images using CNN trained with the clustering algorithm generated dataset

Background

Lung segmentation constitutes a critical procedure for any clinical-decision supporting system aimed to improve the early diagnosis and treatment of lung diseases. Abnormal lungs mainly include lung parenchyma with commonalities on CT images across subjects, diseases and CT scanners, and lung lesions presenting various appearances. Segmentation of lung parenchyma can help locate and analyze the neighboring lesions, but is not well studied in the framework of machine learning.

Methods

We proposed to segment lung parenchyma using a convolutional neural network (CNN) model. To reduce the workload of manually preparing the dataset for training the CNN, one clustering algorithm based method is proposed firstly. Specifically, after splitting CT slices into image patches, the k-means clustering algorithm with two categories is performed twice using the mean and minimum intensity of image patch, respectively. A cross-shaped verification, a volume intersection, a connected component analysis and a patch expansion are followed to generate final dataset. Secondly, we design a CNN architecture consisting of only one convolutional layer with six kernels, followed by one maximum pooling layer and two fully connected layers. Using the generated dataset, a variety of CNN models are trained and optimized, and their performances are evaluated by eightfold cross-validation. A separate validation experiment is further conducted using a dataset of 201 subjects (4.62 billion patches) with lung cancer or chronic obstructive pulmonary disease, scanned by CT or PET/CT. The segmentation results by our method are compared with those yielded by manual segmentation and some available methods.

Results

A total of 121,728 patches are generated to train and validate the CNN models. After the parameter optimization, our CNN model achieves an average F-score of 0.9917 and an area of curve up to 0.9991 for classification of lung parenchyma and non-lung-parenchyma. The obtain model can segment the lung parenchyma accurately for 201 subjects with heterogeneous lung diseases and CT scanners. The overlap ratio between the manual segmentation and the one by our method reaches 0.96.

Conclusions

The results demonstrated that the proposed clustering algorithm based method can generate the training dataset for CNN models. The obtained CNN model can segment lung parenchyma with very satisfactory performance and have the potential to locate and analyze lung lesions.

Reliable Label-Efficient Learning for Biomedical Image Recognition

The use of deep neural networks for biomedical image analysis requires a sufficient number of labeled datasets. To acquire accurate labels as the gold standard, multiple observers with specific expertise are required for both annotation and proofreading. This process can be time-consuming and labor-intensive, making high-quality, and large-annotated biomedical datasets difficult. To address this problem, we propose a deep active learning framework that enables the active selection of both informative queries and reliable experts. To measure the uncertainty of the unlabeled data, a dropout-based strategy is integrated with a similarity criterion for both data selection and random error elimination. To select the reliable labelers, we adopt an expertise estimator to learn the expertise levels of labelers via offline-testing and online consistency evaluation. The proposed method is applied to classification tasks on two types of medical images including confocal endomicroscopy images and gastrointestinal endoscopic images. The annotations are acquired from multiple labelers with diverse levels of expertise. The experiments demonstrate the efficiency and promising performance of the proposed method compared to a set of baseline methods.