Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning

Tumor detection on bronchoscopic images by unsupervised learning

The diagnosis and early identification of intratracheal tumors relies on the experience of the operators and the specialists. Operations by physicians with insufficient experience may lead to misdiagnosis or misjudgment of tumors. To address this issue, a datasets for intratracheal tumor detection has been constructed to simulate the diagnostic level of experienced specialists, and a Knowledge Distillation-based Memory Feature Unsupervised Anomaly Detection (KD-MFAD) model was proposed to learn from this simulated experience. The unsupervised training approach could effectively deal with the irregular features of the tumorous appearance. The Downward Deformable Convolution Module (DDC) allowed the encoding phase to provide more detailed internal airway environment features. The Memory Matrix based on Convolutional Block focusing (CB-Mem) helped the student model store more meaningful normal sample features during training and disrupted the reconstruction of "tumor" images. Our model achieved an AUC-ROC of 97.60%, Acc of 93.33%, and F1-score of 94.94% on our self-built intratracheal endoscopy datasets, improving baseline performance by 5 to 10%. Our model also demonstrated superior performance over existing models in the public datasets in the same field.

Diagnosing Respiratory Variability: Convolutional Neural Networks for Chest X-ray Classification Across Diverse Pulmonary Conditions

The global burden of lung diseases is a pressing issue, particularly in developing nations with limited healthcare access. Accurate diagnosis of lung conditions is crucial for effective treatment, but diagnosing lung ailments using medical imaging techniques like chest radiograph images and CT scans is challenging due to the complex anatomical intricacies of the lungs. Deep learning methods, particularly convolutional neural networks (CNN), offer promising solutions for automated disease classification using imaging data. This research has the potential to significantly improve healthcare access in developing countries with limited medical resources, providing hope for better diagnosis and treatment of lung diseases. The study employed a diverse range of CNN models for training, including a baseline model and transfer learning models such as VGG16, VGG19, InceptionV3, and ResNet50. The models were trained using image datasets sourced from the NIH and COVID-19 repositories containing 8000 chest radiograph images depicting four lung conditions (lung opacity, COVID-19, pneumonia, and pneumothorax) and 2000 healthy chest radiograph images, with a ten-fold cross-validation approach. The VGG19-based model outperformed the baseline model in diagnosing lung diseases with an average accuracy of 0.995 and 0.996 on validation and external test datasets. The proposed model also outperformed published lung-disease prediction models; these findings underscore the superior performance of the VGG19 model compared to other architectures in accurately classifying and detecting lung diseases from chest radiograph images. This study highlights AI's potential, especially CNNs like VGG19, in improving diagnostic accuracy for lung disorders, promising better healthcare outcomes. The predictive model is available on GitHub at https://github.com/PGlab-NIPER/Lung_disease_classification .

A Dual-Branch Residual Network with Attention Mechanisms for Enhanced Classification of Vaginal Lesions in Colposcopic Images

Vaginal intraepithelial neoplasia (VAIN), linked to HPV infection, is a condition that is often overlooked during colposcopy, especially in the vaginal vault area, as clinicians tend to focus more on cervical lesions. This oversight can lead to missed or delayed diagnosis and treatment for patients with VAIN. Timely and accurate classification of VAIN plays a crucial role in the evaluation of vaginal lesions and the formulation of effective diagnostic approaches. The challenge is the high similarity between different classes and the low variability in the same class in colposcopic images, which can affect the accuracy, precision, and recall rates, depending on the image quality and the clinician's experience. In this study, a dual-branch lesion-aware residual network (DLRNet), designed for small medical sample sizes, is introduced, which classifies vaginal lesions by examining the relationship between cervical and vaginal lesions. The DLRNet model includes four main components: a lesion localization module, a dual-branch classification module, an attention-guidance module, and a pretrained network module. The dual-branch classification module combines the original images with segmentation maps obtained from the lesion localization module using a pretrained ResNet network to fine-tune parameters at different levels, explore lesion-specific features from both global and local perspectives, and facilitate layered interactions. The feature guidance module focuses the local branch network on vaginal-specific features by using spatial and channel attention mechanisms. The final integration involves a shared feature extraction module and independent fully connected layers, which represent and merge the dual-branch inputs. The weighted fusion method effectively integrates multiple inputs, enhancing the discriminative and generalization capabilities of the model. Classification experiments on 1142 collected colposcopic images demonstrate that this method raises the existing classification levels, achieving the classification of VAIN into three lesion grades, thus providing a valuable tool for the early screening of vaginal diseases.

An accurate prediction for respiratory diseases using deep learning on bronchoscopy diagnosis images

INTRODUCTION

Bronchoscopy is of great significance in diagnosing and treating respiratory illness. Using deep learning, a diagnostic system for bronchoscopy images can improve the accuracy of tracheal, bronchial, and pulmonary disease diagnoses for physicians and ensure timely pathological or etiological examinations for patients. Improving the diagnostic accuracy of the algorithms remains the key to this technology.

OBJECTIVES

To deal with the problem, we proposed a multiscale attention residual network (MARN) for diagnosing lung conditions through bronchoscopic images. The multiscale convolutional block attention module (MCBAM) was designed to enable accurate focus on lesion regions by enhancing spatial and channel features. Gradient-weighted Class Activation Map (Grad-CAM) was provided to increase the interpretability of diagnostic results.

METHODS

We collected 615 cases from Harbin Medical University Cancer Hospital, including 2900 images. The dataset was partitioned randomly into training sets, validation sets and test sets to update model parameters, evaluate the model's training performance, select network architecture and parameters, and estimate the final model. In addition, we compared MARN with other algorithms. Furthermore, three physicians with different qualifications were invited to diagnose the same test images, and the results were compared to those of the model.

RESULTS

In the dataset of normal and lesion images, our model displayed an accuracy of 97.76% and an AUC of 99.79%. The model recorded 92.26% accuracy and 96.82% AUC for datasets of benign and malignant lesion images, while it achieved 93.10% accuracy and 99.02% AUC for normal, benign, and malignant lesion images.

CONCLUSION

These results demonstrated that our network outperforms other methods in diagnostic performance. The accuracy of our model is roughly the same as that of experienced physicians and the efficiency is much higher than doctors. MARN has great potential for assisting physicians with assessing the bronchoscopic images precisely.

Predicting malignancy in breast lesions: enhancing accuracy with fine-tuned convolutional neural network models

BACKGROUND

This study aims to explore the accuracy of Convolutional Neural Network (CNN) models in predicting malignancy in Dynamic Contrast-Enhanced Breast Magnetic Resonance Imaging (DCE-BMRI).

METHODS

A total of 273 benign lesions (benign group) and 274 malignant lesions (malignant group) were collected and randomly divided into a training set (246 benign and 245 malignant lesions) and a testing set (28 benign and 28 malignant lesions) in a 9:1 ratio. An additional 53 lesions from 53 patients were designated as the validation set. Five models-VGG16, VGG19, DenseNet201, ResNet50, and MobileNetV2-were evaluated. Model performance was assessed using accuracy (Ac) in the training and testing sets, and precision (Pr), recall (Rc), F1 score (F1), and area under the receiver operating characteristic curve (AUC) in the validation set.

RESULTS

The accuracy of VGG19 on the test set (0.96) is higher than that of VGG16 (0.91), DenseNet201 (0.91), ResNet50 (0.67), and MobileNetV2 (0.88). For the validation set, VGG19 achieved higher performance metrics (Pr 0.75, Rc 0.76, F1 0.73, AUC 0.76) compared to the other models, specifically VGG16 (Pr 0.73, Rc 0.75, F1 0.70, AUC 0.73), DenseNet201 (Pr 0.71, Rc 0.74, F1 0.69, AUC 0.71), ResNet50 (Pr 0.65, Rc 0.68, F1 0.60, AUC 0.65), and MobileNetV2 (Pr 0.73, Rc 0.75, F1 0.71, AUC 0.73). S4 model achieved higher performance metrics (Pr 0.89, Rc 0.88, F1 0.87, AUC 0.89) compared to the other four fine-tuned models, specifically S1 (Pr 0.75, Rc 0.76, F1 0.74, AUC 0.75), S2 (Pr 0.77, Rc 0.79, F1 0.75, AUC 0.77), S3 (Pr 0.76, Rc 0.76, F1 0.73, AUC 0.75), and S5 (Pr 0.77, Rc 0.79, F1 0.75, AUC 0.77). Additionally, S4 model showed the lowest loss value in the testing set. Notably, the AUC of S4 for BI-RADS 3 was 0.90 and for BI-RADS 4 was 0.86, both significantly higher than the 0.65 AUC for BI-RADS 5.

CONCLUSIONS

The S4 model we propose has demonstrated superior performance in predicting the likelihood of malignancy in DCE-BMRI, making it a promising candidate for clinical application in patients with breast diseases. However, further validation is essential, highlighting the need for additional data to confirm its efficacy.

Deep Transfer Learning Method Using Self-Pixel and Global Channel Attentive Regularization

The purpose of this paper is to propose a novel transfer learning regularization method based on knowledge distillation. Recently, transfer learning methods have been used in various fields. However, problems such as knowledge loss still occur during the process of transfer learning to a new target dataset. To solve these problems, there are various regularization methods based on knowledge distillation techniques. In this paper, we propose a transfer learning regularization method based on feature map alignment used in the field of knowledge distillation. The proposed method is composed of two attention-based submodules: self-pixel attention (SPA) and global channel attention (GCA). The self-pixel attention submodule utilizes both the feature maps of the source and target models, so that it provides an opportunity to jointly consider the features of the target and the knowledge of the source. The global channel attention submodule determines the importance of channels through all layers, unlike the existing methods that calculate these only within a single layer. Accordingly, transfer learning regularization is performed by considering both the interior of each single layer and the depth of the entire layer. Consequently, the proposed method using both of these submodules showed overall improved classification accuracy than the existing methods in classification experiments on commonly used datasets.

A multi-class brain tumor grading system based on histopathological images using a hybrid YOLO and RESNET networks

Gliomas are primary brain tumors caused by glial cells. These cancers' classification and grading are crucial for prognosis and treatment planning. Deep learning (DL) can potentially improve the digital pathology investigation of brain tumors. In this paper, we developed a technique for visualizing a predictive tumor grading model on histopathology pictures to help guide doctors by emphasizing characteristics and heterogeneity in forecasts. The proposed technique is a hybrid model based on YOLOv5 and ResNet50. The function of YOLOv5 is to localize and classify the tumor in large histopathological whole slide images (WSIs). The suggested technique incorporates ResNet into the feature extraction of the YOLOv5 framework, and the detection results show that our hybrid network is effective for identifying brain tumors from histopathological images. Next, we estimate the glioma grades using the extreme gradient boosting classifier. The high-dimensional characteristics and nonlinear interactions present in histopathology images are well-handled by this classifier. DL techniques have been used in previous computer-aided diagnosis systems for brain tumor diagnosis. However, by combining the YOLOv5 and ResNet50 architectures into a hybrid model specifically designed for accurate tumor localization and predictive grading within histopathological WSIs, our study presents a new approach that advances the field. By utilizing the advantages of both models, this creative integration goes beyond traditional techniques to produce improved tumor localization accuracy and thorough feature extraction. Additionally, our method ensures stable training dynamics and strong model performance by integrating ResNet50 into the YOLOv5 framework, addressing concerns about gradient explosion. The proposed technique is tested using the cancer genome atlas dataset. During the experiments, our model outperforms the other standard ways on the same dataset. Our results indicate that the proposed hybrid model substantially impacts tumor subtype discrimination between low-grade glioma (LGG) II and LGG III. With 97.2% of accuracy, 97.8% of precision, 98.6% of sensitivity, and the Dice similarity coefficient of 97%, the proposed model performs well in classifying four grades. These results outperform current approaches for identifying LGG from high-grade glioma and provide competitive performance in classifying four categories of glioma in the literature.

PKDN: Prior Knowledge Distillation Network for bronchoscopy diagnosis

Bronchoscopy plays a crucial role in diagnosing and treating lung diseases. The deep learning-based diagnostic system for bronchoscopic images can assist physicians in accurately and efficiently diagnosing lung diseases, enabling patients to undergo timely pathological examinations and receive appropriate treatment. However, the existing diagnostic methods overlook the utilization of prior knowledge of medical images, and the limited feature extraction capability hinders precise focus on lesion regions, consequently affecting the overall diagnostic effectiveness. To address these challenges, this paper proposes a prior knowledge distillation network (PKDN) for identifying lung diseases through bronchoscopic images. The proposed method extracts color and edge features from lesion images using the prior knowledge guidance module, and subsequently enhances spatial and channel features by employing the dynamic spatial attention module and gated channel attention module, respectively. Finally, the extracted features undergo refinement and self-regulation through feature distillation. Furthermore, decoupled distillation is implemented to balance the importance of target and non-target class distillation, thereby enhancing the diagnostic performance of the network. The effectiveness of the proposed method is validated on the bronchoscopic dataset provided by Harbin Medical University Cancer Hospital, which consists of 2,029 bronchoscopic images from 200 patients. Experimental results demonstrate that the proposed method achieves an accuracy of 94.78% and an AUC of 98.17%, outperforming other methods significantly in diagnostic performance. These results indicate that the computer-aided diagnostic system based on PKDN provides satisfactory accuracy in diagnosing lung diseases during bronchoscopy.

Multiple Field-of-View Based Attention Driven Network for Weakly Supervised Common Bile Duct Stone Detection

OBJECTIVE

Common bile duct (CBD) stones caused diseases are life-threatening. Because CBD stones locate in the distal part of the CBD and have relatively small sizes, detecting CBD stones from CT scans is a challenging issue in the medical domain.

METHODS AND PROCEDURES

We propose a deep learning based weakly-supervised method called multiple field-of-view based attention driven network (MFADNet) to detect CBD stones from CT scans based on image-level labels. Three dominant modules including a multiple field-of-view encoder, an attention driven decoder and a classification network are collaborated in the network. The encoder learns the feature of multi-scale contextual information while the decoder with the classification network is applied to locate the CBD stones based on spatial-channel attentions. To drive the learning of the whole network in a weakly-supervised and end-to-end trainable manner, four losses including the foreground loss, background loss, consistency loss and classification loss are proposed.

RESULTS

Compared with state-of-the-art weakly-supervised methods in the experiments, the proposed method can accurately classify and locate CBD stones based on the quantitative and qualitative results.

CONCLUSION

We propose a novel multiple field-of-view based attention driven network for a new medical application of CBD stone detection from CT scans while only image-levels are required to reduce the burdens of labeling and help physicians automatically diagnose CBD stones. The source code is available at https://github.com/nchucvml/MFADNet after acceptance.

CLINICAL IMPACT

Our deep learning method can help physicians localize relatively small CBD stones for effectively diagnosing CBD stone caused diseases.

A Comprehensive Survey on the Progress, Process, and Challenges of Lung Cancer Detection and Classification

Lung cancer is the primary reason of cancer deaths worldwide, and the percentage of death rate is increasing step by step. There are chances of recovering from lung cancer by detecting it early. In any case, because the number of radiologists is limited and they have been working overtime, the increase in image data makes it hard for them to evaluate the images accurately. As a result, many researchers have come up with automated ways to predict the growth of cancer cells using medical imaging methods in a quick and accurate way. Previously, a lot of work was done on computer-aided detection (CADe) and computer-aided diagnosis (CADx) in computed tomography (CT) scan, magnetic resonance imaging (MRI), and X-ray with the goal of effective detection and segmentation of pulmonary nodule, as well as classifying nodules as malignant or benign. But still, no complete comprehensive review that includes all aspects of lung cancer has been done. In this paper, every aspect of lung cancer is discussed in detail, including datasets, image preprocessing, segmentation methods, optimal feature extraction and selection methods, evaluation measurement matrices, and classifiers. Finally, the study looks into several lung cancer-related issues with possible solutions.

Clinical value of video-assisted single-lumen endotracheal intubation and application of artificial intelligence in it

Visualization techniques and artificial intelligence (AI) are currently used for intubation device. By providing airway visualization during tracheal intubation, the technologies provide safe and accurate access to the trachea. The ability of AI to automatically identify airways from images of intubation device makes it attractive for use in intubation devices. The purpose of this review is to introduce the state of application of visualization techniques and AI in certain intubation devices. We reviewed the evidence of clinical implications of the use of video-assisted intubation device in the intubation time, first attempt success rate, and intubation of the difficult airway. Especially, VivaSight single-lumen tube with an incorporated optics allows direct viewing of the airway. VivaSight single-lumen tube has more advantages in tracheal intubation. AI has been applied to fiberoptic bronchoscopy (FOB) and video laryngoscope with automatic airway image recognition, and has achieved certain accomplishment. Further, we discussed the possibility of applying AI to the VivaSight single-lumen tube and proposed future directions of research and application.

Differentiation of breast lesions on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) using deep transfer learning based on DenseNet201

In order to achieve better performance, artificial intelligence is used in breast cancer diagnosis. In this study, we evaluated the efficacy of different fine-tuning strategies of deep transfer learning (DTL) based on the DenseNet201 model to differentiate malignant from benign lesions on breast dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). We collected 4260 images of benign lesions and 4140 images of malignant lesions of the breast pertaining to pathologically confirmed cases. The benign and malignant groups was randomly divided into a training set and a testing set at a ratio of 9:1. A DTL model based on the DenseNet201 model was established, and the effectiveness of 4 fine-tuning strategies (S0: strategy 0, S1: strategy; S2: strategy; and S3: strategy) was compared. Additionally, DCE-MRI images of 48 breast lesions were selected to verify the robustness of the model. Ten images were obtained for each lesion. The classification was considered correct if more than 5 images were correctly classified. The metrics for model performance evaluation included accuracy (Ac) in the training and testing sets, precision (Pr), recall rate (Rc), f1 score (f1), and area under the receiver operating characteristic curve (AUROC) in the validation set. The Ac of the 4 fine-tuning strategies reached 100.00% in the training set. The S2 strategy exhibited good convergence in the testing set. The Ac of S2 was 98.01% in the testing set, which was higher than those of S0 (93.10%), S1 (90.45%), and S3 (93.90%). The average classification Pr, Rc, f1, and AUROC of S2 in the validation set were (89.00%, 80.00%, 0.81, and 0.79, respectively) higher than those of S0 (76.00%, 67.00%, 0.69, and 0.65, respectively), S1 (60.00%, 60.00%, 0.60, 0.66, and respectively), and S3 (77.00%, 73.00%, 0.74, 0.72, respectively). The degree of coincidence between S2 and the histopathological method for differentiating between benign and malignant breast lesions was high (κ = 0.749). The S2 strategy can improve the robustness of the DenseNet201 model in relatively small breast DCE-MRI datasets, and this is a reliable method to increase the Ac of discriminating benign from malignant breast lesions on DCE-MRI.

Development and validation of the artificial intelligence (AI)-based diagnostic model for bronchial lumen identification

BACKGROUND

Bronchoscopy is a key step in the diagnosis and treatment of respiratory diseases. However, the level of expertise varies among different bronchoscopists. Artificial intelligence (AI) may help them identify bronchial lumens. Thus, a bronchoscopy quality-control system based on AI was built to improve the performance of bronchoscopists.

METHODS

This single-center observational study consecutively collected bronchoscopy videos from Shanghai Chest Hospital and segmented each video into 31 different anatomical locations to develop an AI-assisted system based on a convolutional neural network (CNN) model. We then designed a single-center trial to compare the accuracy of lumen recognition by bronchoscopists with and without the assistance of the AI system.

RESULTS

A total of 28,441 qualified images of bronchial lumen were used to train the CNNs. In the cross-validation set, the optimal accuracy of the six models was between 91.83% and 96.62%. In the test set, the visual geometry group 16 (VGG-16) achieved optimal performance with an accuracy of 91.88%, and an area under the curve of 0.995. In the clinical evaluation, the accuracy rate of the AI system alone was 54.30% (202/372). For the identification of bronchi except for segmental bronchi, the accuracy was 82.69% (129/156). In group 1, the recognition accuracy rates of doctors A, B, a and b alone were 42.47%, 34.68%, 28.76%, and 29.57%, respectively, but increased to 57.53%, 54.57%, 54.57%, and 46.24% respectively when combined with the AI system. Similarly, in group 2, the recognition accuracy rates of doctors C, D, c, and d were 37.90%, 41.40%, 30.91%, and 33.60% respectively, but increased to 51.61%, 47.85%, 53.49%, and 54.30% respectively, when combined with the AI system. Except for doctor D, the accuracy of doctors in recognizing lumen was significantly higher with AI assistance than without AI assistance, regardless of their experience (P<0.001).

CONCLUSIONS

Our AI system could better recognize bronchial lumen and reduce differences in the operation levels of different bronchoscopists. It could be used to improve the quality of everyday bronchoscopies.

The investigation of construction and clinical application of image recognition technology assisted bronchoscopy diagnostic model of lung cancer

Background

The incidence and mortality of lung cancer ranks first in China. Bronchoscopy is one of the most common diagnostic methods for lung cancer. In recent years, image recognition technology(IRT) has been more and more widely studied and applied in the medical field. We developed a diagnostic model of lung cancer under bronchoscopy based on deep learning method and tried to classify pathological types.

Methods

A total of 2238 lesion images were collected retrospectively from 666 cases of lung cancer diagnosed by pathology in the bronchoscopy center of the Third Xiangya Hospital from Oct.01 2017 to Dec.31 2020 and 152 benign cases from Jun.01 2015 to Dec.31 2020. The benign and malignant images were divided into training, verification and test set according to 7:1:2 respectively. The model was trained and tested based on deep learning method. We also tried to classify different pathological types of lung cancer using the model. Furthermore, 9 clinicians with different experience were invited to diagnose the same test images and the results were compared with the model.

Results

The diagnostic model took a total of 30s to diagnose 467 test images. The overall accuracy, sensitivity, specificity and area under curve (AUC) of the model to differentiate benign and malignant lesions were 0.951, 0.978, 0.833 and 0.940, which were equivalent to the judgment results of 2 doctors in the senior group and higher than those of other doctors. In the classification of squamous cell carcinoma (SCC) and adenocarcinoma (AC), the overall accuracy was 0.745, including 0.790 for SCC, 0.667 for AC and AUC was 0.728.

Conclusion

The performance of our diagnostic model to distinguish benign and malignant lesions in bronchoscopy is roughly the same as that of experienced clinicians and the efficiency is much higher than manually. Our study verifies the possibility of applying IRT in diagnosis of lung cancer during white light bronchoscopy.

A brain-like classification method for computed tomography images based on adaptive feature matching dual-source domain heterogeneous transfer learning

Transfer learning can improve the robustness of deep learning in the case of small samples. However, when the semantic difference between the source domain data and the target domain data is large, transfer learning easily introduces redundant features and leads to negative transfer. According the mechanism of the human brain focusing on effective features while ignoring redundant features in recognition tasks, a brain-like classification method based on adaptive feature matching dual-source domain heterogeneous transfer learning is proposed for the preoperative aided diagnosis of lung granuloma and lung adenocarcinoma for patients with solitary pulmonary solid nodule in the case of small samples. The method includes two parts: (1) feature extraction and (2) feature classification. In the feature extraction part, first, By simulating the feature selection mechanism of the human brain in the process of drawing inferences about other cases from one instance, an adaptive selected-based dual-source domain feature matching network is proposed to determine the matching weight of each pair of feature maps and each pair of convolution layers between the two source networks and the target network, respectively. These two weights can, respectively, adaptive select the features in the source network that are conducive to the learning of the target task, and the destination of feature transfer to improve the robustness of the target network. Meanwhile, a target network based on diverse branch block is proposed, which made the target network have different receptive fields and complex paths to further improve the feature expression ability of the target network. Second, the convolution kernel of the target network is used as the feature extractor to extract features. In the feature classification part, an ensemble classifier based on sparse Bayesian extreme learning machine is proposed that can automatically decide how to combine the output of base classifiers to improve the classification performance. Finally, the experimental results (the AUCs were 0.9542 and 0.9356, respectively) on the data of two center data show that this method can provide a better diagnostic reference for doctors.

A Deep Learning Computer-Aided Diagnosis Approach for Breast Cancer

Breast cancer is a gigantic burden on humanity, causing the loss of enormous numbers of lives and amounts of money. It is the world’s leading type of cancer among women and a leading cause of mortality and morbidity. The histopathological examination of breast tissue biopsies is the gold standard for diagnosis. In this paper, a computer-aided diagnosis (CAD) system based on deep learning is developed to ease the pathologist’s mission. For this target, five pre-trained convolutional neural network (CNN) models are analyzed and tested—Xception, DenseNet201, InceptionResNetV2, VGG19, and ResNet152—with the help of data augmentation techniques, and a new approach is introduced for transfer learning. These models are trained and tested with histopathological images obtained from the BreakHis dataset. Multiple experiments are performed to analyze the performance of these models through carrying out magnification-dependent and magnification-independent binary and eight-class classifications. The Xception model has shown promising performance through achieving the highest classification accuracies for all the experiments. It has achieved a range of classification accuracies from 93.32% to 98.99% for magnification-independent experiments and from 90.22% to 100% for magnification-dependent experiments.

A computer-aided diagnosis system for detecting various diabetic retinopathy grades based on a hybrid deep learning technique

Diabetic retinopathy (DR) is a serious disease that may cause vision loss unawares without any alarm. Therefore, it is essential to scan and audit the DR progress continuously. In this respect, deep learning techniques achieved great success in medical image analysis. Deep convolution neural network (CNN) architectures are widely used in multi-label (ML) classification. It helps in diagnosing normal and various DR grades: mild, moderate, and severe non-proliferative DR (NPDR) and proliferative DR (PDR). DR grades are formulated by appearing multiple DR lesions simultaneously on the color retinal fundus images. Many lesion types have various features that are difficult to segment and distinguished by utilizing conventional and hand-crafted methods. Therefore, the practical solution is to utilize an effective CNN model. In this paper, we present a novel hybrid, deep learning technique, which is called E-DenseNet. We integrated EyeNet and DenseNet models based on transfer learning. We customized the traditional EyeNet by inserting the dense blocks and optimized the resulting hybrid E-DensNet model's hyperparameters. The proposed system based on the E-DenseNet model can accurately diagnose healthy and different DR grades from various small and large ML color fundus images. We trained and tested our model on four different datasets that were published from 2006 to 2019. The proposed system achieved an average accuracy (ACC), sensitivity (SEN), specificity (SPE), Dice similarity coefficient (DSC), the quadratic Kappa score (QKS), and the calculation time (T) in minutes (m) equal [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], 0.883, and 3.5m respectively. The experiments show promising results as compared with other systems.

Extravasation Screening and Severity Prediction from Skin Lesion Image using Deep Neural Networks

Extravasation occurs secondary to the leakage of medication from blood vessels into the surrounding tissue during intravenous administration resulting in significant soft tissue injury and necrosis. If treatment is delayed, invasive management such as surgical debridement, skin grafting, and even amputation may be required. Thus, it is imperative to develop a smartphone application for predicting extravasation severity from skin image. Two Deep Neural Network (DNN) architectures, U-Net and DenseNet-121, were used to segment skin and lesion, and to classify extravasation severity. Sensitivity and specificity for predicting between asymptomatic and abnormal cases were 77.78 and 90.24%. For each severity in abnormal cases, mild extravasation attained the highest F1-score of 0.8049, followed by severe extravasation of 0.6429, and moderate extravasation of 0.6250. The F1-score of moderate-to-severe extravasation classification can improve by applying the our proposed rule-based for multi-class classification. These findings proposed a novel and feasible DNN approach for screening extravasation from skin images. The implementation of DNN-based applications on mobile devices has a strong potential for clinical application in low-resource countries. Clinical relevance- The application can serve as a valuable tool in monitoring when extravasation occurs during intravaneous administration. It can also help in the scheduling process across worksite to reduce the risks associated with working shifts.

Shock due to an Obstructed Endotracheal Tube

Endotracheal tube obstruction by a mucus plug causing a ball-valve effect is a rare but significant complication. The inability to pass a suction catheter through the endotracheal tube with high peak and plateau pressure differences are classical features of an endotracheal tube obstruction. A case is described of endotracheal tube obstruction from a mucus plug that compounded severe respiratory acidosis and hypotension in a patient who simultaneously had abdominal compartment syndrome. The mucus plug was not identified until a bronchoscopic assessment of the airway was performed. Due to the absence of classical signs, the delayed identification of the obstructing mucus plug exacerbated diagnostic confusion. It resulted in various treatments being trialed whilst the patient continued to deteriorate from the evasive offending culprit. We suggest that earlier and more routine use of bronchoscopy should be employed in an intensive care unit, especially as a definitive way to rule out endotracheal obstruction.

Lung Cancer Detection and Improving Accuracy Using Linear Subspace Image Classification Algorithm

The ability to identify lung cancer at an early stage is critical, because it can help patients live longer. However, predicting the affected area while diagnosing cancer is a huge challenge. An intelligent computer-aided diagnostic system can be utilized to detect and diagnose lung cancer by detecting the damaged region. The suggested Linear Subspace Image Classification Algorithm (LSICA) approach classifies images in a linear subspace. This methodology is used to accurately identify the damaged region, and it involves three steps: image enhancement, segmentation, and classification. The spatial image clustering technique is used to quickly segment and identify the impacted area in the image. LSICA is utilized to determine the accuracy value of the affected region for classification purposes. Therefore, a lung cancer detection system with classification-dependent image processing is used for lung cancer CT imaging. Therefore, a new method to overcome these deficiencies of the process for detection using LSICA is proposed in this work on lung cancer. MATLAB has been used in all programs. A proposed system designed to easily identify the affected region with help of the classification technique to enhance and get more accurate results.

Lesion Segmentation in Ultrasound Using Semi-Pixel-Wise Cycle Generative Adversarial Nets

Breast cancer is the most common invasive cancer with the highest cancer occurrence in females. Handheld ultrasound is one of the most efficient ways to identify and diagnose the breast cancer. The area and the shape information of a lesion is very helpful for clinicians to make diagnostic decisions. In this study we propose a new deep-learning scheme, semi-pixel-wise cycle generative adversarial net (SPCGAN) for segmenting the lesion in 2D ultrasound. The method takes the advantage of a fully convolutional neural network (FCN) and a generative adversarial net to segment a lesion by using prior knowledge. We compared the proposed method to a fully connected neural network and the level set segmentation method on a test dataset consisting of 32 malignant lesions and 109 benign lesions. Our proposed method achieved a Dice similarity coefficient (DSC) of 0.92 while FCN and the level set achieved 0.90 and 0.79 respectively. Particularly, for malignant lesions, our method increases the DSC (0.90) of the fully connected neural network to 0.93 significantly (p 0.001). The results show that our SPCGAN can obtain robust segmentation results. The framework of SPCGAN is particularly effective when sufficient training samples are not available compared to FCN. Our proposed method may be used to relieve the radiologists' burden for annotation.

Auxiliary Diagnosis for COVID-19 with Deep Transfer Learning

To assist physicians identify COVID-19 and its manifestations through the automatic COVID-19 recognition and classification in chest CT images with deep transfer learning. In this retrospective study, the used chest CT image dataset covered 422 subjects, including 72 confirmed COVID-19 subjects (260 studies, 30,171 images), 252 other pneumonia subjects (252 studies, 26,534 images) that contained 158 viral pneumonia subjects and 94 pulmonary tuberculosis subjects, and 98 normal subjects (98 studies, 29,838 images). In the experiment, subjects were split into training (70%), validation (15%) and testing (15%) sets. We utilized the convolutional blocks of ResNets pretrained on the public social image collections and modified the top fully connected layer to suit our task (the COVID-19 recognition). In addition, we tested the proposed method on a finegrained classification task; that is, the images of COVID-19 were further split into 3 main manifestations (ground-glass opacity with 12,924 images, consolidation with 7418 images and fibrotic streaks with 7338 images). Similarly, the data partitioning strategy of 70%-15%-15% was adopted. The best performance obtained by the pretrained ResNet50 model is 94.87% sensitivity, 88.46% specificity, 91.21% accuracy for COVID-19 versus all other groups, and an overall accuracy of 89.01% for the three-category classification in the testing set. Consistent performance was observed from the COVID-19 manifestation classification task on images basis, where the best overall accuracy of 94.08% and AUC of 0.993 were obtained by the pretrained ResNet18 (P < 0.05). All the proposed models have achieved much satisfying performance and were thus very promising in both the practical application and statistics. Transfer learning is worth for exploring to be applied in recognition and classification of COVID-19 on CT images with limited training data. It not only achieved higher sensitivity (COVID-19 vs the rest) but also took far less time than radiologists, which is expected to give the auxiliary diagnosis and reduce the workload for the radiologists.

Fully automatic deep learning trained on limited data for carotid artery segmentation from large image volumes

BACKGROUND

The objectives of this study were to develop a 3D convolutional deep learning framework (CarotidNet) for fully automatic segmentation of carotid bifurcations in computed tomography angiography (CTA) images and to facilitate the quantification of carotid stenosis and risk assessment of stroke.

METHODS

Our pipeline was a two-stage cascade network that included a localization phase and a segmentation phase. The network framework was based on the 3D version of U-Net, but was refined in three ways: (I) by adding residual connections and a deep supervision strategy to cope with the vanishing problem in back-propagation; (II) by adopting dilated convolution in order to strengthen the capacity to capture contextual information; and (III) by establishing a hybrid objective function to address the extreme imbalance between foreground and background voxels.

RESULTS

We trained our networks on 15 cases and evaluated their performance based on 41 cases from the MICCAI Challenge 2009 dataset. A Dice similarity coefficient of 82.3% was achieved for the test cases.

CONCLUSIONS

We developed a carotid segmentation method based on U-Net that can segment tiny carotid bifurcation lumens from very large backgrounds with no manual intervention. This was the first attempt to use deep learning to achieve carotid bifurcation segmentation in 3D CTA images. Our results indicate that deep learning is a promising method for automatically extracting carotid bifurcation lumens.

Integration of cardiovascular risk assessment with COVID-19 using artificial intelligence

Artificial Intelligence (AI), in general, refers to the machines (or computers) that mimic "cognitive" functions that we associate with our mind, such as "learning" and "solving problem". New biomarkers derived from medical imaging are being discovered and are then fused with non-imaging biomarkers (such as office, laboratory, physiological, genetic, epidemiological, and clinical-based biomarkers) in a big data framework, to develop AI systems. These systems can support risk prediction and monitoring. This perspective narrative shows the powerful methods of AI for tracking cardiovascular risks. We conclude that AI could potentially become an integral part of the COVID-19 disease management system. Countries, large and small, should join hands with the WHO in building biobanks for scientists around the world to build AI-based platforms for tracking the cardiovascular risk assessment during COVID-19 times and long-term follow-up of the survivors.

From Local to Global: A Graph Framework for Retinal Artery/Vein Classification

Fundus photography has been widely used for inspecting eye disorders by ophthalmologists or computer algorithms. Biomarkers related to retinal vessels plays an essential role to detect early diabetes. To quantify vascular biomarkers or the corresponding changes, an accurate artery and vein classification is necessary. In this work, we propose a new framework to boost local vessel classification with a global vascular network model using graph convolution. We compare our proposed method with two traditional state-of-the-art methods on a testing dataset of 750 images from the Maastricht Study. After incorporating global information, our model achieves the best accuracy of 86.45% compared to 85.5% from convolutional neural networks (CNN) and 82.9% from handcrafted pixel feature classification (HPFC). Our model also obtains the best area under receiver operating characteristic curve (AUC) of 0.95, compared to 0.93 from CNN and 0.90 from HPFC. The new classification framework has the advantage of easy deployment on top of local classification features. It corrects the local classification error by minimizing global classification error and it brings free additional classification performance.

MobileNetV2 Ensemble for Cervical Precancerous Lesions Classification

Women’s cancers remain a major challenge for many health systems. Between 1991 and 2017, the death rate for all major cancers fell continuously in the United States, excluding uterine cervix and uterine corpus cancers. Together with HPV (Human Papillomavirus) testing and cytology, colposcopy has played a central role in cervical cancer screening. This medical procedure allows physicians to view the cervix at a magnification of up to 10%. This paper presents an automated colposcopy image analysis framework for the classification of precancerous and cancerous lesions of the uterine cervix. This framework is based on an ensemble of MobileNetV2 networks. Our experimental results show that this method achieves accuracies of 83.33% and 91.66% on the four-class and binary classification tasks, respectively. These results are promising for the future use of automatic classification methods based on deep learning as tools to support medical doctors.

Cloud-Based Automated Clinical Decision Support System for Detection and Diagnosis of Lung Cancer in Chest CT

Lung cancer is a major cause for cancer-related deaths. The detection of pulmonary cancer in the early stages can highly increase survival rate. Manual delineation of lung nodules by radiologists is a tedious task. We developed a novel computer-aided decision support system for lung nodule detection based on a 3D Deep Convolutional Neural Network (3DDCNN) for assisting the radiologists. Our decision support system provides a second opinion to the radiologists in lung cancer diagnostic decision making. In order to leverage 3-dimensional information from Computed Tomography (CT) scans, we applied median intensity projection and multi-Region Proposal Network (mRPN) for automatic selection of potential region-of-interests. Our Computer Aided Diagnosis (CAD) system has been trained and validated using LUNA16, ANODE09, and LIDC-IDR datasets; the experiments demonstrate the superior performance of our system, attaining sensitivity, specificity, AUROC, accuracy, of 98.4%, 92%, 96% and 98.51% with 2.1 FPs per scan. We integrated cloud computing, trained and validated our Cloud-Based 3DDCNN on the datasets provided by Shanghai Sixth People's Hospital, as well as LUNA16, ANODE09, and LIDC-IDR. Our system outperformed the state-of-the-art systems and obtained an impressive 98.7% sensitivity at 1.97 FPs per scan. This shows the potentials of deep learning, in combination with cloud computing, for accurate and efficient lung nodule detection via CT imaging, which could help doctors and radiologists in treating lung cancer patients.

Detecting discomfort in infants through facial expressions

OBJECTIVE

Detecting discomfort status of infants is particularly clinically relevant. Late treatment of discomfort infants can lead to adverse problems such as abnormal brain development, central nervous system damage and changes in responsiveness of the neuroendocrine and immune systems to stress at maturity. In this study, we exploit deep convolutional neural network (CNN) algorithms to address the problem of discomfort detection for infants by analyzing their facial expressions.

APPROACH

A dataset of 55 videos about facial expressions, recorded from 24 infants, is used in our study. Given the limited available data for training, we employ a pre-trained CNN model, which is followed by fine-tuning the networks using a public dataset with labeled facial expressions (the shoulder-pain dataset). The CNNs are further refined with our data of infants.

MAIN RESULTS

Using a two-fold cross-validation, we achieve an area under the curve (AUC) value of 0.96, which is substantially higher than the results without any pre-training steps (AUC  =  0.77). Our method also achieves better results than the existing method based on handcrafted features. By fusing individual frame results, the AUC is further improved from 0.96 to 0.98.

SIGNIFICANCE

The proposed system has great potential for continuous discomfort and pain monitoring in clinical practice.