DermViT: Diagnosis-Guided Vision Transformer for Robust and Efficient Skin Lesion Classification

Early diagnosis of skin cancer can significantly improve patient survival. Currently, skin lesion classification faces challenges such as lesion-background semantic entanglement, high intra-class variability, artifactual interference, and more, while existing classification models lack modeling of physicians' diagnostic paradigms. To this end, we propose DermViT, a medically driven deep learning architecture that addresses the above issues through a medically-inspired modular design. DermViT consists of three main modules: (1) Dermoscopic Context Pyramid (DCP), which mimics the multi-scale observation process of pathological diagnosis to adapt to the high intraclass variability of lesions such as melanoma, then extract stable and consistent data at different scales; (2) Dermoscopic Hierarchical Attention (DHA), which can reduce computational complexity while realizing intelligent focusing on lesion areas through a coarse screening-fine inspection mechanism; (3). Dermoscopic Feature Gate (DFG), which simulates the observation-verification operation of doctors through a convolutional gating mechanism and effectively suppresses semantic leakage of artifact regions. Our experimental results show that DermViT significantly outperforms existing methods in terms of classification accuracy (86.12%, a 7.8% improvement over ViT-Base) and number of parameters (40% less than ViT-Base) on the ISIC2018 and ISIC2019 datasets. Our visualization results further validate DermViT's ability to locate lesions under interference conditions. By introducing a modular design that mimics a physician's observation mode, DermViT achieves more logical feature extraction and decision-making processes for medical diagnosis, providing an efficient and reliable solution for dermoscopic image analysis.

RemixFormer++: A Multi-Modal Transformer Model for Precision Skin Tumor Differential Diagnosis With Memory-Efficient Attention

Diagnosing malignant skin tumors accurately at an early stage can be challenging due to ambiguous and even confusing visual characteristics displayed by various categories of skin tumors. To improve diagnosis precision, all available clinical data from multiple sources, particularly clinical images, dermoscopy images, and medical history, could be considered. Aligning with clinical practice, we propose a novel Transformer model, named RemixFormer++ that consists of a clinical image branch, a dermoscopy image branch, and a metadata branch. Given the unique characteristics inherent in clinical and dermoscopy images, specialized attention strategies are adopted for each type. Clinical images are processed through a top-down architecture, capturing both localized lesion details and global contextual information. Conversely, dermoscopy images undergo a bottom-up processing with two-level hierarchical encoders, designed to pinpoint fine-grained structural and textural features. A dedicated metadata branch seamlessly integrates non-visual information by encoding relevant patient data. Fusing features from three branches substantially boosts disease classification accuracy. RemixFormer++ demonstrates exceptional performance on four single-modality datasets (PAD-UFES-20, ISIC 2017/2018/2019). Compared with the previous best method using a public multi-modal Derm7pt dataset, we achieved an absolute 5.3% increase in averaged F1 and 1.2% in accuracy for the classification of five skin tumors. Furthermore, using a large-scale in-house dataset of 10,351 patients with the twelve most common skin tumors, our method obtained an overall classification accuracy of 92.6%. These promising results, on par or better with the performance of 191 dermatologists through a comprehensive reader study, evidently imply the potential clinical usability of our method.

Skin Cancer Image Classification Using Artificial Intelligence Strategies: A Systematic Review

The increasing incidence of and resulting deaths associated with malignant skin tumors are a public health problem that can be minimized if detection strategies are improved. Currently, diagnosis is heavily based on physicians' judgment and experience, which can occasionally lead to the worsening of the lesion or needless biopsies. Several non-invasive imaging modalities, e.g., confocal scanning laser microscopy or multiphoton laser scanning microscopy, have been explored for skin cancer assessment, which have been aligned with different artificial intelligence (AI) strategies to assist in the diagnostic task, based on several image features, thus making the process more reliable and faster. This systematic review concerns the implementation of AI methods for skin tumor classification with different imaging modalities, following the PRISMA guidelines. In total, 206 records were retrieved and qualitatively analyzed. Diagnostic potential was found for several techniques, particularly for dermoscopy images, with strategies yielding classification results close to perfection. Learning approaches based on support vector machines and artificial neural networks seem to be preferred, with a recent focus on convolutional neural networks. Still, detailed descriptions of training/testing conditions are lacking in some reports, hampering reproduction. The use of AI methods in skin cancer diagnosis is an expanding field, with future work aiming to construct optimal learning approaches and strategies. Ultimately, early detection could be optimized, improving patient outcomes, even in areas where healthcare is scarce.

SkinFormer: Learning Statistical Texture Representation With Transformer for Skin Lesion Segmentation

Accurate skin lesion segmentation from dermoscopic images is of great importance for skin cancer diagnosis. However, automatic segmentation of melanoma remains a challenging task because it is difficult to incorporate useful texture representations into the learning process. Texture representations are not only related to the local structural information learned by CNN, but also include the global statistical texture information of the input image. In this paper, we propose a transFormer network (SkinFormer) that efficiently extracts and fuses statistical texture representation for Skin lesion segmentation. Specifically, to quantify the statistical texture of input features, a Kurtosis-guided Statistical Counting Operator is designed. We propose Statistical Texture Fusion Transformer and Statistical Texture Enhance Transformer with the help of Kurtosis-guided Statistical Counting Operator by utilizing the transformer's global attention mechanism. The former fuses structural texture information and statistical texture information, and the latter enhances the statistical texture of multi-scale features. Extensive experiments on three publicly available skin lesion datasets validate that our SkinFormer outperforms other SOAT methods, and our method achieves 93.2% Dice score on ISIC 2018. It can be easy to extend SkinFormer to segment 3D images in the future.

Echoes of images: multi-loss network for image retrieval in vision transformers

This paper introduces a novel approach to enhance content-based image retrieval, validated on two benchmark datasets: ISIC-2017 and ISIC-2018. These datasets comprise skin lesion images that are crucial for innovations in skin cancer diagnosis and treatment. We advocate the use of pre-trained Vision Transformer (ViT), a relatively uncharted concept in the realm of image retrieval, particularly in medical scenarios. In contrast to the traditionally employed Convolutional Neural Networks (CNNs), our findings suggest that ViT offers a more comprehensive understanding of the image context, essential in medical imaging. We further incorporate a weighted multi-loss function, delving into various losses such as triplet loss, distillation loss, contrastive loss, and cross-entropy loss. Our exploration investigates the most resilient combination of these losses to create a robust multi-loss function, thus enhancing the robustness of the learned feature space and ameliorating the precision and recall in the retrieval process. Instead of using all the loss functions, the proposed multi-loss function utilizes the combination of only cross-entropy loss, triplet loss, and distillation loss and gains improvement of 6.52% and 3.45% for mean average precision over ISIC-2017 and ISIC-2018. Another innovation in our methodology is a two-branch network strategy, which concurrently boosts image retrieval and classification. Through our experiments, we underscore the effectiveness and the pitfalls of diverse loss configurations in image retrieval. Furthermore, our approach underlines the advantages of retrieval-based classification through majority voting rather than relying solely on the classification head, leading to enhanced prediction for melanoma - the most lethal type of skin cancer. Our results surpass existing state-of-the-art techniques on the ISIC-2017 and ISIC-2018 datasets by improving mean average precision by 1.01% and 4.36% respectively, emphasizing the efficacy and promise of Vision Transformers paired with our tailor-made weighted loss function, especially in medical contexts. The proposed approach's effectiveness is substantiated through thorough ablation studies and an array of quantitative and qualitative outcomes. To promote reproducibility and support forthcoming research, our source code will be accessible on GitHub.

PDFF-CNN: An attention-guided dynamic multi-orientation feature fusion method for gestational age prediction on imbalanced fetal brain MRI dataset

BACKGROUND

Fetal brain magnetic resonance imaging (MRI)-based gestational age prediction has been widely used to characterize normal fetal brain development and diagnose congenital brain malformations.

PURPOSE

The uncertainty of fetal position and external interference leads to variable localization and direction of the fetal brain. In addition, pregnant women typically concentrate on receiving MRI scans during the fetal anomaly scanning week, leading to an imbalanced distribution of fetal brain MRI data. The above-mentioned problems pose great challenges for deep learning-based fetal brain MRI gestational age prediction.

METHODS

In this study, a pyramid squeeze attention (PSA)-guided dynamic feature fusion CNN (PDFF-CNN) is proposed to robustly predict gestational ages from fetal brain MRI images on an imbalanced dataset. PDFF-CNN contains four components: transformation module, feature extraction module, dynamic feature fusion module, and balanced mean square error (MSE) loss. The transformation and feature extraction modules are employed by using the PSA to learn multiscale and multi-orientation feature representations in a parallel weight-sharing Siamese network. The dynamic feature fusion module automatically learns the weights of feature vectors generated in the feature extraction module to dynamically fuse multiscale and multi-orientation brain sulci and gyri features. Considering the fact of the imbalanced dataset, the balanced MSE loss is used to mitigate the negative impact of imbalanced data distribution on gestational age prediction performance.

RESULTS

Evaluated on an imbalanced fetal brain MRI dataset of 1327 routine clinical T2-weighted MRI images from 157 subjects, PDFF-CNN achieved promising gestational age prediction performance with an overall mean absolute error of 0.848 weeks and anR 2 $R^2$ of 0.904. Furthermore, the attention activation maps of PDFF-CNN were derived, which revealed regional features that contributed to gestational age prediction at each gestational stage.

CONCLUSIONS

These results suggest that the proposed PDFF-CNN might have broad clinical applicability in guiding treatment interventions and delivery planning, which has the potential to be helpful with prenatal diagnosis.

SCANED: Siamese collateral assessment network for evaluation of collaterals from ischemic damage

This study conducts collateral evaluation from ischemic damage using a deep learning-based Siamese network, addressing the challenges associated with a small and imbalanced dataset. The collateral network provides an alternative oxygen and nutrient supply pathway in ischemic stroke cases, influencing treatment decisions. Research in this area focuses on automated collateral assessment using deep learning (DL) methods to expedite decision-making processes and enhance accuracy. Our study employed a 3D ResNet-based Siamese network, referred to as SCANED, to classify collaterals as good/intermediate or poor. Utilizing non-contrast computed tomography (NCCT) images, the network automates collateral identification and assessment by analyzing tissue degeneration around the ischemic site. Relevant features from the left/right hemispheres were extracted, and Euclidean Distance (ED) was employed for similarity measurement. Finally, dichotomized classification of good/intermediate or poor collateral is performed by SCANED using an optimal threshold derived from ROC analysis. SCANED provides a sensitivity of 0.88, a specificity of 0.63, and a weighted F1 score of 0.86 in the dichotomized classification.

Artificial Intelligence Applied to Non-Invasive Imaging Modalities in Identification of Nonmelanoma Skin Cancer: A Systematic Review

BACKGROUND

The objective of this study is to systematically analyze the current state of the literature regarding novel artificial intelligence (AI) machine learning models utilized in non-invasive imaging for the early detection of nonmelanoma skin cancers. Furthermore, we aimed to assess their potential clinical relevance by evaluating the accuracy, sensitivity, and specificity of each algorithm and assessing for the risk of bias.

METHODS

Two reviewers screened the MEDLINE, Cochrane, PubMed, and Embase databases for peer-reviewed studies that focused on AI-based skin cancer classification involving nonmelanoma skin cancers and were published between 2018 and 2023. The search terms included skin neoplasms, nonmelanoma, basal-cell carcinoma, squamous-cell carcinoma, diagnostic techniques and procedures, artificial intelligence, algorithms, computer systems, dermoscopy, reflectance confocal microscopy, and optical coherence tomography. Based on the search results, only studies that directly answered the review objectives were included and the efficacy measures for each were recorded. A QUADAS-2 risk assessment for bias in included studies was then conducted.

RESULTS

A total of 44 studies were included in our review; 40 utilizing dermoscopy, 3 using reflectance confocal microscopy (RCM), and 1 for hyperspectral epidermal imaging (HEI). The average accuracy of AI algorithms applied to all imaging modalities combined was 86.80%, with the same average for dermoscopy. Only one of the three studies applying AI to RCM measured accuracy, with a result of 87%. Accuracy was not measured in regard to AI based HEI interpretation.

CONCLUSION

AI algorithms exhibited an overall favorable performance in the diagnosis of nonmelanoma skin cancer via noninvasive imaging techniques. Ultimately, further research is needed to isolate pooled diagnostic accuracy for nonmelanoma skin cancers as many testing datasets also include melanoma and other pigmented lesions.

CR-Conformer: a fusion network for clinical skin lesion classification

Deep convolutional neural network (DCNN) models have been widely used to diagnose skin lesions, and some of them have achieved diagnostic results comparable to or even better than dermatologists. Most publicly available skin lesion datasets used to train DCNN were dermoscopic images. Expensive dermoscopic equipment is rarely available in rural clinics or small hospitals in remote areas. Therefore, it is of great significance to rely on clinical images for computer-aided diagnosis of skin lesions. This paper proposes an improved dual-branch fusion network called CR-Conformer. It integrates a DCNN branch that can effectively extract local features and a Transformer branch that can extract global features to capture more valuable features in clinical skin lesion images. In addition, we improved the DCNN branch to extract enhanced features in four directions through the convolutional rotation operation, further improving the classification performance of clinical skin lesion images. To verify the effectiveness of our proposed method, we conducted comprehensive tests on a private dataset named XJUSL, which contains ten types of clinical skin lesions. The test results indicate that our proposed method reduced the number of parameters by 11.17 M and improved the accuracy of clinical skin lesion image classification by 1.08%. It has the potential to realize automatic diagnosis of skin lesions in mobile devices.

Precision in Dermatology: Developing an Optimal Feature Selection Framework for Skin Lesion Classification

Melanoma is widely recognized as one of the most lethal forms of skin cancer, with its incidence showing an upward trend in recent years. Nonetheless, the timely detection of this malignancy substantially enhances the likelihood of patients' long-term survival. Several computer-based methods have recently been proposed, in the pursuit of diagnosing skin lesions at their early stages. Despite achieving some level of success, there still remains a margin of error that the machine learning community considers to be an unresolved research challenge. The primary objective of this study was to maximize the input feature information by combining multiple deep models in the first phase, and then to avoid noisy and redundant information by downsampling the feature set, using a novel evolutionary feature selection technique, in the second phase. By maintaining the integrity of the original feature space, the proposed idea generated highly discriminant feature information. Recent deep models, including Darknet53, DenseNet201, InceptionV3, and InceptionResNetV2, were employed in our study, for the purpose of feature extraction. Additionally, transfer learning was leveraged, to enhance the performance of our approach. In the subsequent phase, the extracted feature information from the chosen pre-existing models was combined, with the aim of preserving maximum information, prior to undergoing the process of feature selection, using a novel entropy-controlled gray wolf optimization (ECGWO) algorithm. The integration of fusion and selection techniques was employed, initially to incorporate the feature vector with a high level of information and, subsequently, to eliminate redundant and irrelevant feature information. The effectiveness of our concept is supported by an assessment conducted on three benchmark dermoscopic datasets: PH2, ISIC-MSK, and ISIC-UDA. In order to validate the proposed methodology, a comprehensive evaluation was conducted, including a rigorous comparison to established techniques in the field.

Linear-exponential loss incorporated deep learning for imbalanced classification

The class imbalance issue is a pretty common and enduring topic all the time. When encountering unbalanced data distribution, conventional methods are prone to classify minority samples as majority ones, which may cause severe consequences in reality. It is crucial yet challenging to cope with such problems. In this paper, inspired by our previous work, we borrow the linear-exponential (LINEX) loss function in statistics into deep learning for the first time and extend it into a multi-class form, denoted as DLINEX. Compared with existing loss functions in class imbalance learning (e.g., the weighted cross entropy-loss and the focal loss), DLINEX has an asymmetric geometry interpretation, which can adaptively focus more on the minority and hard-to-classify samples by solely adjusting one parameter. Besides, it simultaneously achieves between and within class diversities via caring about the inherent properties of each instance. As a result, DLINEX achieves 42.08% G-means on the CIFAR-10 dataset at the imbalance ratio of 200, 79.06% G-means on the HAM10000 dataset, 82.74% F1 on the DRIVE dataset, 83.93% F1 on the CHASEDB1 dataset and 79.55% F1 on the STARE dataset The quantitative and qualitative experiments convincingly demonstrate that DLINEX can work favorably in imbalanced classifications, either at the image-level or the pixel-level.

OM-NAS: pigmented skin lesion image classification based on a neural architecture search

Because pigmented skin lesion image classification based on manually designed convolutional neural networks (CNNs) requires abundant experience in neural network design and considerable parameter tuning, we proposed the macro operation mutation-based neural architecture search (OM-NAS) approach in order to automatically build a CNN for image classification of pigmented skin lesions. We first used an improved search space that was oriented toward cells and contained micro and macro operations. The macro operations include InceptionV1, Fire and other well-designed neural network modules. During the search process, an evolutionary algorithm based on macro operation mutation was employed to iteratively change the operation type and connection mode of parent cells so that the macro operation was inserted into the child cell similar to the injection of virus into host DNA. Ultimately, the searched best cells were stacked to build a CNN for the image classification of pigmented skin lesions, which was then assessed on the HAM10000 and ISIC2017 datasets. The test results showed that the CNN built with this approach was more accurate than or almost as accurate as state-of-the-art (SOTA) approaches such as AmoebaNet, InceptionV3 + Attention and ARL-CNN in terms of image classification. The average sensitivity of this method on the HAM10000 and ISIC2017 datasets was 72.4% and 58.5%, respectively.

A survey, review, and future trends of skin lesion segmentation and classification

The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis.

Skin lesion classification based on two-modal images using a multi-scale fully-shared fusion network

BACKGROUND AND OBJECTIVE

Due to the complexity of skin lesion features, computer-aided diagnosis of skin diseases based on multi-modal images is considered a challenging task. Dermoscopic images and clinical images are commonly used to diagnose skin diseases in clinical scenarios, and the complementarity of their features promotes the research of multi-modality classification in the computer-aided diagnosis field. Most current methods focus on the fusion between modalities and ignore the complementary information within each of them, which leads to the loss of the intra-modality relation. Multi-modality models for integrating features both within single modalities and across multiple modalities are limited in the literature. Therefore, a multi-modality model based on dermoscopic and clinical images is proposed to address this issue.

METHODS

We propose a Multi-scale Fully-shared Fusion Network (MFF-Net) that gathers features of dermoscopic images and clinical images for skin lesion classification. In MFF-Net, the multi-scale fusion structure combines deep and shallow features within individual modalities to reduce the loss of spatial information in high-level feature maps. Then Dermo-Clinical Block (DCB) integrates the feature maps from dermoscopic images and clinical images through channel-wise concatenation and using a fully-shared fusion strategy that explores complementary information at different stages.

RESULTS

We validated our model on a four-class two-modal skin diseases dataset, and proved that the proposed multi-scale structure, the fusion module DCBs, and the fully-shared fusion strategy improve the performance of MFF-Net independently. Our method achieved the highest average accuracy of 72.9% on the 7-point checklist dataset, outperforming the state-of-the-art single-modality and multi-modality methods with an accuracy boost of 7.1% and 3.4%, respectively.

CONCLUSIONS

The multi-scale fusion structure demonstrates the significance of intra-modality relations between clinical images and dermoscopic images. The proposed network combined with the multi-scale structure, DCBs, and the fully-shared fusion strategy, can effectively integrate the features of the skin lesions across the two modalities and achieved a promising accuracy among different skin diseases.

Convolutional neural network-based skin image segmentation model to improve classification of skin diseases in conventional and non-standardized picture images

BACKGROUND

For dermatological practices, non-standardized conventional photo images are taken and collected as a mixture of variable fields of the image view, including close-up images focusing on designated lesions and long-shot images including normal skin and background of the body surface. Computer-aided detection/diagnosis (CAD) models trained using non-standardized conventional photo images exhibit lower performance rates than CAD models that detect lesions in a localized small area, such as dermoscopic images.

OBJECTIVE

We aimed to develop a convolutional neural network (CNN) model for skin image segmentation to generate a skin disease image dataset suitable for CAD of multiple skin disease classification.

METHODS

We trained a DeepLabv3 + -based CNN segmentation model to detect skin and lesion areas and segmented out areas that satisfy the following conditions: more than 80% of the image will be the skin area, and more than 10% of the image will be the lesion area.

RESULTS

The generated CNN-segmented image database was examined using CAD of skin disease classification and achieved approximately 90% sensitivity and specificity to differentiate atopic dermatitis from malignant diseases and complications, such as mycosis fungoides, impetigo, and herpesvirus infection. The accuracy of skin disease classification in the CNN-segmented image dataset was almost equal to that of the manually cropped image dataset and higher than that of the original image dataset.

CONCLUSION

Our CNN segmentation model, which automatically extracts lesions and segmented images of the skin area regardless of image fields, will reduce the burden of physician annotation and improve CAD performance.

Toward Multicenter Skin Lesion Classification Using Deep Neural Network With Adaptively Weighted Balance Loss

Recently, deep neural network-based methods have shown promising advantages in accurately recognizing skin lesions from dermoscopic images. However, most existing works focus more on improving the network framework for better feature representation but ignore the data imbalance issue, limiting their flexibility and accuracy across multiple scenarios in multi-center clinics. Generally, different clinical centers have different data distributions, which presents challenging requirements for the network's flexibility and accuracy. In this paper, we divert the attention from framework improvement to the data imbalance issue and propose a new solution for multi-center skin lesion classification by introducing a novel adaptively weighted balance (AWB) loss to the conventional classification network. Benefiting from AWB, the proposed solution has the following advantages: 1) it is easy to satisfy different practical requirements by only changing the backbone; 2) it is user-friendly with no tuning on hyperparameters; and 3) it adaptively enables small intraclass compactness and pays more attention to the minority class. Extensive experiments demonstrate that, compared with solutions equipped with state-of-the-art loss functions, the proposed solution is more flexible and more competent for tackling the multi-center imbalanced skin lesion classification task with considerable performance on two benchmark datasets. In addition, the proposed solution is proved to be effective in handling the imbalanced gastrointestinal disease classification task and the imbalanced DR grading task. Code is available at https://github.com/Weipeishan2021.

An Ensemble of Transfer Learning Models for the Prediction of Skin Cancers with Conditional Generative Adversarial Networks

Skin cancer is one of the most severe forms of the disease, and it can spread to other parts of the body if not detected early. Therefore, diagnosing and treating skin cancer patients at an early stage is crucial. Since a manual skin cancer diagnosis is both time-consuming and expensive, an incorrect diagnosis is made due to the high similarity between the various skin cancers. Improved categorization of multiclass skin cancers requires the development of automated diagnostic systems. Herein, we propose a fully automatic method for classifying several skin cancers by fine-tuning the deep learning models VGG16, ResNet50, and ResNet101. Prior to model creation, the training dataset should undergo data augmentation using traditional image transformation techniques and Generative Adversarial Networks (GANs) to prevent class imbalance issues that may lead to model overfitting. In this study, we investigate the feasibility of creating dermoscopic images that have a realistic appearance using Conditional Generative Adversarial Network (CGAN) techniques. Thereafter, the traditional augmentation methods are used to augment our existing training set to improve the performance of pre-trained deep models on the skin cancer classification task. This improved performance is then compared to the models developed using the unbalanced dataset. In addition, we formed an ensemble of finely tuned transfer learning models, which we trained on balanced and unbalanced datasets. These models were used to make predictions about the data. With appropriate data augmentation, the proposed models attained an accuracy of 92% for VGG16, 92% for ResNet50, and 92.25% for ResNet101, respectively. The ensemble of these models increased the accuracy to 93.5%. A comprehensive discussion on the performance of the models concluded that using this method possibly leads to enhanced performance in skin cancer categorization compared to the efforts made in the past.

Attention Cost-Sensitive Deep Learning-Based Approach for Skin Cancer Detection and Classification

Deep learning-based models have been employed for the detection and classification of skin diseases through medical imaging. However, deep learning-based models are not effective for rare skin disease detection and classification. This is mainly due to the reason that rare skin disease has very a smaller number of data samples. Thus, the dataset will be highly imbalanced, and due to the bias in learning, most of the models give better performances. The deep learning models are not effective in detecting the affected tiny portions of skin disease in the overall regions of the image. This paper presents an attention-cost-sensitive deep learning-based feature fusion ensemble meta-classifier approach for skin cancer detection and classification. Cost weights are included in the deep learning models to handle the data imbalance during training. To effectively learn the optimal features from the affected tiny portions of skin image samples, attention is integrated into the deep learning models. The features from the finetuned models are extracted and the dimensionality of the features was further reduced by using a kernel-based principal component (KPCA) analysis. The reduced features of the deep learning-based finetuned models are fused and passed into ensemble meta-classifiers for skin disease detection and classification. The ensemble meta-classifier is a two-stage model. The first stage performs the prediction of skin disease and the second stage performs the classification by considering the prediction of the first stage as features. Detailed analysis of the proposed approach is demonstrated for both skin disease detection and skin disease classification. The proposed approach demonstrated an accuracy of 99% on skin disease detection and 99% on skin disease classification. In all the experimental settings, the proposed approach outperformed the existing methods and demonstrated a performance improvement of 4% accuracy for skin disease detection and 9% accuracy for skin disease classification. The proposed approach can be used as a computer-aided diagnosis (CAD) tool for the early diagnosis of skin cancer detection and classification in healthcare and medical environments. The tool can accurately detect skin diseases and classify the skin disease into their skin disease family.

AGMR-Net: Attention-guided multiscale recovery framework for stroke segmentation

Automatic and accurate lesion segmentation is critical to the clinical estimation of the lesion status of stroke diseases and appropriate diagnostic systems. Although existing methods have achieved remarkable results, their further adoption is hindered by: (1) intraclass inconsistency, i.e., large variability between different areas of the lesion; and (2) interclass indistinction, in which normal brain tissue resembles the lesion in appearance. To meet these challenges in stroke segmentation, we propose a novel method, namely attention-guided multiscale recovery framework (AGMR-Net) in this paper. Firstly, a coarse-grained patch attention (CPA) module in the encoding is adopted to obtain a patch-based coarse-grained attention map in a multistage, explicitly supervised way, enabling target spatial context saliency representation with a patch-based weighting technique that eliminates the effect of intraclass inconsistency. Secondly, to obtain more detailed boundary partitioning to meet the challenge of interclass indistinction, a newly designed cross-dimensional feature fusion (CFF) module is used to capture global contextual information to further guide the selective aggregation of 2D and 3D features, which can compensate for the lack of boundary learning capability of 2D convolution. Lastly, in the decoding stage, an innovative designed multiscale deconvolution upsampling (MDU) is used for enhanced recovery of target spatial and boundary information. AGMR-Net is evaluated on the open-source dataset Anatomical Tracings of Lesions After Stroke, achieving the highest Dice similarity coefficient of 0.594, Hausdorff distance of 27.005 mm, and average symmetry surface distance of 7.137 mm, which demonstrates that our proposed method outperforms state-of-the-art methods and has great potential for stroke diagnosis.

Dermoscopy lesion classification based on GANs and a fuzzy rank-based ensemble of CNN models

Background and Objective. Skin lesion classification by using deep learning technologies is still a considerable challenge due to high similarity among classes and large intraclass differences, serious class imbalance in data, and poor classification accuracy with low robustness. Approach. To address these issues, a two-stage framework for dermoscopy lesion classification using adversarial training and a fuzzy rank-based ensemble of multilayer feature fusion convolutional neural network (CNN) models is proposed. In the first stage, dermoscopy dataset augmentation based on generative adversarial networks is proposed to obtain realistic dermoscopy lesion images, enabling significant improvement for balancing the number of lesions in each class. In the second stage, a fuzzy rank-based ensemble of multilayer feature fusion CNN models is proposed to classify skin lesions. In addition, an efficient channel integrating spatial attention module, in which a novel dilated pyramid pooling structure is designed to extract multiscale features from an enlarged receptive field and filter meaningful information of the initial features. Combining the cross-entropy loss function with the focal loss function, a novel united loss function is designed to reduce the intraclass sample distance and to focus on difficult and error-prone samples to improve the recognition accuracy of the proposed model. Main results. In this paper, the common dataset (HAM10000) is selected to conduct simulation experiments to evaluate and verify the effectiveness of the proposed method. The subjective and objective experimental results demonstrate that the proposed method is superior over the state-of-the-art methods for skin lesion classification due to its higher accuracy, specificity and robustness. Significance. The proposed method effectively improves the classification performance of the model for skin diseases, which will help doctors make accurate and efficient diagnoses, reduce the incidence rate and improve the survival rates of patients.

Adversarial multimodal fusion with attention mechanism for skin lesion classification using clinical and dermoscopic images

Accurate skin lesion diagnosis requires a great effort from experts to identify the characteristics from clinical and dermoscopic images. Deep multimodal learning-based methods can reduce intra- and inter-reader variability and improve diagnostic accuracy compared to the single modality-based methods. This study develops a novel method, named adversarial multimodal fusion with attention mechanism (AMFAM), to perform multimodal skin lesion classification. Specifically, we adopt a discriminator that uses adversarial learning to enforce the feature extractor to learn the correlated information explicitly. Moreover, we design an attention-based reconstruction strategy to encourage the feature extractor to concentrate on learning the features of the lesion area, thus, enhancing the feature vector from each modality with more discriminative information. Unlike existing multimodal-based approaches, which only focus on learning complementary features from dermoscopic and clinical images, our method considers both correlated and complementary information of the two modalities for multimodal fusion. To verify the effectiveness of our method, we conduct comprehensive experiments on a publicly available multimodal and multi-task skin lesion classification dataset: 7-point criteria evaluation database. The experimental results demonstrate that our proposed method outperforms the current state-of-the-art methods and improves the average AUC score by above 2% on the test set.

Fusion of Local and Global Feature Representation With Sparse Autoencoder for Improved Melanoma Classification

Automated skin cancer diagnosis is challenging due to inter-class uniformity, intra-class variation, and the complex structure of dermoscopy images. Convolutional neural networks (CNN) have recently made considerable progress in melanoma classification, even in the presence of limited skin images. One of the drawbacks of these methods is the loss of image details caused by downsampling high-resolution skin images to a low resolution. Further, most approaches extract features only from the whole skin image. This paper proposes an ensemble feature fusion and sparse autoencoder (SAE) based framework to overcome the above issues and improve melanoma classification performance. The proposed method extracts features from two streams, local and global, using a pre-trained CNN model. The local stream extracts features from image patches, while the global stream derives features from the whole skin image, preserving both local and global representation. The features are then fused, and an SAE framework is subsequently designed to enrich the feature representation further. The proposed method is validated on ISIC 2016 dataset and the experimental results indicate the superiority of the proposed approach.

Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets

Melanoma is a fatal skin cancer that is curable and has dramatically increasing survival rate when diagnosed at early stages. Learning-based methods hold significant promise for the detection of melanoma from dermoscopic images. However, since melanoma is a rare disease, existing databases of skin lesions predominantly contain highly imbalanced numbers of benign versus malignant samples. In turn, this imbalance introduces substantial bias in classification models due to the statistical dominance of the majority class. To address this issue, we introduce a deep clustering approach based on the latent-space embedding of dermoscopic images. Clustering is achieved using a novel center-oriented margin-free triplet loss (COM-Triplet) enforced on image embeddings from a convolutional neural network backbone. The proposed method aims to form maximally-separated cluster centers as opposed to minimizing classification error, so it is less sensitive to class imbalance. To avoid the need for labeled data, we further propose to implement COM-Triplet based on pseudo-labels generated by a Gaussian mixture model (GMM). Comprehensive experiments show that deep clustering with COM-Triplet loss outperforms clustering with triplet loss, and competing classifiers in both supervised and unsupervised settings.

KIEGLFN: A unified acne grading framework on face images

BACKGROUND AND OBJECTIVE

Grading the severity level is an extremely important procedure for correct diagnoses and personalized treatment schemes for acne. However, the acne grading criteria are not unified in the medical field. This work aims to develop an acne diagnosis system that can be generalized to various criteria.

METHODS

A unified acne grading framework that can be generalized to apply referring to different grading criteria is developed. It imitates the global estimation of the dermatologist diagnosis in two steps. First, an adaptive image preprocessing method effectively filters meaningless information and enhances key information. Next, an innovative network structure fuses global deep features with local features to simulate the dermatologists' comparison of local skin and global observation. In addition, a transfer fine-tuning strategy is proposed to transfer prior knowledge on one criterion to another criterion, which effectively improves the framework performance in case of insufficient data.

RESULTS

The Preprocessing method effectively filters meaningless areas and improves the performance of downstream models.The framework reaches accuracies of 84.52% and 59.35% on two datasets separately.

CONCLUSIONS

The application of the framework on acne grading exceeds the state-of-the-art method by 1.71%, reaches the diagnostic level of a professional dermatologist and the transfer fine-tuning strategy improves the accuracy of 6.5% on the small data.

Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification

Deep convolutional neural network (DCNN) models have been widely explored for skin disease diagnosis and some of them have achieved the diagnostic outcomes comparable or even superior to those of dermatologists. However, broad implementation of DCNN in skin disease detection is hindered by small size and data imbalance of the publically accessible skin lesion datasets. This paper proposes a novel single-model based strategy for classification of skin lesions on small and imbalanced datasets. First, various DCNNs are trained on different small and imbalanced datasets to verify that the models with moderate complexity outperform the larger models. Second, regularization DropOut and DropBlock are added to reduce overfitting and a Modified RandAugment augmentation strategy is proposed to deal with the defects of sample underrepresentation in the small dataset. Finally, a novel Multi-Weighted New Loss (MWNL) function and an end-to-end cumulative learning strategy (CLS) are introduced to overcome the challenge of uneven sample size and classification difficulty and to reduce the impact of abnormal samples on training. By combining Modified RandAugment, MWNL and CLS, our single DCNN model method achieved the classification accuracy comparable or superior to those of multiple ensembling models on different dermoscopic image datasets. Our study shows that this method is able to achieve a high classification performance at a low cost of computational resources and inference time, potentially suitable to implement in mobile devices for automated screening of skin lesions and many other malignancies in low resource settings.

Two-Stage Deep Neural Network via Ensemble Learning for Melanoma Classification

Melanoma is a skin disease with a high fatality rate. Early diagnosis of melanoma can effectively increase the survival rate of patients. There are three types of dermoscopy images, malignant melanoma, benign nevis, and seborrheic keratosis, so using dermoscopy images to classify melanoma is an indispensable task in diagnosis. However, early melanoma classification works can only use the low-level information of images, so the melanoma cannot be classified efficiently; the recent deep learning methods mainly depend on a single network, although it can extract high-level features, the poor scale and type of the features limited the results of the classification. Therefore, we need an automatic classification method for melanoma, which can make full use of the rich and deep feature information of images for classification. In this study, we propose an ensemble method that can integrate different types of classification networks for melanoma classification. Specifically, we first use U-net to segment the lesion area of images to generate a lesion mask, thus resize images to focus on the lesion; then, we use five excellent classification models to classify dermoscopy images, and adding squeeze-excitation block (SE block) to models to emphasize the more informative features; finally, we use our proposed new ensemble network to integrate five different classification results. The experimental results prove the validity of our results. We test our method on the ISIC 2017 challenge dataset and obtain excellent results on multiple metrics; especially, we get 0.909 on accuracy. Our classification framework can provide an efficient and accurate way for melanoma classification using dermoscopy images, laying the foundation for early diagnosis and later treatment of melanoma.

Deep metric attention learning for skin lesion classification in dermoscopy images

Currently, convolutional neural networks (CNNs) have made remarkable achievements in skin lesion classification because of their end-to-end feature representation abilities. However, precise skin lesion classification is still challenging because of the following three issues: (1) insufficient training samples, (2) inter-class similarities and intra-class variations, and (3) lack of the ability to focus on discriminative skin lesion parts. To address these issues, we propose a deep metric attention learning CNN (DeMAL-CNN) for skin lesion classification. In DeMAL-CNN, a triplet-based network (TPN) is first designed based on deep metric learning, which consists of three weight-shared embedding extraction networks. TPN adopts a triplet of samples as input and uses the triplet loss to optimize the embeddings, which can not only increase the number of training samples, but also learn the embeddings robust to inter-class similarities and intra-class variations. In addition, a mixed attention mechanism considering both the spatial-wise and channel-wise attention information is designed and integrated into the construction of each embedding extraction network, which can further strengthen the skin lesion localization ability of DeMAL-CNN. After extracting the embeddings, three weight-shared classification layers are used to generate the final predictions. In the training procedure, we combine the triplet loss with the classification loss as a hybrid loss to train DeMAL-CNN. We compare DeMAL-CNN with the baseline method, attention methods, advanced challenge methods, and state-of-the-art skin lesion classification methods on the ISIC 2016 and ISIC 2017 datasets, and test its generalization ability on the PH2 dataset. The results demonstrate its effectiveness.

Efficient structural pseudoinverse learning-based hierarchical representation learning for skin lesion classification

The success of deep learning in skin lesion classification mainly depends on the ultra-deep neural network and the significantly large training data set. Deep learning training is usually time-consuming, and large datasets with labels are hard to obtain, especially skin lesion images. Although pre-training and data augmentation can alleviate these issues, there are still some problems: (1) the data domain is not consistent, resulting in the slow convergence; and (2) low robustness to confusing skin lesions. To solve these problems, we propose an efficient structural pseudoinverse learning-based hierarchical representation learning method. Preliminary feature extraction, shallow network feature extraction and deep learning feature extraction are carried out respectively before the classification of skin lesion images. Gabor filter and pre-trained deep convolutional neural network are used for preliminary feature extraction. The structural pseudoinverse learning (S-PIL) algorithm is used to extract the shallow features. Then, S-PIL preliminarily identifies the skin lesion images that are difficult to be classified to form a new training set for deep learning feature extraction. Through the hierarchical representation learning, we analyze the features of skin lesion images layer by layer to improve the final classification. Our method not only avoid the slow convergence caused by inconsistency of data domain but also enhances the training of confusing examples. Without using additional data, our approach outperforms existing methods in the ISIC 2017 and ISIC 2018 datasets.

Gastrointestinal Disease Classification in Endoscopic Images Using Attention-Guided Convolutional Neural Networks

Gastrointestinal (GI) diseases constitute a leading problem in the human digestive system. Consequently, several studies have explored automatic classification of GI diseases as a means of minimizing the burden on clinicians and improving patient outcomes, for both diagnostic and treatment purposes. The challenge in using deep learning-based (DL) approaches, specifically a convolutional neural network (CNN), is that spatial information is not fully utilized due to the inherent mechanism of CNNs. This paper proposes the application of spatial factors in improving classification performance. Specifically, we propose a deep CNN-based spatial attention mechanism for the classification of GI diseases, implemented with encoder–decoder layers. To overcome the data imbalance problem, we adapt data-augmentation techniques. A total of 12,147 multi-sited, multi-diseased GI images, drawn from publicly available and private sources, were used to validate the proposed approach. Furthermore, a five-fold cross-validation approach was adopted to minimize inconsistencies in intra- and inter-class variability and to ensure that results were robustly assessed. Our results, compared with other state-of-the-art models in terms of mean accuracy (ResNet50 = 90.28, GoogLeNet = 91.38, DenseNets = 91.60, and baseline = 92.84), demonstrated better outcomes (Precision = 92.8, Recall = 92.7, F1-score = 92.8, and Accuracy = 93.19). We also implemented t-distributed stochastic neighbor embedding (t–SNE) and confusion matrix analysis techniques for better visualization and performance validation. Overall, the results showed that the attention mechanism improved the automatic classification of multi-sited GI disease images. We validated clinical tests based on the proposed method by overcoming previous limitations, with the goal of improving automatic classification accuracy in future work.

Vocal cord lesions classification based on deep convolutional neural network and transfer learning

PURPOSE

Laryngoscopy, the most common diagnostic method for vocal cord lesions (VCLs), is based mainly on the visual subjective inspection of otolaryngologists. This study aimed to establish a highly objective computer-aided VCLs diagnosis system based on deep convolutional neural network (DCNN) and transfer learning.

METHODS

To classify VCLs, our method combined the DCNN backbone with transfer learning on a system specifically finetuned for a laryngoscopy image dataset. Laryngoscopy image database was collected to train the proposed system. The diagnostic performance was compared with other DCNN-based models. Analysis of F1 score and receiver operating characteristic curves were conducted to evaluate the performance of the system.

RESULTS

Beyond the existing VCLs diagnosis method, the proposed system achieved an overall accuracy of 80.23%, an F1 score of 0.7836, and an area under the curve (AUC) of 0.9557 for four fine-grained classes of VCLs, namely, normal, polyp, keratinization, and carcinoma. It also demonstrated robust classification capacity for detecting urgent (keratinization, carcinoma) and non-urgent (normal, polyp), with an overall accuracy of 0.939, a sensitivity of 0.887, a specificity of 0.993, and an AUC of 0.9828. The proposed method also outperformed clinicians in the classification of normal, polyps, and carcinoma at an extremely low time cost.

CONCLUSION

The VCLs diagnosis system succeeded in using DCNN to distinguish the most common VCLs and normal cases, holding a practical potential for improving the overall diagnostic efficacy in VCLs examinations. The proposed VCLs diagnosis system could be appropriately integrated into the conventional workflow of VCLs laryngoscopy as a highly objective auxiliary method.

Deep attention branch networks for skin lesion classification

BACKGROUND AND OBJECTIVE

The skin lesion usually covers a small region of the dermoscopy image, and the lesions of different categories might own high similarities. Therefore, it is essential to design an elaborate network for accurate skin lesion classification, which can focus on semantically meaningful lesion parts. Although the Class Activation Mapping (CAM) shows good localization capability of highlighting the discriminative parts, it cannot be obtained in the forward propagation process.

METHODS

We propose a Deep Attention Branch Network (DABN) model, which introduces the attention branches to expand the conventional Deep Convolutional Neural Networks (DCNN). The attention branch is designed to obtain the CAM in the training stage, which is then utilized as an attention map to make the network focus on discriminative parts of skin lesions. DABN is applicable to multiple DCNN structures and can be trained in an end-to-end manner. Moreover, a novel Entropy-guided Loss Weighting (ELW) strategy is designed to counter class imbalance influence in the skin lesion datasets.

RESULTS

The proposed method achieves an Average Precision (AP) of 0.719 on the ISIC-2016 dataset and an average area under the ROC curve (AUC) of 0.922 on the ISIC-2017 dataset. Compared with other state-of-the-art methods, our method obtains better performance without external data and ensemble learning. Moreover, extensive experiments demonstrate that it can be applied to multi-class classification tasks and improves mean sensitivity by more than 2.6% in different DCNN structures.

CONCLUSIONS

The proposed method can adaptively focus on the discriminative regions of dermoscopy images and allows for effective training when facing class imbalance, leading to the performance improvement of skin lesion classification, which could also be applied to other clinical applications.

Melanoma classification using light-Fields with morlet scattering transform and CNN: Surface depth as a valuable tool to increase detection rate

Medical image classification through learning-based approaches has been increasingly used, namely in the discrimination of melanoma. However, for skin lesion classification in general, such methods commonly rely on dermoscopic or other 2D-macro RGB images. This work proposes to exploit beyond conventional 2D image characteristics, by considering a third dimension (depth) that characterises the skin surface rugosity, which can be obtained from light-field images, such as those available in the SKINL2 dataset. To achieve this goal, a processing pipeline was deployed using a morlet scattering transform and a CNN model, allowing to perform a comparison between using 2D information, only 3D information, or both. Results show that discrimination between Melanoma and Nevus reaches an accuracy of 84.00, 74.00 or 94.00% when using only 2D, only 3D, or both, respectively. An increase of 14.29pp in sensitivity and 8.33pp in specificity is achieved when expanding beyond conventional 2D information by also using depth. When discriminating between Melanoma and all other types of lesions (a further imbalanced setting), an increase of 28.57pp in sensitivity and decrease of 1.19pp in specificity is achieved for the same test conditions. Overall the results of this work demonstrate significant improvements over conventional approaches.

Comprehensive Assessment of Fine-Grained Wound Images Using a Patch-Based CNN With Context-Preserving Attention

Goal: Chronic wounds affect 6.5 million Americans. Wound assessment via algorithmic analysis of smartphone images has emerged as a viable option for remote assessment. Methods: We comprehensively score wounds based on the clinically-validated Photographic Wound Assessment Tool (PWAT), which comprehensively assesses clinically important ranges of eight wound attributes: Size, Depth, Necrotic Tissue Type, Necrotic Tissue Amount, Granulation Tissue type, Granulation Tissue Amount, Edges, Periulcer Skin Viability. We proposed a DenseNet Convolutional Neural Network (CNN) framework with patch-based context-preserving attention to assess the 8 PWAT attributes of four wound types: diabetic ulcers, pressure ulcers, vascular ulcers and surgical wounds. Results: In an evaluation on our dataset of 1639 wound images, our model estimated all 8 PWAT sub-scores with classification accuracies and F1 scores of over 80%. Conclusions: Our work is the first intelligent system that autonomously grades wounds comprehensively based on criteria in the PWAT rubric, alleviating the significant burden that manual wound grading imposes on wound care nurses.

AI outperformed every dermatologist in dermoscopic melanoma diagnosis, using an optimized deep-CNN architecture with custom mini-batch logic and loss function

Melanoma, one of the most dangerous types of skin cancer, results in a very high mortality rate. Early detection and resection are two key points for a successful cure. Recent researches have used artificial intelligence to classify melanoma and nevus and to compare the assessment of these algorithms to that of dermatologists. However, training neural networks on an imbalanced dataset leads to imbalanced performance, the specificity is very high but the sensitivity is very low. This study proposes a method for improving melanoma prediction on an imbalanced dataset by reconstructed appropriate CNN architecture and optimized algorithms. The contributions involve three key features as custom loss function, custom mini-batch logic, and reformed fully connected layers. In the experiment, the training dataset is kept up to date including 17,302 images of melanoma and nevus which is the largest dataset by far. The model performance is compared to that of 157 dermatologists from 12 university hospitals in Germany based on the same dataset. The experimental results prove that our proposed approach outperforms all 157 dermatologists and achieves higher performance than the state-of-the-art approach with area under the curve of 94.4%, sensitivity of 85.0%, and specificity of 95.0%. Moreover, using the best threshold shows the most balanced measure compare to other researches, and is promisingly application to medical diagnosis, with sensitivity of 90.0% and specificity of 93.8%. To foster further research and allow for replicability, we made the source code and data splits of all our experiments publicly available.

Effective diagnostic model construction based on discriminative breast ultrasound image regions using deep feature extraction

PURPOSE

This research aims to analyze the diagnostic contribution of different discriminative regions of the breast ultrasound image and develop a more effective diagnosis method taking advantage of the discriminative regions' complementarity.

METHODS

First, the discriminative regions of the original breast ultrasound image as the inner region of the lesion, the marginal zone of the lesion, and the posterior echo region of the lesion were defined. The pretrained Inception-V3 network was used to analyze the diagnostic contribution of these discriminative regions. Then, the network was applied to extract the deep features of the original image and the other three discriminative region images. Since there are many features, principal components analysis (PCA) was used to reduce the dimensionality of the extracted deep features. The selected deep features from different discriminative regions were fused to original image features and sent to the stacking ensemble learning classifier for classification experiments. In this study, 479 cases of breast ultrasound images, including 356 benign lesions and 123 malignant ones, were collected retrospectively and randomly divided into the training and validation set.

RESULTS

Experimental results show that by using Inception-V3, the diagnostic performance of each discriminative region is different, and the diagnostic accuracy and the area under the ROC curve (AUC) of the lesion marginal zone image (78.3%, 0.798) are higher than those of the lesion inner region image (73.3%, 0.763) and the posterior echo region image (71.7%, 0.688), but lower than those of the original image (80.0%, 0.817). Furthermore, the best classification performance was obtained when all the four types of deep features (from the original image and three discriminative region images) were fused, and the ensemble learning for classification evaluation was employed. Compared with the original image, the classification accuracy and AUC increased from 80.83%, 0.818 to 85.00%, 0.872, and the classification sensitivity and specificity varied from 0.710, 0.798 to 0.871, 0.787.

CONCLUSIONS

The inner region of the lesion, the marginal zone of the lesion, and the posterior echo region of the lesion play significant roles in the diagnosis of the breast ultrasound image. Deep feature fusion of these three kinds of images and the original image can effectively improve the accuracy of diagnosis.

CMC: A consensus multi-view clustering model for predicting Alzheimer's disease progression

Machine learning has been used in the past for the auxiliary diagnosis of Alzheimer's Disease (AD). However, most existing technologies only explore single-view data, require manual parameter setting and focus on two-class (i.e., dementia or not) classification problems. Unlike single-view data, multi-view data provide more powerful feature representation capability. Learning with multi-view data is referred to as multi-view learning, which has received certain attention in recent years. In this paper, we propose a new multi-view clustering model called Consensus Multi-view Clustering (CMC) based on nonnegative matrix factorization for predicting the multiple stages of AD progression. The proposed CMC performs multi-view learning idea to fully capture data features with limited medical images, approaches similarity relations between different entities, addresses the shortcoming from multi-view fusion that requires manual setting parameters, and further acquires a consensus representation containing shared features and complementary knowledge of multiple view data. It not only can improve the predication performance of AD, but also can screen and classify the symptoms of different AD's phases. Experimental results using data with twelve views constructed by brain Magnetic Resonance Imaging (MRI) database from Alzheimer's Disease Neuroimaging Initiative expound and prove the effectiveness of the proposed model.

Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images

As an analytic tool in medicine, deep learning has gained great attention and opened new ways for disease diagnosis. Recent studies validate the effectiveness of deep learning algorithms for binary classification of skin lesions (i.e., melanomas and nevi classes) with dermoscopic images. Nonetheless, those binary classification methods cannot be applied to the general clinical situation of skin cancer screening in which multi-class classification must be taken into account. The main objective of this research is to develop, implement, and calibrate an advanced deep learning model in the context of automated multi-class classification of skin lesions. The proposed Deep Convolutional Neural Network (DCNN) model is carefully designed with several layers, and multiple filter sizes, but fewer filters and parameters to improve efficacy and performance. Dermoscopic images are acquired from the International Skin Imaging Collaboration databases (ISIC-17, ISIC-18, and ISIC-19) for experiments. The experimental results of the proposed DCNN approach are presented in terms of precision, sensitivity, specificity, and other metrics. Specifically, it attains 94 % precision, 93 % sensitivity, and 91 % specificity in ISIC-17. It is demonstrated by the experimental results that this proposed DCNN approach outperforms state-of-the-art algorithms, exhibiting 0.964 area under the receiver operating characteristics (AUROC) in ISIC-17 for the classification of skin lesions and can be used to assist dermatologists in classifying skin lesions. As a result, this proposed approach provides a novel and feasible way for automating and expediting the skin lesion classification task as well as saving effort, time, and human life.

Skin Lesion Classification Using Densely Connected Convolutional Networks with Attention Residual Learning

Skin lesion classification is an effective approach aided by computer vision for the diagnosis of skin cancer. Though deep learning models presented advantages over traditional methods and brought tremendous breakthroughs, a precise diagnosis is still challenging because of the intra-class variation and inter-class similarity caused by the diversity of imaging methods and clinicopathology. In this paper, we propose a densely connected convolutional network with an attention and residual learning (ARDT-DenseNet) method for skin lesion classification. Each ARDT block consists of dense blocks, transition blocks and attention and residual modules. Compared to a residual network with the same number of convolutional layers, the size of the parameters of the densely connected network proposed in this paper has been reduced by half, while the accuracy of skin lesion classification is preserved. Our improved densely connected network adds an attention mechanism and residual learning after each dense block and transition block without introducing additional parameters. We evaluate the ARDT-DenseNet model with the ISIC 2016 and ISIC 2017 datasets. Our method achieves an ACC of 85.7% and an AUC of 83.7% in skin lesion classification with ISIC 2016 and an average AUC of 91.8% in skin lesion classification with ISIC 2017. The experimental results show that the method proposed in this paper has achieved a significant improvement in skin lesion classification, which is superior to that of the state-of-the-art method.

Ensemble of ROI-based convolutional neural network classifiers for staging the Alzheimer disease spectrum from magnetic resonance imaging

Patches from three orthogonal views of selected cerebral regions can be utilized to learn convolutional neural network (CNN) models for staging the Alzheimer disease (AD) spectrum including preclinical AD, mild cognitive impairment due to AD, and dementia due to AD and normal controls. Hippocampi, amygdalae and insulae were selected from the volumetric analysis of structured magnetic resonance images (MRIs). Three-view patches (TVPs) from these regions were fed to the CNN for training. MRIs were classified with the SoftMax-normalized scores of individual model predictions on TVPs. The significance of each region of interest (ROI) for staging the AD spectrum was evaluated and reported. The results of the ensemble classifier are compared with state-of-the-art methods using the same evaluation metrics. Patch-based ROI ensembles provide comparable diagnostic performance for AD staging. In this work, TVP-based ROI analysis using a CNN provides informative landmarks in cerebral MRIs and may have significance in clinical studies and computer-aided diagnosis system design.

Effective Melanoma Recognition Using Deep Convolutional Neural Network with Covariance Discriminant Loss

Melanoma recognition is challenging due to data imbalance and high intra-class variations and large inter-class similarity. Aiming at the issues, we propose a melanoma recognition method using deep convolutional neural network with covariance discriminant loss in dermoscopy images. Deep convolutional neural network is trained under the joint supervision of cross entropy loss and covariance discriminant loss, rectifying the model outputs and the extracted features simultaneously. Specifically, we design an embedding loss, namely covariance discriminant loss, which takes the first and second distance into account simultaneously for providing more constraints. By constraining the distance between hard samples and minority class center, the deep features of melanoma and non-melanoma can be separated effectively. To mine the hard samples, we also design the corresponding algorithm. Further, we analyze the relationship between the proposed loss and other losses. On the International Symposium on Biomedical Imaging (ISBI) 2018 Skin Lesion Analysis dataset, the two schemes in the proposed method can yield a sensitivity of 0.942 and 0.917, respectively. The comprehensive results have demonstrated the efficacy of the designed embedding loss and the proposed methodology.