Attention-Guided Learning With Feature Reconstruction for Skin Lesion Diagnosis Using Clinical and Ultrasound Images

Skin lesion is one of the most common diseases, and most categories are highly similar in morphology and appearance. Deep learning models effectively reduce the variability between classes and within classes, and improve diagnostic accuracy. However, the existing multi-modal methods are only limited to the surface information of lesions in skin clinical and dermatoscopic modalities, which hinders the further improvement of skin lesion diagnostic accuracy. This requires us to further study the depth information of lesions in skin ultrasound. In this paper, we propose a novel skin lesion diagnosis network, which combines clinical and ultrasound modalities to fuse the surface and depth information of the lesion to improve diagnostic accuracy. Specifically, we propose an attention-guided learning (AL) module that fuses clinical and ultrasound modalities from both local and global perspectives to enhance feature representation. The AL module consists of two parts, attention-guided local learning (ALL) computes the intra-modality and inter-modality correlations to fuse multi-scale information, which makes the network focus on the local information of each modality, and attention-guided global learning (AGL) fuses global information to further enhance the feature representation. In addition, we propose a feature reconstruction learning (FRL) strategy which encourages the network to extract more discriminative features and corrects the focus of the network to enhance the model's robustness and certainty. We conduct extensive experiments and the results confirm the superiority of our proposed method. Our code is available at: https://github.com/XCL-hub/AGFnet.

AI in dermatology: a comprehensive review into skin cancer detection

Background

Artificial Intelligence (AI) is significantly transforming dermatology, particularly in early skin cancer detection and diagnosis. This technological advancement addresses a crucial public health issue by enhancing diagnostic accuracy, efficiency, and accessibility. AI integration in medical imaging and diagnostic procedures offers promising solutions to the limitations of traditional methods, which often rely on subjective clinical evaluations and histopathological analyses. This study systematically reviews current AI applications in skin cancer classification, providing a comprehensive overview of their advantages, challenges, methodologies, and functionalities.

Methodology

In this study, we conducted a comprehensive analysis of artificial intelligence (AI) applications in the classification of skin cancer. We evaluated publications from three prominent journal databases: Scopus, IEEE, and MDPI. We conducted a thorough selection process using the PRISMA guidelines, collecting 1,156 scientific articles. Our methodology included evaluating the titles and abstracts and thoroughly examining the full text to determine their relevance and quality. Consequently, we included a total of 95 publications in the final study. We analyzed and categorized the articles based on four key dimensions: advantages, difficulties, methodologies, and functionalities.

Results

AI-based models exhibit remarkable performance in skin cancer detection by leveraging advanced deep learning algorithms, image processing techniques, and feature extraction methods. The advantages of AI integration include significantly improved diagnostic accuracy, faster turnaround times, and increased accessibility to dermatological expertise, particularly benefiting underserved areas. However, several challenges remain, such as concerns over data privacy, complexities in integrating AI systems into existing workflows, and the need for large, high-quality datasets. AI-based methods for skin cancer detection, including CNNs, SVMs, and ensemble learning techniques, aim to improve lesion classification accuracy and increase early detection. AI systems enhance healthcare by enabling remote consultations, continuous patient monitoring, and supporting clinical decision-making, leading to more efficient care and better patient outcomes.

Conclusions

This comprehensive review highlights the transformative potential of AI in dermatology, particularly in skin cancer detection and diagnosis. While AI technologies have significantly improved diagnostic accuracy, efficiency, and accessibility, several challenges remain. Future research should focus on ensuring data privacy, developing robust AI systems that can generalize across diverse populations, and creating large, high-quality datasets. Integrating AI tools into clinical workflows is critical to maximizing their utility and effectiveness. Continuous innovation and interdisciplinary collaboration will be essential for fully realizing the benefits of AI in skin cancer detection and diagnosis.

Using AI to Differentiate Mpox From Common Skin Lesions in a Sexual Health Clinic: Algorithm Development and Validation Study

BACKGROUND

The 2022 global outbreak of mpox has significantly impacted health facilities, and necessitated additional infection prevention and control measures and alterations to clinic processes. Early identification of suspected mpox cases will assist in mitigating these impacts.

OBJECTIVE

We aimed to develop and evaluate an artificial intelligence (AI)-based tool to differentiate mpox lesion images from other skin lesions seen in a sexual health clinic.

METHODS

We used a data set with 2200 images, that included mpox and non-mpox lesions images, collected from Melbourne Sexual Health Centre and web resources. We adopted deep learning approaches which involved 6 different deep learning architectures to train our AI models. We subsequently evaluated the performance of each model using a hold-out data set and an external validation data set to determine the optimal model for differentiating between mpox and non-mpox lesions.

RESULTS

The DenseNet-121 model outperformed other models with an overall area under the receiver operating characteristic curve (AUC) of 0.928, an accuracy of 0.848, a precision of 0.942, a recall of 0.742, and an F1-score of 0.834. Implementation of a region of interest approach significantly improved the performance of all models, with the AUC for the DenseNet-121 model increasing to 0.982. This approach resulted in an increase in the correct classification of mpox images from 79% (55/70) to 94% (66/70). The effectiveness of this approach was further validated by a visual analysis with gradient-weighted class activation mapping, demonstrating a reduction in false detection within the background of lesion images. On the external validation data set, ResNet-18 and DenseNet-121 achieved the highest performance. ResNet-18 achieved an AUC of 0.990 and an accuracy of 0.947, and DenseNet-121 achieved an AUC of 0.982 and an accuracy of 0.926.

CONCLUSIONS

Our study demonstrated it was possible to use an AI-based image recognition algorithm to accurately differentiate between mpox and common skin lesions. Our findings provide a foundation for future investigations aimed at refining the algorithm and establishing the place of such technology in a sexual health clinic.

All you need is data preparation: A systematic review of image harmonization techniques in Multi-center/device studies for medical support systems

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) models trained on multi-centric and multi-device studies can provide more robust insights and research findings compared to single-center studies. However, variability in acquisition protocols and equipment can introduce inconsistencies that hamper the effective pooling of multi-source datasets. This systematic review evaluates strategies for image harmonization, which standardizes appearances to enable reliable AI analysis of multi-source medical imaging.

METHODS

A literature search using PRISMA guidelines was conducted to identify relevant papers published between 2013 and 2023 analyzing multi-centric and multi-device medical imaging studies that utilized image harmonization approaches.

RESULTS

Common image harmonization techniques included grayscale normalization (improving classification accuracy by up to 24.42 %), resampling (increasing the percentage of robust radiomics features from 59.5 % to 89.25 %), and color normalization (enhancing AUC by up to 0.25 in external test sets). Initially, mathematical and statistical methods dominated, but machine and deep learning adoption has risen recently. Color imaging modalities like digital pathology and dermatology have remained prominent application areas, though harmonization efforts have expanded to diverse fields including radiology, nuclear medicine, and ultrasound imaging. In all the modalities covered by this review, image harmonization improved AI performance, with increasing of up to 24.42 % in classification accuracy and 47 % in segmentation Dice scores.

CONCLUSIONS

Continued progress in image harmonization represents a promising strategy for advancing healthcare by enabling large-scale, reliable analysis of integrated multi-source datasets using AI. Standardizing imaging data across clinical settings can help realize personalized, evidence-based care supported by data-driven technologies while mitigating biases associated with specific populations or acquisition protocols.

A novel multi-task learning network for skin lesion classification based on multi-modal clues and label-level fusion

In this paper, we propose a multi-task learning (MTL) network based on the label-level fusion of metadata and hand-crafted features by unsupervised clustering to generate new clustering labels as an optimization goal. We propose a MTL module (MTLM) that incorporates an attention mechanism to enable the model to learn more integrated, variable information. We propose a dynamic strategy to adjust the loss weights of different tasks, and trade off the contributions of multiple branches. Instead of feature-level fusion, we propose label-level fusion and combine the results of our proposed MTLM with the results of the image classification network to achieve better lesion prediction on multiple dermatological datasets. We verify the effectiveness of the proposed model by quantitative and qualitative measures. The MTL network using multi-modal clues and label-level fusion can yield the significant performance improvement for skin lesion classification.

Semi-supervised skin cancer diagnosis based on self-feedback threshold focal learning

Worldwide, skin cancer prevalence necessitates accurate diagnosis to alleviate public health burdens. Although the application of artificial intelligence in image analysis and pattern recognition has improved the accuracy and efficiency of early skin cancer diagnosis, existing supervised learning methods are limited due to their reliance on a large amount of labeled data. To overcome the limitations of data labeling and enhance the performance of diagnostic models, this study proposes a semi-supervised skin cancer diagnostic model based on Self-feedback Threshold Focal Learning (STFL), capable of utilizing partial labeled and a large scale of unlabeled medical images for training models in unseen scenarios. The proposed model dynamically adjusts the selection threshold of unlabeled samples during training, effectively filtering reliable unlabeled samples and using focal learning to mitigate the impact of class imbalance in further training. The study is experimentally validated on the HAM10000 dataset, which includes images of various types of skin lesions, with experiments conducted across different scales of labeled samples. With just 500 annotated samples, the model demonstrates robust performance (0.77 accuracy, 0.6408 Kappa, 0.77 recall, 0.7426 precision, and 0.7462 F1-score), showcasing its efficiency with limited labeled data. Further, comprehensive testing validates the semi-supervised model's significant advancements in diagnostic accuracy and efficiency, underscoring the value of integrating unlabeled data. This model offers a new perspective on medical image processing and contributes robust scientific support for the early diagnosis and treatment of skin cancer.

QGD-Net: A Lightweight Model Utilizing Pixels of Affinity in Feature Layer for Dermoscopic Lesion Segmentation

RESPONSE

Pixels with location affinity, which can be also called "pixels of affinity," have similar semantic information. Group convolution and dilated convolution can utilize them to improve the capability of the model. However, for group convolution, it does not utilize pixels of affinity between layers. For dilated convolution, after multiple convolutions with the same dilated rate, the pixels utilized within each layer do not possess location affinity with each other. To solve the problem of group convolution, our proposed quaternion group convolution uses the quaternion convolution, which promotes the communication between to promote utilizing pixels of affinity between channels. In quaternion group convolution, the feature layers are divided into 4 layers per group, ensuring the quaternion convolution can be performed. To solve the problem of dilated convolution, we propose the quaternion sawtooth wave-like dilated convolutions module (QS module). QS module utilizes quaternion convolution with sawtooth wave-like dilated rates to effectively leverage the pixels that share the location affinity both between and within layers. This allows for an expanded receptive field, ultimately enhancing the performance of the model. In particular, we perform our quaternion group convolution in QS module to design the quaternion group dilated neutral network (QGD-Net). Extensive experiments on Dermoscopic Lesion Segmentation based on ISIC 2016 and ISIC 2017 indicate that our method has significantly reduced the model parameters and highly promoted the precision of the model in Dermoscopic Lesion Segmentation. And our method also shows generalizability in retinal vessel segmentation.

Consistent representation via contrastive learning for skin lesion diagnosis

BACKGROUND

Skin lesions are a prevalent ailment, with melanoma emerging as a particularly perilous variant. Encouragingly, artificial intelligence displays promising potential in early detection, yet its integration within clinical contexts, particularly involving multi-modal data, presents challenges. While multi-modal approaches enhance diagnostic efficacy, the influence of modal bias is often disregarded.

METHODS

In this investigation, a multi-modal feature learning technique termed "Contrast-based Consistent Representation Disentanglement" for dermatological diagnosis is introduced. This approach employs adversarial domain adaptation to disentangle features from distinct modalities, fostering a shared representation. Furthermore, a contrastive learning strategy is devised to incentivize the model to preserve uniformity in common lesion attributes across modalities. Emphasizing the learning of a uniform representation among models, this approach circumvents reliance on supplementary data.

RESULTS

Assessment of the proposed technique on a 7-point criteria evaluation dataset yields an average accuracy of 76.1% for multi-classification tasks, surpassing researched state-of-the-art methods. The approach tackles modal bias, enabling the acquisition of a consistent representation of common lesion appearances across diverse modalities, which transcends modality boundaries. This study underscores the latent potential of multi-modal feature learning in dermatological diagnosis.

CONCLUSION

In summation, a multi-modal feature learning strategy is posited for dermatological diagnosis. This approach outperforms other state-of-the-art methods, underscoring its capacity to enhance diagnostic precision for skin lesions.

An Efficient Artificial Rabbits Optimization Based on Mutation Strategy For Skin Cancer Prediction

Accurate skin lesion diagnosis is critical for the early detection of melanoma. However, the existing approaches are unable to attain substantial levels of accuracy. Recently, pre-trained Deep Learning (DL) models have been applied to tackle and improve efficiency on tasks such as skin cancer detection instead of training models from scratch. Therefore, we develop a robust model for skin cancer detection with a DL-based model as a feature extraction backbone, which is achieved using MobileNetV3 architecture. In addition, a novel algorithm called the Improved Artificial Rabbits Optimizer (IARO) is introduced, which uses the Gaussian mutation and crossover operator to ignore the unimportant features from those features extracted using MobileNetV3. The PH2, ISIC-2016, and HAM10000 datasets are used to validate the developed approach's efficiency. The empirical results show that the developed approach yields outstanding accuracy results of 87.17% on the ISIC-2016 dataset, 96.79% on the PH2 dataset, and 88.71 % on the HAM10000 dataset. Experiments show that the IARO can significantly improve the prediction of skin cancer.

Assist-Dermo: A Lightweight Separable Vision Transformer Model for Multiclass Skin Lesion Classification

A dermatologist-like automatic classification system is developed in this paper to recognize nine different classes of pigmented skin lesions (PSLs), using a separable vision transformer (SVT) technique to assist clinical experts in early skin cancer detection. In the past, researchers have developed a few systems to recognize nine classes of PSLs. However, they often require enormous computations to achieve high performance, which is burdensome to deploy on resource-constrained devices. In this paper, a new approach to designing SVT architecture is developed based on SqueezeNet and depthwise separable CNN models. The primary goal is to find a deep learning architecture with few parameters that has comparable accuracy to state-of-the-art (SOTA) architectures. This paper modifies the SqueezeNet design for improved runtime performance by utilizing depthwise separable convolutions rather than simple conventional units. To develop this Assist-Dermo system, a data augmentation technique is applied to control the PSL imbalance problem. Next, a pre-processing step is integrated to select the most dominant region and then enhance the lesion patterns in a perceptual-oriented color space. Afterwards, the Assist-Dermo system is designed to improve efficacy and performance with several layers and multiple filter sizes but fewer filters and parameters. For the training and evaluation of Assist-Dermo models, a set of PSL images is collected from different online data sources such as Ph2, ISBI-2017, HAM10000, and ISIC to recognize nine classes of PSLs. On the chosen dataset, it achieves an accuracy (ACC) of 95.6%, a sensitivity (SE) of 96.7%, a specificity (SP) of 95%, and an area under the curve (AUC) of 0.95. The experimental results show that the suggested Assist-Dermo technique outperformed SOTA algorithms when recognizing nine classes of PSLs. The Assist-Dermo system performed better than other competitive systems and can support dermatologists in the diagnosis of a wide variety of PSLs through dermoscopy. The Assist-Dermo model code is freely available on GitHub for the scientific community.

Optimal Skin Cancer Detection Model Using Transfer Learning and Dynamic-Opposite Hunger Games Search

Recently, pre-trained deep learning (DL) models have been employed to tackle and enhance the performance on many tasks such as skin cancer detection instead of training models from scratch. However, the existing systems are unable to attain substantial levels of accuracy. Therefore, we propose, in this paper, a robust skin cancer detection framework for to improve the accuracy by extracting and learning relevant image representations using a MobileNetV3 architecture. Thereafter, the extracted features are used as input to a modified Hunger Games Search (HGS) based on Particle Swarm Optimization (PSO) and Dynamic-Opposite Learning (DOLHGS). This modification is used as a novel feature selection to alloacte the most relevant feature to maximize the model's performance. For evaluation of the efficiency of the developed DOLHGS, the ISIC-2016 dataset and the PH2 dataset were employed, including two and three categories, respectively. The proposed model has accuracy 88.19% on the ISIC-2016 dataset and 96.43% on PH2. Based on the experimental results, the proposed approach showed more accurate and efficient performance in skin cancer detection than other well-known and popular algorithms in terms of classification accuracy and optimized features.

Skin lesion segmentation using two-phase cross-domain transfer learning framework

BACKGROUND AND OBJECTIVE

Deep learning (DL) models have been used for medical imaging for a long time but they did not achieve their full potential in the past because of insufficient computing power and scarcity of training data. In recent years, we have seen substantial growth in DL networks because of improved technology and an abundance of data. However, previous studies indicate that even a well-trained DL algorithm may struggle to generalize data from multiple sources because of domain shifts. Additionally, ineffectiveness of basic data fusion methods, complexity of segmentation target and low interpretability of current DL models limit their use in clinical decisions. To meet these challenges, we present a new two-phase cross-domain transfer learning system for effective skin lesion segmentation from dermoscopic images.

METHODS

Our system is based on two significant technical inventions. We examine a two- phase cross-domain transfer learning approach, including model-level and data-level transfer learning, by fine-tuning the system on two datasets, MoleMap and ImageNet. We then present nSknRSUNet, a high-performing DL network, for skin lesion segmentation using broad receptive fields and spatial edge attention feature fusion. We examine the trained model's generalization capabilities on skin lesion segmentation to quantify these two inventions. We cross-examine the model using two skin lesion image datasets, MoleMap and HAM10000, obtained from varied clinical contexts.

RESULTS

At data-level transfer learning for the HAM10000 dataset, the proposed model obtained 94.63% of DSC and 99.12% accuracy. In cross-examination at data-level transfer learning for the Molemap dataset, the proposed model obtained 93.63% of DSC and 97.01% of accuracy.

CONCLUSION

Numerous experiments reveal that our system produces excellent performance and improves upon state-of-the-art methods on both qualitative and quantitative measures.

Ultra-Attention: Automatic Recognition of Liver Ultrasound Standard Sections Based on Visual Attention Perception Structures

Acquisition of a standard section is a prerequisite for ultrasound diagnosis. For a long time, there has been a lack of clear definitions of standard liver views because of physician experience. The accurate automated scanning of standard liver sections, however, remains one of ultrasonography medicine's most important issues. In this article, we enrich and expand the classification criteria of liver ultrasound standard sections from clinical practice and propose an Ultra-Attention structured perception strategy to automate the recognition of these sections. Inspired by the attention mechanism in natural language processing, the standard liver ultrasound views will participate in the global attention algorithm as modular local images in computer vision of ultrasound images, which will significantly amplify small features that would otherwise go unnoticed. In addition to using the dropout mechanism, we also use a Part-Transfer Learning training approach to fine-tune the model's rate of convergence to increase its robustness. The proposed Ultra-Attention model outperforms various traditional convolutional neural network-based techniques, achieving the best known performance in the field with a classification accuracy of 93.2%. As part of the feature extraction procedure, we also illustrate and compare the convolutional structure and the Ultra-Attention approach. This analysis provides a reasonable view for future research on local modular feature capture in ultrasound images. By developing a standard scan guideline for liver ultrasound-based illness diagnosis, this work will advance the research on automated disease diagnosis that is directed by standard sections of liver ultrasound.

Machine Learning Methods in Skin Disease Recognition: A Systematic Review

Skin lesions affect millions of people worldwide. They can be easily recognized based on their typically abnormal texture and color but are difficult to diagnose due to similar symptoms among certain types of lesions. The motivation for this study is to collate and analyze machine learning (ML) applications in skin lesion research, with the goal of encouraging the development of automated systems for skin disease diagnosis. To assist dermatologists in their clinical diagnosis, several skin image datasets have been developed and published online. Such efforts have motivated researchers and medical staff to develop automatic skin diagnosis systems using image segmentation and classification processes. This paper summarizes the fundamental steps in skin lesion diagnosis based on papers mainly published since 2013. The applications of ML methods (including traditional ML and deep learning (DL)) in skin disease recognition are reviewed based on their contributions, methods, and achieved results. Such technical analysis is beneficial to the continuing development of reliable and effective computer-aided skin disease diagnosis systems. We believe that more research efforts will lead to the current automatic skin diagnosis studies being used in real clinical settings in the near future.

CI-Net: Clinical-Inspired Network for Automated Skin Lesion Recognition

The lesion recognition of dermoscopy images is significant for automated skin cancer diagnosis. Most of the existing methods ignore the medical perspective, which is crucial since this task requires a large amount of medical knowledge. A few methods are designed according to medical knowledge, but they ignore to be fully in line with doctors' entire learning and diagnosis process, since certain strategies and steps of those are conducted in practice for doctors. Thus, we put forward Clinical-Inspired Network (CI-Net) to involve the learning strategy and diagnosis process of doctors, as for a better analysis. The diagnostic process contains three main steps: the zoom step, the observe step and the compare step. To simulate these, we introduce three corresponding modules: a lesion area attention module, a feature extraction module and a lesion feature attention module. To simulate the distinguish strategy, which is commonly used by doctors, we introduce a distinguish module. We evaluate our proposed CI-Net on six challenging datasets, including ISIC 2016, ISIC 2017, ISIC 2018, ISIC 2019, ISIC 2020 and PH2 datasets, and the results indicate that CI-Net outperforms existing work. The code is publicly available at https://github.com/lzh19961031/Dermoscopy_classification.

Medical Image Classifications for 6G IoT-Enabled Smart Health Systems

As day-to-day-generated data become massive in the 6G-enabled Internet of medical things (IoMT), the process of medical diagnosis becomes critical in the healthcare system. This paper presents a framework incorporated into the 6G-enabled IoMT to improve prediction accuracy and provide a real-time medical diagnosis. The proposed framework integrates deep learning and optimization techniques to render accurate and precise results. The medical computed tomography images are preprocessed and fed into an efficient neural network designed for learning image representations and converting each image to a feature vector. The extracted features from each image are then learned using a MobileNetV3 architecture. Furthermore, we enhanced the performance of the arithmetic optimization algorithm (AOA) based on the hunger games search (HGS). In the developed method, named AOAHG, the operators of the HGS are applied to enhance the AOA's exploitation ability while allocating the feasible region. The developed AOAG selects the most relevant features and ensures the overall model classification improvement. To assess the validity of our framework, we conducted evaluation experiments on four datasets, including ISIC-2016 and PH2 for skin cancer detection, white blood cell (WBC) detection, and optical coherence tomography (OCT) classification, using different evaluation metrics. The framework showed remarkable performance compared to currently existing methods in the literature. In addition, the developed AOAHG provided results better than other FS approaches according to the obtained accuracy, precision, recall, and F1-score as performance measures. For example, AOAHG had 87.30%, 96.40%, 88.60%, and 99.69% for the ISIC, PH2, WBC, and OCT datasets, respectively.

New AI-algorithms on smartphones to detect skin cancer in a clinical setting-A validation study

BACKGROUND AND OBJECTIVES

The incidence of skin cancer is rising worldwide and there is medical need to optimize its early detection. This study was conducted to determine the diagnostic and risk-assessment accuracy of two new diagnosis-based neural networks (analyze and detect), which comply with the CE-criteria, in evaluating the malignant potential of various skin lesions on a smartphone. Of note, the intention of our study was to evaluate the performance of these medical products in a clinical setting for the first time.

METHODS

This was a prospective, single-center clinical study at one tertiary referral center in Graz, Austria. Patients, who were either scheduled for preventive skin examination or removal of at least one skin lesion were eligible for participation. Patients were assessed by at least two dermatologists and by the integrated algorithms on different mobile phones. The lesions to be recorded were randomly selected by the dermatologists. The diagnosis of the algorithm was stated as correct if it matched the diagnosis of the two dermatologists or the histology (if available). The histology was the reference standard, however, if both clinicians considered a lesion as being benign no histology was performed and the dermatologists were stated as reference standard.

RESULTS

A total of 238 patients with 1171 lesions (86 female; 36.13%) with an average age of 66.19 (SD = 17.05) was included. Sensitivity and specificity of the detect algorithm were 96.4% (CI 93.94-98.85) and 94.85% (CI 92.46-97.23); for the analyze algorithm a sensitivity of 95.35% (CI 93.45-97.25) and a specificity of 90.32% (CI 88.1-92.54) were achieved.

DISCUSSION

The studied neural networks succeeded analyzing the risk of skin lesions with a high diagnostic accuracy showing that they are sufficient tools in calculating the probability of a skin lesion being malignant. In conjunction with the wide spread use of smartphones this new AI approach opens the opportunity for a higher early detection rate of skin cancer with consecutive lower epidemiological burden of metastatic cancer and reducing health care costs. This neural network moreover facilitates the empowerment of patients, especially in regions with a low density of medical doctors.

REGISTRATION

Approved and registered at the ethics committee of the Medical University of Graz, Austria (Approval number: 30-199 ex 17/18).

The Role of Machine Learning and Deep Learning Approaches for the Detection of Skin Cancer

Machine learning (ML) can enhance a dermatologist's work, from diagnosis to customized care. The development of ML algorithms in dermatology has been supported lately regarding links to digital data processing (e.g., electronic medical records, Image Archives, omics), quicker computing and cheaper data storage. This article describes the fundamentals of ML-based implementations, as well as future limits and concerns for the production of skin cancer detection and classification systems. We also explored five fields of dermatology using deep learning applications: (1) the classification of diseases by clinical photos, (2) der moto pathology visual classification of cancer, and (3) the measurement of skin diseases by smartphone applications and personal tracking systems. This analysis aims to provide dermatologists with a guide that helps demystify the basics of ML and its different applications to identify their possible challenges correctly. This paper surveyed studies on skin cancer detection using deep learning to assess the features and advantages of other techniques. Moreover, this paper also defined the basic requirements for creating a skin cancer detection application, which revolves around two main issues: the full segmentation image and the tracking of the lesion on the skin using deep learning. Most of the techniques found in this survey address these two problems. Some of the methods also categorize the type of cancer too.

Skin lesion classification based on two-modal images using a multi-scale fully-shared fusion network

BACKGROUND AND OBJECTIVE

Due to the complexity of skin lesion features, computer-aided diagnosis of skin diseases based on multi-modal images is considered a challenging task. Dermoscopic images and clinical images are commonly used to diagnose skin diseases in clinical scenarios, and the complementarity of their features promotes the research of multi-modality classification in the computer-aided diagnosis field. Most current methods focus on the fusion between modalities and ignore the complementary information within each of them, which leads to the loss of the intra-modality relation. Multi-modality models for integrating features both within single modalities and across multiple modalities are limited in the literature. Therefore, a multi-modality model based on dermoscopic and clinical images is proposed to address this issue.

METHODS

We propose a Multi-scale Fully-shared Fusion Network (MFF-Net) that gathers features of dermoscopic images and clinical images for skin lesion classification. In MFF-Net, the multi-scale fusion structure combines deep and shallow features within individual modalities to reduce the loss of spatial information in high-level feature maps. Then Dermo-Clinical Block (DCB) integrates the feature maps from dermoscopic images and clinical images through channel-wise concatenation and using a fully-shared fusion strategy that explores complementary information at different stages.

RESULTS

We validated our model on a four-class two-modal skin diseases dataset, and proved that the proposed multi-scale structure, the fusion module DCBs, and the fully-shared fusion strategy improve the performance of MFF-Net independently. Our method achieved the highest average accuracy of 72.9% on the 7-point checklist dataset, outperforming the state-of-the-art single-modality and multi-modality methods with an accuracy boost of 7.1% and 3.4%, respectively.

CONCLUSIONS

The multi-scale fusion structure demonstrates the significance of intra-modality relations between clinical images and dermoscopic images. The proposed network combined with the multi-scale structure, DCBs, and the fully-shared fusion strategy, can effectively integrate the features of the skin lesions across the two modalities and achieved a promising accuracy among different skin diseases.

Toward Multicenter Skin Lesion Classification Using Deep Neural Network With Adaptively Weighted Balance Loss

Recently, deep neural network-based methods have shown promising advantages in accurately recognizing skin lesions from dermoscopic images. However, most existing works focus more on improving the network framework for better feature representation but ignore the data imbalance issue, limiting their flexibility and accuracy across multiple scenarios in multi-center clinics. Generally, different clinical centers have different data distributions, which presents challenging requirements for the network's flexibility and accuracy. In this paper, we divert the attention from framework improvement to the data imbalance issue and propose a new solution for multi-center skin lesion classification by introducing a novel adaptively weighted balance (AWB) loss to the conventional classification network. Benefiting from AWB, the proposed solution has the following advantages: 1) it is easy to satisfy different practical requirements by only changing the backbone; 2) it is user-friendly with no tuning on hyperparameters; and 3) it adaptively enables small intraclass compactness and pays more attention to the minority class. Extensive experiments demonstrate that, compared with solutions equipped with state-of-the-art loss functions, the proposed solution is more flexible and more competent for tackling the multi-center imbalanced skin lesion classification task with considerable performance on two benchmark datasets. In addition, the proposed solution is proved to be effective in handling the imbalanced gastrointestinal disease classification task and the imbalanced DR grading task. Code is available at https://github.com/Weipeishan2021.

Classification of skin lesion images using modified Inception V3 model with transfer learning and augmentation techniques

In this article, a methodological approach to classifying malignant melanoma in dermoscopy images is presented. Early treatment of skin cancer increases the patient’s survival rate. The classification of melanoma skin cancer in the early stages is decided by dermatologists to treat the patient appropriately. Dermatologists need more time to diagnose affected skin lesions due to high resemblance between melanoma and benign. In this paper, a deep learning based Computer-Aided Diagnosis (CAD) system is developed to accurately classify skin lesions with a high classification rate. A new architecture has been framed to classify the skin lesion diseases using the Inception v3 model as a baseline architecture. The extracted features from the Inception Net are then flattened and are given to the DenseNet block to extracts more fine grained features of the lesion disease. The International Skin Imaging Collaboration (ISIC) archive datasets contains 3307 dermoscopy images which includes both benign and malignant skin images. The dataset images are trained using the proposed architecture with the learning rate of 0.0001, batch size 64 using various optimizer. The performance of the proposed model has also been evaluated using confusion matrix and ROC-AUC curves. The experimental results show that the proposed model attains a highest accuracy rate of 91.29 % compared to other state-of-the-art methods like ResNet, VGG-16, DenseNet, MobileNet. A confusion matrix and ROC curve are used to evaluate the performance analysis of skin images. The classification accuracy, sensitivity, specificity, testing accuracy, and AUC values were obtained at 90.33%, 82.87%, 91.29%, 87.12%, and 87.40% .

SkiNet: A deep learning framework for skin lesion diagnosis with uncertainty estimation and explainability

Skin cancer is considered to be the most common human malignancy. Around 5 million new cases of skin cancer are recorded in the United States annually. Early identification and evaluation of skin lesions are of great clinical significance, but the disproportionate dermatologist-patient ratio poses a significant problem in most developing nations. Therefore a novel deep architecture, named as SkiNet, is proposed to provide faster screening solution and assistance to newly trained physicians in the process of clinical diagnosis of skin cancer. The main motive behind SkiNet's design and development is to provide a white box solution, addressing a critical problem of trust and interpretability which is crucial for the wider adoption of Computer-aided diagnosis systems by medical practitioners. The proposed SkiNet is a two-stage pipeline wherein the lesion segmentation is followed by the lesion classification. Monte Carlo dropout and test time augmentation techniques have been employed in the proposed method to estimate epistemic and aleatoric uncertainty. A novel segmentation model named Bayesian MultiResUNet is used to estimate the uncertainty on the predicted segmentation map. Saliency-based methods like XRAI, Grad-CAM and Guided Backprop are explored to provide post-hoc explanations of the deep learning models. The ISIC-2018 dataset is used to perform the experimentation and ablation studies. The results establish the robustness of the proposed model on the traditional benchmarks while addressing the black-box nature of such models to alleviate the skepticism of medical practitioners by incorporating transparency and confidence to the model's prediction.

Skin lesion classification of dermoscopic images using machine learning and convolutional neural network

Detecting dangerous illnesses connected to the skin organ, particularly malignancy, requires the identification of pigmented skin lesions. Image detection techniques and computer classification capabilities can boost skin cancer detection accuracy. The dataset used for this research work is based on the HAM10000 dataset which consists of 10015 images. The proposed work has chosen a subset of the dataset and performed augmentation. A model with data augmentation tends to learn more distinguishing characteristics and features rather than a model without data augmentation. Involving data augmentation can improve the accuracy of the model. But that model cannot give significant results with the testing data until it is robust. The k-fold cross-validation technique makes the model robust which has been implemented in the proposed work. We have analyzed the classification accuracy of the Machine Learning algorithms and Convolutional Neural Network models. We have concluded that Convolutional Neural Network provides better accuracy compared to other machine learning algorithms implemented in the proposed work. In the proposed system, as the highest, we obtained an accuracy of 95.18% with the CNN model. The proposed work helps early identification of seven classes of skin disease and can be validated and treated appropriately by medical practitioners.

AHA-AO: Artificial Hummingbird Algorithm with Aquila Optimization for Efficient Feature Selection in Medical Image Classification

This paper presents a system for medical image diagnosis that uses transfer learning (TL) and feature selection techniques. The main aim of TL on pre-trained models such as MobileNetV3 is to extract features from raw images. Here, a novel feature selection optimization algorithm called the Artificial Hummingbird Algorithm based on Aquila Optimization (AHA-AO) is proposed. The AHA-AO is used to select only the most relevant features and ensure the improvement of the overall model classification. Our methodology was evaluated using four datasets, namely, ISIC-2016, PH2, Chest-XRay, and Blood-Cell. We compared the proposed feature selection algorithm with five of the most popular feature selection optimization algorithms. We obtained an accuracy of 87.30% for the ISIC-2016 dataset, 97.50% for the PH2 dataset, 86.90% for the Chest-XRay dataset, and 88.60% for the Blood-cell dataset. The AHA-AO outperformed the other optimization techniques. Moreover, the developed AHA-AO was faster than the other feature selection models during the process of determining the relevant features. The proposed feature selection algorithm successfully improved the performance and the speed of the overall deep learning models.

N-Net: A novel dense fully convolutional neural network for thyroid nodule segmentation

Medical image segmentation is an essential component of computer-aided diagnosis (CAD) systems. Thyroid nodule segmentation using ultrasound images is a necessary step for the early diagnosis of thyroid diseases. An encoder-decoder based deep convolutional neural network (DCNN), like U-Net architecture and its variants, has been extensively used to deal with medical image segmentation tasks. In this article, we propose a novel N-shape dense fully convolutional neural network for medical image segmentation, referred to as N-Net. The proposed framework is composed of three major components: a multi-scale input layer, an attention guidance module, and an innovative stackable dilated convolution (SDC) block. First, we apply the multi-scale input layer to construct an image pyramid, which achieves multi-level receiver field sizes and obtains rich feature representation. After that, the U-shape convolutional network is employed as the backbone structure. Moreover, we use the attention guidance module to filter the features before several skip connections, which can transfer structural information from previous feature maps to the following layers. This module can also remove noise and reduce the negative impact of the background. Finally, we propose a stackable dilated convolution (SDC) block, which is able to capture deep semantic features that may be lost in bilinear upsampling. We have evaluated the proposed N-Net framework on a thyroid nodule ultrasound image dataset (called the TNUI-2021 dataset) and the DDTI publicly available dataset. The experimental results show that our N-Net model outperforms several state-of-the-art methods in the thyroid nodule segmentation tasks.

Adversarial multimodal fusion with attention mechanism for skin lesion classification using clinical and dermoscopic images

Accurate skin lesion diagnosis requires a great effort from experts to identify the characteristics from clinical and dermoscopic images. Deep multimodal learning-based methods can reduce intra- and inter-reader variability and improve diagnostic accuracy compared to the single modality-based methods. This study develops a novel method, named adversarial multimodal fusion with attention mechanism (AMFAM), to perform multimodal skin lesion classification. Specifically, we adopt a discriminator that uses adversarial learning to enforce the feature extractor to learn the correlated information explicitly. Moreover, we design an attention-based reconstruction strategy to encourage the feature extractor to concentrate on learning the features of the lesion area, thus, enhancing the feature vector from each modality with more discriminative information. Unlike existing multimodal-based approaches, which only focus on learning complementary features from dermoscopic and clinical images, our method considers both correlated and complementary information of the two modalities for multimodal fusion. To verify the effectiveness of our method, we conduct comprehensive experiments on a publicly available multimodal and multi-task skin lesion classification dataset: 7-point criteria evaluation database. The experimental results demonstrate that our proposed method outperforms the current state-of-the-art methods and improves the average AUC score by above 2% on the test set.

Medical Image Classification Using Transfer Learning and Chaos Game Optimization on the Internet of Medical Things

The Internet of Medical Things (IoMT) has dramatically benefited medical professionals that patients and physicians can access from all regions. Although the automatic detection and prediction of diseases such as melanoma and leukemia is still being investigated and studied in IoMT, existing approaches are not able to achieve a high degree of efficiency. Thus, with a new approach that provides better results, patients would access the adequate treatments earlier and the death rate would be reduced. Therefore, this paper introduces an IoMT proposal for medical images' classification that may be used anywhere, i.e., it is an ubiquitous approach. It was designed in two stages: first, we employ a transfer learning (TL)-based method for feature extraction, which is carried out using MobileNetV3; second, we use the chaos game optimization (CGO) for feature selection, with the aim of excluding unnecessary features and improving the performance, which is key in IoMT. Our methodology was evaluated using ISIC-2016, PH2, and Blood-Cell datasets. The experimental results indicated that the proposed approach obtained an accuracy of 88.39% on ISIC-2016, 97.52% on PH2, and 88.79% on Blood-cell datsets. Moreover, our approach had successful performances for the metrics employed compared to other existing methods.

Fusion of Local and Global Feature Representation With Sparse Autoencoder for Improved Melanoma Classification

Automated skin cancer diagnosis is challenging due to inter-class uniformity, intra-class variation, and the complex structure of dermoscopy images. Convolutional neural networks (CNN) have recently made considerable progress in melanoma classification, even in the presence of limited skin images. One of the drawbacks of these methods is the loss of image details caused by downsampling high-resolution skin images to a low resolution. Further, most approaches extract features only from the whole skin image. This paper proposes an ensemble feature fusion and sparse autoencoder (SAE) based framework to overcome the above issues and improve melanoma classification performance. The proposed method extracts features from two streams, local and global, using a pre-trained CNN model. The local stream extracts features from image patches, while the global stream derives features from the whole skin image, preserving both local and global representation. The features are then fused, and an SAE framework is subsequently designed to enrich the feature representation further. The proposed method is validated on ISIC 2016 dataset and the experimental results indicate the superiority of the proposed approach.

Skin Cancer Disease Detection Using Transfer Learning Technique

Melanoma is a fatal type of skin cancer; the fury spread results in a high fatality rate when the malignancy is not treated at an initial stage. The patients’ lives can be saved by accurately detecting skin cancer at an initial stage. A quick and precise diagnosis might help increase the patient’s survival rate. It necessitates the development of a computer-assisted diagnostic support system. This research proposes a novel deep transfer learning model for melanoma classification using MobileNetV2. The MobileNetV2 is a deep convolutional neural network that classifies the sample skin lesions as malignant or benign. The performance of the proposed deep learning model is evaluated using the ISIC 2020 dataset. The dataset contains less than 2% malignant samples, raising the class imbalance. Various data augmentation techniques were applied to tackle the class imbalance issue and add diversity to the dataset. The experimental results demonstrate that the proposed deep learning technique outperforms state-of-the-art deep learning techniques in terms of accuracy and computational cost.

NeDSeM: Neutrosophy Domain-Based Segmentation Method for Malignant Melanoma Images

Skin lesion segmentation is the first and indispensable step of malignant melanoma recognition and diagnosis. At present, most of the existing skin lesions segmentation techniques often used traditional methods like optimum thresholding, etc., and deep learning methods like U-net, etc. However, the edges of skin lesions in malignant melanoma images are gradually changed in color, and this change is nonlinear. The existing methods can not effectively distinguish banded edges between lesion areas and healthy skin areas well. Aiming at the uncertainty and fuzziness of banded edges, the neutrosophic set theory is used in this paper which is better than fuzzy theory to deal with banded edge segmentation. Therefore, we proposed a neutrosophy domain-based segmentation method that contains six steps. Firstly, an image is converted into three channels and the pixel matrix of each channel is obtained. Secondly, the pixel matrixes are converted into Neutrosophic Set domain by using the neutrosophic set conversion method to express the uncertainty and fuzziness of banded edges of malignant melanoma images. Thirdly, a new Neutrosophic Entropy model is proposed to combine the three memberships according to some rules by using the transformations in the neutrosophic space to comprehensively express three memberships and highlight the banded edges of the images. Fourthly, the feature augment method is established by the difference of three components. Fifthly, the dilation is used on the neutrosophic entropy matrixes to fill in the noise region. Finally, the image that is represented by transformed matrix is segmented by the Hierarchical Gaussian Mixture Model clustering method to obtain the banded edge of the image. Qualitative and quantitative experiments are performed on malignant melanoma image dataset to evaluate the performance of the NeDSeM method. Compared with some state-of-the-art methods, our method has achieved good results in terms of performance and accuracy.

Medical Image Classification Utilizing Ensemble Learning and Levy Flight-Based Honey Badger Algorithm on 6G-Enabled Internet of Things

Recently, the 6G-enabled Internet of Medical Things (IoMT) has played a key role in the development of functional health systems due to the massive data generated daily from the hospitals. Therefore, the automatic detection and prediction of future risks such as pneumonia and retinal diseases are still under research and study. However, traditional approaches did not yield good results for accurate diagnosis. In this paper, a robust 6G-enabled IoMT framework is proposed for medical image classification with an ensemble learning (EL)-based model. EL is achieved using MobileNet and DenseNet architecture as a feature extraction backbone. In addition, the developed framework uses a modified honey badger algorithm (HBA) based on Levy flight (LFHBA) as a feature selection method that aims to remove the irrelevant features from those extracted features using the EL model. For evaluation of the performance of the proposed framework, the chest X-ray (CXR) dataset and the optical coherence tomography (OCT) dataset were employed. The accuracy of our technique was 87.10% on the CXR dataset and 94.32% on OCT dataset—both very good results. Compared to other current methods, the proposed method is more accurate and efficient than other well-known and popular algorithms.

Structural and diffusion MRI based schizophrenia classification using 2D pretrained and 3D naive Convolutional Neural Networks

The ability of automatic feature learning makes Convolutional Neural Network (CNN) potentially suitable to uncover the complex and widespread brain changes in schizophrenia. Despite that, limited studies have been done on schizophrenia identification using interpretable deep learning approaches on multimodal neuroimaging data. Here, we developed a deep feature approach based on pre-trained 2D CNN and naive 3D CNN models trained from scratch for schizophrenia classification by integrating 3D structural and diffusion magnetic resonance imaging (MRI) data. We found that the naive 3D CNN models outperformed the pretrained 2D CNN models and the handcrafted feature-based machine learning approach using support vector machine during both cross-validation and testing on an independent dataset. Multimodal neuroimaging-based models accomplished performance superior to models based on a single modality. Furthermore, we identified brain grey matter and white matter regions critical for illness classification at the individual- and group-level which supported the salience network and striatal dysfunction hypotheses in schizophrenia. Our findings underscore the potential of CNN not only to automatically uncover and integrate multimodal 3D brain imaging features for schizophrenia identification, but also to provide relevant neurobiological interpretations which are crucial for developing objective and interpretable imaging-based probes for prognosis and diagnosis in psychiatric disorders.

Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification

Deep convolutional neural network (DCNN) models have been widely explored for skin disease diagnosis and some of them have achieved the diagnostic outcomes comparable or even superior to those of dermatologists. However, broad implementation of DCNN in skin disease detection is hindered by small size and data imbalance of the publically accessible skin lesion datasets. This paper proposes a novel single-model based strategy for classification of skin lesions on small and imbalanced datasets. First, various DCNNs are trained on different small and imbalanced datasets to verify that the models with moderate complexity outperform the larger models. Second, regularization DropOut and DropBlock are added to reduce overfitting and a Modified RandAugment augmentation strategy is proposed to deal with the defects of sample underrepresentation in the small dataset. Finally, a novel Multi-Weighted New Loss (MWNL) function and an end-to-end cumulative learning strategy (CLS) are introduced to overcome the challenge of uneven sample size and classification difficulty and to reduce the impact of abnormal samples on training. By combining Modified RandAugment, MWNL and CLS, our single DCNN model method achieved the classification accuracy comparable or superior to those of multiple ensembling models on different dermoscopic image datasets. Our study shows that this method is able to achieve a high classification performance at a low cost of computational resources and inference time, potentially suitable to implement in mobile devices for automated screening of skin lesions and many other malignancies in low resource settings.

Refined Residual Deep Convolutional Network for Skin Lesion Classification

Skin cancer is the most common type of cancer that affects humans and is usually diagnosed by initial clinical screening, which is followed by dermoscopic analysis. Automated classification of skin lesions is still a challenging task because of the high visual similarity between melanoma and benign lesions. This paper proposes a new residual deep convolutional neural network (RDCNN) for skin lesions diagnosis. The proposed neural network is trained and tested using six well-known skin cancer datasets, PH2, DermIS and Quest, MED-NODE, ISIC2016, ISIC2017, and ISIC2018. Three different experiments are carried out to measure the performance of the proposed RDCNN. In the first experiment, the proposed RDCNN is trained and tested using the original dataset images without any pre-processing or segmentation. In the second experiment, the proposed RDCNN is tested using segmented images. Finally, the utilized trained model in the second experiment is saved and reused in the third experiment as a pre-trained model. Then, it is trained again using a different dataset. The proposed RDCNN shows significant high performance and outperforms the existing deep convolutional networks.

Early Melanoma Diagnosis With Sequential Dermoscopic Images

Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.

MPMR: Multi-Scale Feature and Probability Map for Melanoma Recognition

Malignant melanoma (MM) recognition in whole-slide images (WSIs) is challenging due to the huge image size of billions of pixels and complex visual characteristics. We propose a novel automatic melanoma recognition method based on the multi-scale features and probability map, named MPMR. First, we introduce the idea of breaking up the WSI into patches to overcome the difficult-to-calculate problem of WSIs with huge sizes. Second, to obtain and visualize the recognition result of MM tissues in WSIs, a probability mapping method is proposed to generate the mask based on predicted categories, confidence probabilities, and location information of patches. Third, considering that the pathological features related to melanoma are at different scales, such as tissue, cell, and nucleus, and to enhance the representation of multi-scale features is important for melanoma recognition, we construct a multi-scale feature fusion architecture by additional branch paths and shortcut connections, which extracts the enriched lesion features from low-level features containing more detail information and high-level features containing more semantic information. Fourth, to improve the extraction feature of the irregular-shaped lesion and focus on essential features, we reconstructed the residual blocks by a deformable convolution and channel attention mechanism, which further reduces information redundancy and noisy features. The experimental results demonstrate that the proposed method outperforms the compared algorithms, and it has a potential for practical applications in clinical diagnosis.

Modified U-NET Architecture for Segmentation of Skin Lesion

Dermoscopy images can be classified more accurately if skin lesions or nodules are segmented. Because of their fuzzy borders, irregular boundaries, inter- and intra-class variances, and so on, nodule segmentation is a difficult task. For the segmentation of skin lesions from dermoscopic pictures, several algorithms have been developed. However, their accuracy lags well behind the industry standard. In this paper, a modified U-Net architecture is proposed by modifying the feature map’s dimension for an accurate and automatic segmentation of dermoscopic images. Apart from this, more kernels to the feature map allowed for a more precise extraction of the nodule. We evaluated the effectiveness of the proposed model by considering several hyper parameters such as epochs, batch size, and the types of optimizers, testing it with augmentation techniques implemented to enhance the amount of photos available in the PH2 dataset. The best performance achieved by the proposed model is with an Adam optimizer using a batch size of 8 and 75 epochs.

Multiclass Skin Lesion Classification Using Hybrid Deep Features Selection and Extreme Learning Machine

The variation in skin textures and injuries, as well as the detection and classification of skin cancer, is a difficult task. Manually detecting skin lesions from dermoscopy images is a difficult and time-consuming process. Recent advancements in the domains of the internet of things (IoT) and artificial intelligence for medical applications demonstrated improvements in both accuracy and computational time. In this paper, a new method for multiclass skin lesion classification using best deep learning feature fusion and an extreme learning machine is proposed. The proposed method includes five primary steps: image acquisition and contrast enhancement; deep learning feature extraction using transfer learning; best feature selection using hybrid whale optimization and entropy-mutual information (EMI) approach; fusion of selected features using a modified canonical correlation based approach; and, finally, extreme learning machine based classification. The feature selection step improves the system's computational efficiency and accuracy. The experiment is carried out on two publicly available datasets, HAM10000 and ISIC2018. The achieved accuracy on both datasets is 93.40 and 94.36 percent. When compared to state-of-the-art (SOTA) techniques, the proposed method's accuracy is improved. Furthermore, the proposed method is computationally efficient.

Two-Stage Deep Neural Network via Ensemble Learning for Melanoma Classification

Melanoma is a skin disease with a high fatality rate. Early diagnosis of melanoma can effectively increase the survival rate of patients. There are three types of dermoscopy images, malignant melanoma, benign nevis, and seborrheic keratosis, so using dermoscopy images to classify melanoma is an indispensable task in diagnosis. However, early melanoma classification works can only use the low-level information of images, so the melanoma cannot be classified efficiently; the recent deep learning methods mainly depend on a single network, although it can extract high-level features, the poor scale and type of the features limited the results of the classification. Therefore, we need an automatic classification method for melanoma, which can make full use of the rich and deep feature information of images for classification. In this study, we propose an ensemble method that can integrate different types of classification networks for melanoma classification. Specifically, we first use U-net to segment the lesion area of images to generate a lesion mask, thus resize images to focus on the lesion; then, we use five excellent classification models to classify dermoscopy images, and adding squeeze-excitation block (SE block) to models to emphasize the more informative features; finally, we use our proposed new ensemble network to integrate five different classification results. The experimental results prove the validity of our results. We test our method on the ISIC 2017 challenge dataset and obtain excellent results on multiple metrics; especially, we get 0.909 on accuracy. Our classification framework can provide an efficient and accurate way for melanoma classification using dermoscopy images, laying the foundation for early diagnosis and later treatment of melanoma.

FusionM4Net: A multi-stage multi-modal learning algorithm for multi-label skin lesion classification

Skin disease is one of the most common diseases in the world. Deep learning-based methods have achieved excellent skin lesion recognition performance, most of which are based on only dermoscopy images. In recent works that use multi-modality data (patient's meta-data, clinical images, and dermoscopy images), the methods adopt a one-stage fusion approach and only optimize the information fusion at the feature level. These methods do not use information fusion at the decision level and thus cannot fully use the data of all modalities. This work proposes a novel two-stage multi-modal learning algorithm (FusionM4Net) for multi-label skin diseases classification. At the first stage, we construct a FusionNet, which exploits and integrates the representation of clinical and dermoscopy images at the feature level, and then uses a Fusion Scheme 1 to conduct the information fusion at the decision level. At the second stage, to further incorporate the patient's meta-data, we propose a Fusion Scheme 2, which integrates the multi-label predictive information from the first stage and patient's meta-data information to train an SVM cluster. The final diagnosis is formed by the fusion of the predictions from the first and second stages. Our algorithm was evaluated on the seven-point checklist dataset, a well-established multi-modality multi-label skin disease dataset. Without using the patient's meta-data, the proposed FusionM4Net's first stage (FusionM4Net-FS) achieved an average accuracy of 75.7% for multi-classification tasks and 74.9% for diagnostic tasks, which is more accurate than other state-of-the-art methods. By further fusing the patient's meta-data at FusionM4Net's second stage (FusionM4Net-SS), the entire FusionM4Net finally boosts the average accuracy to 77.0% and the diagnostic accuracy to 78.5%, which indicates its robust and excellent classification performance on the label-imbalanced dataset. The corresponding code is available at: https://github.com/pixixiaonaogou/MLSDR.

Multi-level Attentive Skin Lesion Learning for Melanoma Classification

Melanoma classification plays an important role in skin lesion diagnosis. Nevertheless, melanoma classification is a challenging task, due to the appearance variation of the skin lesions, and the interference of the noises from dermoscopic imaging. In this paper, we propose a multi-level attentive skin lesion learning (MASLL) network to enhance melanoma classification. Specifically, we design a local learning branch with a skin lesion localization (SLL) module to assist the network in learning the lesion features from the region of interest. In addition, we propose a weighted feature integration (WFI) module to fuse the lesion information from the global and local branches, which further enhances the feature discriminative capability of the skin lesions. Experimental results on ISIC 2017 dataset show the effectiveness of the proposed method on melanoma classification.