Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks

Self-supervised multi-modality learning for multi-label skin lesion classification

BACKGROUND

The clinical diagnosis of skin lesions involves the analysis of dermoscopic and clinical modalities. Dermoscopic images provide detailed views of surface structures, while clinical images offer complementary macroscopic information. Clinicians frequently use the seven-point checklist as an auxiliary tool for melanoma diagnosis and identifying lesion attributes. Supervised deep learning approaches, such as convolutional neural networks, have performed well using dermoscopic and clinical modalities (multi-modality) and further enhanced classification by predicting seven skin lesion attributes (multi-label). However, the performance of these approaches is reliant on the availability of large-scale labeled data, which are costly and time-consuming to obtain, more so with annotating multi-attributes METHODS:: To reduce the dependency on large labeled datasets, we propose a self-supervised learning (SSL) algorithm for multi-modality multi-label skin lesion classification. Compared with single-modality SSL, our algorithm enables multi-modality SSL by maximizing the similarities between paired dermoscopic and clinical images from different views. We introduce a novel multi-modal and multi-label SSL strategy that generates surrogate pseudo-multi-labels for seven skin lesion attributes through clustering analysis. A label-relation-aware module is proposed to refine each pseudo-label embedding, capturing the interrelationships between pseudo-multi-labels. We further illustrate the interrelationships of skin lesion attributes and their relationships with clinical diagnoses using an attention visualization technique.

RESULTS

The proposed algorithm was validated using the well-benchmarked seven-point skin lesion dataset. Our results demonstrate that our method outperforms the state-of-the-art SSL counterparts. Improvements in the area under receiver operating characteristic curve, precision, sensitivity, and specificity were observed across various lesion attributes and melanoma diagnoses.

CONCLUSIONS

Our self-supervised learning algorithm offers a robust and efficient solution for multi-modality multi-label skin lesion classification, reducing the reliance on large-scale labeled data. By effectively capturing and leveraging the complementary information between the dermoscopic and clinical images and interrelationships between lesion attributes, our approach holds the potential for improving clinical diagnosis accuracy in dermatology.

U-shaped deep learning networks for algal bloom detection using Sentinel-2 imagery: Exploring model performance and transferability

Inland water sources, such as lakes, support diverse ecosystems and provide essential services to human societies. However, these valuable resources are under increasing pressure from rapid climate changes and pollution resulting from human activities. Combining remote sensing technologies with advanced artificial intelligence algorithms enables frequent monitoring of these ecosystems, timely detection of potential threats, and effective conservation measures. This study evaluated U-shaped deep learning (DL) networks, including U-Net, Residual U-Net (RU-Net), Attention U-Net, Attention Residual U-Net (ARU-Net), and SegNet, for detecting and mapping algal blooms using Sentinel-2 satellite imagery. Multitemporal Sentinel-2 imagery spanning different dates was used to construct robust DL models, with ground truth datasets representing both high- and low-density algae formations. The study emphasized the importance of diverse datasets in addressing the limitations of previous models, particularly in detecting low-density blooms and generalizing across temporal and geographical contexts. The models' transferability was assessed using imagery from different dates and geographical locations, including Lake Burdur, Lake Chaohu, and Lake Turawskie. RU-Net and ARU-Net consistently outperformed other models, achieving exceptional F-scores, such as 99.80 % for Lake Burdur, 97.23 % for Lake Chaohu, and 99.61 % for Lake Turawskie. ARU-Net demonstrated superior generalization capabilities, effectively detecting low-density algae, which is critical for comprehensive environmental assessments. These findings underscored the efficacy and transferability of U-shaped DL networks in accurately detecting algal blooms, offering valuable insights for environmental monitoring and management applications.

Automatic melanoma and non-melanoma skin cancer diagnosis using advanced adaptive fine-tuned convolution neural networks

Skin Cancer is an extensive and possibly dangerous disorder that requires early detection for effective treatment. Add specific global statistics on skin cancer prevalence and mortality to emphasize the importance of early detection. Example: "Skin cancer accounts for 1 in 5 diagnosed cancers globally, with melanoma causing over 60,000 deaths annually. Manual skin cancer screening is both time-intensive and expensive. Deep learning (DL) techniques have shown exceptional performance in various applications and have been applied to systematize skin cancer diagnosis. However, training DL models for skin cancer diagnosis is challenging due to limited available data and the risk of overfitting. Traditionally approaches have High computational costs, a lack of interpretability, deal with numerous hyperparameters and spatial variation have always been problems with machine learning (ML) and DL. An innovative method called adaptive learning has been developed to overcome these problems. In this research, we advise an intelligent computer-aided system for automatic skin cancer diagnosis using a two-stage transfer learning approach and Pre-trained Convolutional Neural Networks (CNNs). CNNs are well-suited for learning hierarchical features from images. Annotated skin cancer photographs are utilized to detect ROIs and reset the initial layer of the pre-trained CNN. The lower-level layers learn about the characteristics and patterns of lesions and unaffected areas by fine-tuning the model. To capture high-level, global features specific to skin cancer, we replace the fully connected (FC) layers, responsible for encoding such features, with a new FC layer based on principal component analysis (PCA). This unsupervised technique enables the mining of discriminative features from the skin cancer images, effectively mitigating overfitting concerns and letting the model adjust structural features of skin cancer images, facilitating effective detection of skin cancer features. The system shows great potential in facilitating the initial screening of skin cancer patients, empowering healthcare professionals to make timely decisions regarding patient referrals to dermatologists or specialists for further diagnosis and appropriate treatment. Our advanced adaptive fine-tuned CNN approach for automatic skin cancer diagnosis offers a valuable tool for efficient and accurate early detection. By leveraging DL and transfer learning techniques, the system has the possible to transform skin cancer diagnosis and improve patient outcomes.

Automatic segmentation and quantitative analysis of brain CT volume in 2-year-olds using deep learning model

Objective

Our research aims to develop an automated method for segmenting brain CT images in healthy 2-year-old children using the ResU-Net deep learning model. Building on this model, we aim to quantify the volumes of specific brain regions and establish a normative reference database for clinical and research applications.

Methods

In this retrospective study, we included 1,487 head CT scans of 2-year-old children showing normal radiological findings, which were divided into training (n = 1,041) and testing (n = 446) sets. We preprocessed the Brain CT images by resampling, intensity normalization, and skull stripping. Then, we trained the ResU-Net model on the training set and validated it on the testing set. In addition, we compared the performance of the ResU-Net model with different kernel sizes (3 × 3 × 3 and 1 × 3 × 3 convolution kernels) against the baseline model, which was the standard 3D U-Net. The performance of the model was evaluated using the Dice similarity score. Once the segmentation model was established, we derived the regional volume parameters. We then conducted statistical analyses to evaluate differences in brain volumes by sex and hemisphere, and performed a Spearman correlation analysis to assess the relationship between brain volume and age.

Results

The ResU-Net model we proposed achieved a Dice coefficient of 0.94 for the training set and 0.96 for the testing set, demonstrating robust segmentation performance. When comparing different models, ResU-Net (3,3,3) model achieved the highest Dice coefficient of 0.96 in the testing set, followed by ResU-Net (1,3,3) model with 0.92, and the baseline 3D U-Net with 0.88. Statistical analysis showed that the brain volume of males was significantly larger than that of females in all brain regions (p < 0.05), and age was positively correlated with the volume of each brain region. In addition, specific structural asymmetries were observed between the right and left hemispheres.

Conclusion

This study highlights the effectiveness of deep learning for automatic brain segmentation in pediatric CT imaging, providing a reliable reference for normative brain volumes in 2-year-old children. The findings may serve as a benchmark for clinical assessment and research, complementing existing MRI-based reference data and addressing the need for accessible, population-based standards in pediatric neuroimaging.

Deep learning-based classification of lymphedema and other lower limb edema diseases using clinical images

Lymphedema is a chronic condition characterized by lymphatic fluid accumulation, primarily affecting the limbs. Its diagnosis is challenging due to symptom overlap with conditions like chronic venous insufficiency (CVI), deep vein thrombosis (DVT), and systemic diseases, often leading to diagnostic delays that can extend up to ten years. These delays negatively impact patient outcomes and burden healthcare systems. Conventional diagnostic methods rely heavily on clinical expertise, which may fail to distinguish subtle variations between these conditions. This study investigates the application of artificial intelligence (AI), specifically deep learning, to improve diagnostic accuracy for lower limb edema. A dataset of 1622 clinical images was used to train sixteen convolutional neural networks (CNNs) and transformer-based models, including EfficientNetV2, which achieved the highest accuracy of 78.6%. Grad-CAM analyses enhanced model interpretability, highlighting clinically relevant features such as swelling and hyperpigmentation. The AI system consistently outperformed human evaluators, whose diagnostic accuracy plateaued at 62.7%. The findings underscore the transformative potential of AI as a diagnostic tool, particularly in distinguishing conditions with overlapping clinical presentations. By integrating AI with clinical workflows, healthcare systems can reduce diagnostic delays, enhance accuracy, and alleviate the burden on medical professionals. While promising, the study acknowledges limitations, such as dataset diversity and the controlled evaluation environment, which necessitate further validation in real-world settings. This research highlights the potential of AI-driven diagnostics to revolutionize lymphedema care, bridging gaps in conventional methods and supporting healthcare professionals in delivering more precise and timely interventions. Future work should focus on external validation and hybrid systems integrating AI and clinical expertise for comprehensive diagnostic solutions.

A multi-stage multi-modal learning algorithm with adaptive multimodal fusion for improving multi-label skin lesion classification

Skin cancer is frequently occurring and has become a major contributor to both cancer incidence and mortality. Accurate and timely diagnosis of skin cancer holds the potential to save lives. Deep learning-based methods have demonstrated significant advancements in the screening of skin cancers. However, most current approaches rely on a single modality input for diagnosis, thereby missing out on valuable complementary information that could enhance accuracy. Although some multimodal-based methods exist, they often lack adaptability and fail to fully leverage multimodal information. In this paper, we introduce a novel uncertainty-based hybrid fusion strategy for a multi-modal learning algorithm aimed at skin cancer diagnosis. Our approach specifically combines three different modalities: clinical images, dermoscopy images, and metadata, to make the final classification. For the fusion of two image modalities, we employ an intermediate fusion strategy that considers the similarity between clinical and dermoscopy images to extract features containing both complementary and correlated information. To capture the correlated information, we utilize cosine similarity, and we employ concatenation as the means for integrating complementary information. In the fusion of image and metadata modalities, we leverage uncertainty to obtain confident late fusion results, allowing our method to adaptively combine the information from different modalities. We conducted comprehensive experiments using a popular publicly available skin disease diagnosis dataset, and the results of these experiments demonstrate the effectiveness of our proposed method. Our proposed fusion algorithm could enhance the clinical applicability of automated skin lesion classification, offering a more robust and adaptive way to make automatic diagnoses with the help of uncertainty mechanism. Code is available at https://github.com/Zuo-Lihan/CosCatNet-Adaptive_Fusion_Algorithm.

Skin Cancer Detection Using Deep Learning Approaches

Aim: This review examined multiple deep learning (DL) methods, including artificial neural networks (ANNs), convolutional neural networks (CNNs), k-nearest neighbors (KNNs), as well as generative adversarial networks (GANs), relying on their abilities to differentially extract key features for the identification and classification of skin lesions. Background: Skin cancer is among the most prevalent cancer types in humans and is associated with tremendous socioeconomic and psychological burdens for patients and caregivers alike. Incidences of skin cancers have progressively increased during the last decades. Early diagnoses of skin cancers may aid in the implementation of more effective treatment and therapeutic regimens. Indeed, several recent studies have focused on early detection strategies for skin cancer. Among the lesion features that can aid the recognition and characterization of skin cancers are symmetry, color, size, and shape. Results: Our assessment indicates that CNNs delivered maximum accuracy in visual lesion recognition, yet GANs have surfaced as a strong tool for training augmentation through simulated image creation. However, there were significant limitations associated with existing datasets, such as provision of insufficient skin tone variability, demanding computational needs, and unequal lesion representations, which may hamper efficiency, inclusivity, and generalizability of DL models. Researchers must combine diverse high-resolution datasets within a structural framework to develop efficient computational models with unsupervised learning methods to enhance noninvasive and precise skin cancer detection. Conclusion: The breakthroughs in image-based computational skin cancer detection may be crucial in reducing the requirement of invasive diagnostic tests and expanding the scope of skin cancer screening toward broad demographics, thereby aiding early cancer detection in a time- and cost-efficient manner.

Semi-supervised tissue segmentation from histopathological images with consistency regularization and uncertainty estimation

Pathologists have depended on their visual experience to assess tissue structures in smear images, which was time-consuming, error-prone, and inconsistent. Deep learning, particularly Convolutional Neural Networks (CNNs), offers the ability to automate this procedure by recognizing patterns in tissue images. However, training these models necessitates huge amounts of labeled data, which can be difficult to come by due to the skill required for annotation and the unavailability of data, particularly for rare diseases. This work introduces a new semi-supervised method for tissue structure semantic segmentation in histopathological images. The study presents a CNN based teacher model that generates pseudo-labels to train a student model, aiming to overcome the drawbacks of conventional supervised learning approaches. Self-supervised training is used to improve the teacher model's performance on smaller datasets. Consistency regularization is integrated to efficiently train the student model on labeled data. Further, the study uses Monte Carlo dropout to estimate the uncertainty of proposed model. The proposed model demonstrated promising results by achieving an mIoU score of 0.64 on a public dataset, highlighting its potential to improve segmentation accuracy in histopathological image analysis.

A comprehensive analysis of deep learning and transfer learning techniques for skin cancer classification

Accurately and early diagnosis of melanoma is one of the challenging tasks due to its unique characteristics and different shapes of skin lesions. So, in order to solve this issue, the current study examines various deep learning-based approaches and provide an effective approach for classifying dermoscopic images into two categories of skin lesions. This research focus on skin cancer images and provides solution using deep learning approaches. This research investigates three approaches for classifying skin cancer images. (1) Utilizing three fine-tuned pre-trained networks (VGG19, ResNet18, and MobileNet_V2) as classifiers. (2) Employing three pre-trained networks (ResNet-18, VGG19, and MobileNet v2) as feature extractors in conjunction with four machine learning classifiers (SVM, DT, Naïve Bayes, and KNN). (3) Utilizing a combination of the aforementioned pre-trained networks as feature extractors in conjunction with same machine learning classifiers. All these algorithms are trained using segmented images which are achieved by using the active contour approach. Prior to segmentation, preprocessing step is performed which involves scaling, denoising, and enhancing the image. Experimental performance is measured on the ISIC 2018 dataset which contains 3300 images of skin disease including benign and malignant type cancer images. 80% of the images from the ISIC 2018 dataset are allocated for training, while the remaining 20% are designated for testing. All approaches are trained using different parameters like epoch, batch size, and learning rate. The results indicate that combining ResNet-18 and MobileNet pre-trained networks using concatenation with an SVM classifier achieved the maximum accuracy of 92.87%.

Training State-of-the-Art Deep Learning Algorithms with Visible and Extended Near-Infrared Multispectral Images of Skin Lesions for the Improvement of Skin Cancer Diagnosis

An estimated 60,000 people die annually from skin cancer, predominantly melanoma. The diagnosis of skin lesions primarily relies on visual inspection, but around half of lesions pose diagnostic challenges, often necessitating a biopsy. Non-invasive detection methods like Computer-Aided Diagnosis (CAD) using Deep Learning (DL) are becoming more prominent. This study focuses on the use of multispectral (MS) imaging to improve skin lesion classification of DL models. We trained two convolutional neural networks (CNNs)-a simple CNN with six two-dimensional (2D) convolutional layers and a custom VGG-16 model with three-dimensional (3D) convolutional layers-using a dataset of MS images. The dataset included spectral cubes from 327 nevi, 112 melanomas, and 70 basal cell carcinomas (BCCs). We compared the performance of the CNNs trained with full spectral cubes versus using only three spectral bands closest to RGB wavelengths. The custom VGG-16 model achieved a classification accuracy of 71% with full spectral cubes and 45% with RGB-simulated images. The simple CNN achieved an accuracy of 83% with full spectral cubes and 36% with RGB-simulated images, demonstrating the added value of spectral information. These results confirm that MS imaging provides complementary information beyond traditional RGB images, contributing to improved classification performance. Although the dataset size remains a limitation, the findings indicate that MS imaging has significant potential for enhancing skin lesion diagnosis, paving the way for further advancements as larger datasets become available.

Diagnosis and prognosis of melanoma from dermoscopy images using machine learning and deep learning: a systematic literature review

BACKGROUND

Melanoma is a highly aggressive skin cancer, where early and accurate diagnosis is crucial to improve patient outcomes. Dermoscopy, a non-invasive imaging technique, aids in melanoma detection but can be limited by subjective interpretation. Recently, machine learning and deep learning techniques have shown promise in enhancing diagnostic precision by automating the analysis of dermoscopy images.

METHODS

This systematic review examines recent advancements in machine learning (ML) and deep learning (DL) applications for melanoma diagnosis and prognosis using dermoscopy images. We conducted a thorough search across multiple databases, ultimately reviewing 34 studies published between 2016 and 2024. The review covers a range of model architectures, including DenseNet and ResNet, and discusses datasets, methodologies, and evaluation metrics used to validate model performance.

RESULTS

Our results highlight that certain deep learning architectures, such as DenseNet and DCNN demonstrated outstanding performance, achieving over 95% accuracy on the HAM10000, ISIC and other datasets for melanoma detection from dermoscopy images. The review provides insights into the strengths, limitations, and future research directions of machine learning and deep learning methods in melanoma diagnosis and prognosis. It emphasizes the challenges related to data diversity, model interpretability, and computational resource requirements.

CONCLUSION

This review underscores the potential of machine learning and deep learning methods to transform melanoma diagnosis through improved diagnostic accuracy and efficiency. Future research should focus on creating accessible, large datasets and enhancing model interpretability to increase clinical applicability. By addressing these areas, machine learning and deep learning models could play a central role in advancing melanoma diagnosis and patient care.

Skin Lesion Classification Through Test Time Augmentation and Explainable Artificial Intelligence

Despite significant advancements in the automatic classification of skin lesions using artificial intelligence (AI) algorithms, skepticism among physicians persists. This reluctance is primarily due to the lack of transparency and explainability inherent in these models, which hinders their widespread acceptance in clinical settings. The primary objective of this study is to develop a highly accurate AI-based algorithm for skin lesion classification that also provides visual explanations to foster trust and confidence in these novel diagnostic tools. By improving transparency, the study seeks to contribute to earlier and more reliable diagnoses. Additionally, the research investigates the impact of Test Time Augmentation (TTA) on the performance of six Convolutional Neural Network (CNN) architectures, which include models from the EfficientNet, ResNet (Residual Network), and ResNeXt (an enhanced variant of ResNet) families. To improve the interpretability of the models' decision-making processes, techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE) and Gradient-weighted Class Activation Mapping (Grad-CAM) are employed. t-SNE is utilized to visualize the high-dimensional latent features of the CNNs in a two-dimensional space, providing insights into how the models group different skin lesion classes. Grad-CAM is used to generate heatmaps that highlight the regions of input images that influence the model's predictions. Our findings reveal that Test Time Augmentation enhances the balanced multi-class accuracy of CNN models by up to 0.3%, achieving a balanced accuracy rate of 97.58% on the International Skin Imaging Collaboration (ISIC 2019) dataset. This performance is comparable to, or marginally better than, more complex approaches such as Vision Transformers (ViTs), demonstrating the efficacy of our methodology.

A clinical pilot trial of an artificial intelligence-driven smart phone application of bowel preparation for colonoscopy: a randomized clinical trial

BACKGROUND

High-quality bowel preparation is paramount for a successful colonoscopy. This study aimed to explore the effect of artificial intelligence-driven smartphone software on the quality of bowel preparation.

METHODS

Firstly, we utilized 3305 valid liquid dung images collected via mobile phones as training data. the most effective model was employed on mobile phones to evaluate the quality of bowel preparation. Secondly, From May 2023 to September 2023, colonoscopy patients were randomly assigned to two groups - the AI group (n = 116) and the control group (n = 116) - using a randomized, controlled, endoscopist-blinded method. We compared the two groups in terms of Boston Bowel Preparation Scale (BBPS) scores, polyp detection rate, adverse reaction rate, and factors related to bowel preparation quality. The primary endpoint was the percentage of patients who achieved a BBPS ≥6 among those who effectively utilized the smartphone software.

RESULTS

EfficientNetV2 exhibited the highest performance, with an accuracy of 87%, a sensitivity of 83%, and an AUC of 0.86. In the patient validation experiment, the AI group had higher BBPS scores than the control group (6.78 ± 1.41 vs. 5.35 ± 2.01, p = 0.001) and showed an improvement in the detection rate (71.55% vs. 56.90%, p = 0.020) for polyps. Multifactor logistic analysis indicated that compliance with enema solution usage rules (OR: 5.850, 95% confidence interval: 2.022-16.923), total water intake (OR: 1.001, 95% confidence interval: 1.001-1.002), and AI software reminders (OR: 2.316, 95% confidence interval: 1.096-4.893) were independently associated with BBPS scores ≥6.

CONCLUSION

Compared with traditional methods, the use of artificial intelligence combined with software to send reminders can lead to more accurate assessments of bowel preparation quality and an improved detection rate for polyps, thus demonstrating promising clinical value.

Attention-Guided Learning With Feature Reconstruction for Skin Lesion Diagnosis Using Clinical and Ultrasound Images

Skin lesion is one of the most common diseases, and most categories are highly similar in morphology and appearance. Deep learning models effectively reduce the variability between classes and within classes, and improve diagnostic accuracy. However, the existing multi-modal methods are only limited to the surface information of lesions in skin clinical and dermatoscopic modalities, which hinders the further improvement of skin lesion diagnostic accuracy. This requires us to further study the depth information of lesions in skin ultrasound. In this paper, we propose a novel skin lesion diagnosis network, which combines clinical and ultrasound modalities to fuse the surface and depth information of the lesion to improve diagnostic accuracy. Specifically, we propose an attention-guided learning (AL) module that fuses clinical and ultrasound modalities from both local and global perspectives to enhance feature representation. The AL module consists of two parts, attention-guided local learning (ALL) computes the intra-modality and inter-modality correlations to fuse multi-scale information, which makes the network focus on the local information of each modality, and attention-guided global learning (AGL) fuses global information to further enhance the feature representation. In addition, we propose a feature reconstruction learning (FRL) strategy which encourages the network to extract more discriminative features and corrects the focus of the network to enhance the model's robustness and certainty. We conduct extensive experiments and the results confirm the superiority of our proposed method. Our code is available at: https://github.com/XCL-hub/AGFnet.

Development and validation of a CT-based deep learning radiomics signature to predict lymph node metastasis in oropharyngeal squamous cell carcinoma: a multicentre study

OBJECTIVES

Lymph node metastasis (LNM) is a pivotal determinant that influences the treatment strategies and prognosis for oropharyngeal squamous cell carcinoma (OPSCC) patients. This study aims to establish and verify a deep learning (DL) radiomics model for the prediction of LNM in OPSCCs using contrast-enhanced computed tomography (CECT).

METHODS

A retrospective analysis included 279 OPSCC patients from 3 institutions. CECT images were used for handcrafted (HCR) and DL feature extraction. Dimensionality reduction for HCR features used recursive feature elimination (RFE) and least absolute shrinkage and selection operator (LASSO) algorithms, whereas DL feature dimensionality reduction used variance-threshold and RFE algorithms. Radiomics signatures were constructed using six machine learning classifiers. A combined model was then constructed using the screened DL, HCR, and clinical features. The area under the receiver operating characteristic curve (AUC) served to quantify the model's performance, and calibration curves were utilized to assess its calibration.

RESULTS

The combined model exhibited robust performance, achieving AUC values of 0.909 (95% CI, 0.861-0.957) in the training cohort, 0.884 (95% CI, 0.800-0.968) in the internal validation cohort, and 0.865 (95% CI, 0.791-0.939) in the external validation cohort. It outperformed both the clinical model and best-performing radiomics model. Moreover, calibration was deemed satisfactory.

CONCLUSIONS

The combined model based on CECT demonstrates the potential to predict LNM in OPSCCs preoperatively, offering a valuable tool for more precise and tailored treatment strategies.

ADVANCES IN KNOWLEDGE

This study presents a novel combined model integrating clinical factors with DL radiomics, significantly enhancing preoperative LNM prediction in OPSCC.

Artificial Intelligence-Aided Diagnosis System for the Detection and Classification of Private-Part Skin Diseases: Decision Analytical Modeling Study

BACKGROUND

Private-part skin diseases (PPSDs) can cause a patient's stigma, which may hinder the early diagnosis of these diseases. Artificial intelligence (AI) is an effective tool to improve the early diagnosis of PPSDs, especially in preventing the deterioration of skin tumors in private parts such as Paget disease. However, to our knowledge, there is currently no research on using AI to identify PPSDs due to the complex backgrounds of the lesion areas and the challenges in data collection.

OBJECTIVE

This study aimed to develop and evaluate an AI-aided diagnosis system for the detection and classification of PPSDs: aiding patients in self-screening and supporting dermatologists' diagnostic enhancement.

METHODS

In this decision analytical modeling study, a 2-stage AI-aided diagnosis system was developed to classify PPSDs. In the first stage, a multitask detection network was trained to automatically detect and classify skin lesions (type, color, and shape). In the second stage, we proposed a knowledge graph based on dermatology expertise and constructed a decision network to classify seven PPSDs (condyloma acuminatum, Paget disease, eczema, pearly penile papules, genital herpes, syphilis, and Bowen disease). A reader study with 13 dermatologists of different experience levels was conducted. Dermatologists were asked to classify the testing cohort under reading room conditions, first without and then with system support. This AI-aided diagnostic study used the data of 635 patients from two institutes between July 2019 and April 2022. The data of Institute 1 contained 2701 skin lesion samples from 520 patients, which were used for the training of the multitask detection network in the first stage. In addition, the data of Institute 2 consisted of 115 clinical images and the corresponding medical records, which were used for the test of the whole 2-stage AI-aided diagnosis system.

RESULTS

On the test data of Institute 2, the proposed system achieved the average precision, recall, and F1-score of 0.81, 0.86, and 0.83, respectively, better than existing advanced algorithms. For the reader performance test, our system improved the average F1-score of the junior, intermediate, and senior dermatologists by 16%, 7%, and 4%, respectively.

CONCLUSIONS

In this study, we constructed the first skin-lesion-based dataset and developed the first AI-aided diagnosis system for PPSDs. This system provides the final diagnosis result by simulating the diagnostic process of dermatologists. Compared with existing advanced algorithms, this system is more accurate in identifying PPSDs. Overall, our system can not only help patients achieve self-screening and alleviate their stigma but also assist dermatologists in diagnosing PPSDs.

Decoding skin cancer classification: perspectives, insights, and advances through researchers' lens

Skin cancer is a significant global health concern, with timely and accurate diagnosis playing a critical role in improving patient outcomes. In recent years, computer-aided diagnosis systems have emerged as powerful tools for automated skin cancer classification, revolutionizing the field of dermatology. This survey analyzes 107 research papers published over the last 18 years, providing a thorough evaluation of advancements in classification techniques, with a focus on the growing integration of computer vision and artificial intelligence (AI) in enhancing diagnostic accuracy and reliability. The paper begins by presenting an overview of the fundamental concepts of skin cancer, addressing underlying challenges in accurate classification, and highlighting the limitations of traditional diagnostic methods. Extensive examination is devoted to a range of datasets, including the HAM10000 and the ISIC archive, among others, commonly employed by researchers. The exploration then delves into machine learning techniques coupled with handcrafted features, emphasizing their inherent limitations. Subsequent sections provide a comprehensive investigation into deep learning-based approaches, encompassing convolutional neural networks, transfer learning, attention mechanisms, ensemble techniques, generative adversarial networks, vision transformers, and segmentation-guided classification strategies, detailing various architectures, tailored for skin lesion analysis. The survey also sheds light on the various hybrid and multimodal techniques employed for classification. By critically analyzing each approach and highlighting its limitations, this survey provides researchers with valuable insights into the latest advancements, trends, and gaps in skin cancer classification. Moreover, it offers clinicians practical knowledge on the integration of AI tools to enhance diagnostic decision-making processes. This comprehensive analysis aims to bridge the gap between research and clinical practice, serving as a guide for the AI community to further advance the state-of-the-art in skin cancer classification systems.

Artificial Intelligence in the Non-Invasive Detection of Melanoma

Skin cancer is one of the most prevalent cancers worldwide, with increasing incidence. Skin cancer is typically classified as melanoma or non-melanoma skin cancer. Although melanoma is less common than basal or squamous cell carcinomas, it is the deadliest form of cancer, with nearly 8300 Americans expected to die from it each year. Biopsies are currently the gold standard in diagnosing melanoma; however, they can be invasive, expensive, and inaccessible to lower-income individuals. Currently, suspicious lesions are triaged with image-based technologies, such as dermoscopy and confocal microscopy. While these techniques are useful, there is wide inter-user variability and minimal training for dermatology residents on how to properly use these devices. The use of artificial intelligence (AI)-based technologies in dermatology has emerged in recent years to assist in the diagnosis of melanoma that may be more accessible to all patients and more accurate than current methods of screening. This review explores the current status of the application of AI-based algorithms in the detection of melanoma, underscoring its potential to aid dermatologists in clinical practice. We specifically focus on AI application in clinical imaging, dermoscopic evaluation, algorithms that can distinguish melanoma from non-melanoma skin cancers, and in vivo skin imaging devices.

A Unified Multi-Task Learning Model with Joint Reverse Optimization for Simultaneous Skin Lesion Segmentation and Diagnosis

Classifying and segmenting skin cancer represent pivotal objectives for automated diagnostic systems that utilize dermoscopy images. However, these tasks present significant challenges due to the diverse shape variations of skin lesions and the inherently fuzzy nature of dermoscopy images, including low contrast and the presence of artifacts. Given the robust correlation between the classification of skin lesions and their segmentation, we propose that employing a combined learning method holds the promise of considerably enhancing the performance of both tasks. In this paper, we present a unified multi-task learning strategy that concurrently classifies abnormalities of skin lesions and allows for the joint segmentation of lesion boundaries. This approach integrates an optimization technique known as joint reverse learning, which fosters mutual enhancement through extracting shared features and limiting task dominance across the two tasks. The effectiveness of the proposed method was assessed using two publicly available datasets, ISIC 2016 and PH2, which included melanoma and benign skin cancers. In contrast to the single-task learning strategy, which solely focuses on either classification or segmentation, the experimental findings demonstrated that the proposed network improves the diagnostic capability of skin tumor screening and analysis. The proposed method achieves a significant segmentation performance on skin lesion boundaries, with Dice Similarity Coefficients (DSC) of 89.48% and 88.81% on the ISIC 2016 and PH2 datasets, respectively. Additionally, our multi-task learning approach enhances classification, increasing the F1 score from 78.26% (baseline ResNet50) to 82.07% on ISIC 2016 and from 82.38% to 85.50% on PH2. This work showcases its potential applicability across varied clinical scenarios.

Analysis of international publication trends in artificial intelligence in skin cancer

Bibliometric methods were used to analyze publications on the use of artificial intelligence (AI) in skin cancer from 2010 to 2022, aiming to explore current publication trends and future directions. A comprehensive search using four terms, "artificial intelligence," "machine learning," "deep learning," and "skin cancer," was performed in the Web of Science database for original English language publications on AI in skin cancer from 2010 to 2022. We visually analyzed publication, citation, and coupling information, focusing on authors, countries and regions, publishing journals, institutions, and core keywords. The analysis of 989 publications revealed a consistent year-on-year increase in publications from 2010 to 2022 (0.51% versus 33.57%). The United States, India, and China emerged as the leading contributors. IEEE Access was identified as the most prolific journal in this area. Key journals and influential authors were highlighted. Examination of the top 10 most cited publications highlights the significant potential of AI in oncology. Co-citation network analysis identified four primary categories of classical literature on AI in skin tumors. Keyword analysis indicated that "melanoma," "classification," and "deep learning" were the most prevalent keywords, suggesting that deep learning for melanoma diagnosis and grading is the current research focus. The term "pigmented skin lesions" showed the strongest burst and longest duration, whereas "texture" was the latest emerging keyword. AI represents a rapidly growing area of research in skin cancer with the potential to significantly improve skin cancer management. Future research will likely focus on machine learning and deep learning technologies for screening and diagnostic purposes.

TPAFNet: Transformer-Driven Pyramid Attention Fusion Network for 3D Medical Image Segmentation

The field of 3D medical image segmentation is witnessing a growing trend in the utilization of combined networks that integrate convolutional neural networks and transformers. Nevertheless, prevailing hybrid networks are confronted with limitations in their straightforward serial or parallel combination methods and lack an effective mechanism to fuse channel and spatial feature attention. To address these limitations, we present a robust multi-scale 3D medical image segmentation network, the Transformer-Driven Pyramid Attention Fusion Network, which is denoted as TPAFNet, leveraging a hybrid structure of CNN and transformer. Within this framework, we exploit the characteristics of atrous convolution to extract multi-scale information effectively, thereby enhancing the encoding results of the transformer. Furthermore, we introduce the TPAF block in the encoder to seamlessly fuse channel and spatial feature attention from multi-scale feature inputs. In contrast to conventional skip connections that simply concatenate or add features, our decoder is enriched with a TPAF connection, elevating the integration of feature attention between low-level and high-level features. Additionally, we propose a low-level encoding shortcut from the original input to the decoder output, preserving more original image features and contributing to enhanced results. Finally, the deep supervision is implemented using a novel CNN-based voxel-wise classifier to facilitate better network convergence. Experimental results demonstrate that TPAFNet significantly outperforms other state-of-the-art networks on two public datasets, indicating that our research can effectively improve the accuracy of medical image segmentation, thereby assisting doctors in making more precise diagnoses.

LesionNet: an automated approach for skin lesion classification using SIFT features with customized convolutional neural network

Accurate detection of skin lesions through computer-aided diagnosis has emerged as a critical advancement in dermatology, addressing the inefficiencies and errors inherent in manual visual analysis. Despite the promise of automated diagnostic approaches, challenges such as image size variability, hair artifacts, color inconsistencies, ruler markers, low contrast, lesion dimension differences, and gel bubbles must be overcome. Researchers have made significant strides in binary classification problems, particularly in distinguishing melanocytic lesions from normal skin conditions. Leveraging the "MNIST HAM10000" dataset from the International Skin Image Collaboration, this study integrates Scale-Invariant Feature Transform (SIFT) features with a custom convolutional neural network model called LesionNet. The experimental results reveal the model's robustness, achieving an impressive accuracy of 99.28%. This high accuracy underscores the effectiveness of combining feature extraction techniques with advanced neural network models in enhancing the precision of skin lesion detection.

Multi-grained contrastive representation learning for label-efficient lesion segmentation and onset time classification of acute ischemic stroke

Ischemic lesion segmentation and the time since stroke (TSS) onset classification from paired multi-modal MRI imaging of unwitnessed acute ischemic stroke (AIS) patients is crucial, which supports tissue plasminogen activator (tPA) thrombolysis decision-making. Deep learning methods demonstrate superiority in TSS classification. However, they often overfit task-irrelevant features due to insufficient paired labeled data, resulting in poor generalization. We observed that unpaired data are readily available and inherently carry task-relevant cues, but are less often considered and explored. Based on this, in this paper, we propose to fully excavate the potential of unpaired unlabeled data and use them to facilitate the downstream AIS analysis task. We first analyze the utility of features at the varied grain and propose a multi-grained contrastive learning (MGCL) framework to learn task-related prior representations from both coarse-grained and fine-grained levels. The former can learn global prior representations to enhance the location ability for the ischemic lesions and perceive the healthy surroundings, while the latter can learn local prior representations to enhance the perception ability for semantic relation between the ischemic lesion and other health regions. To better transfer and utilize the learned task-related representation, we designed a novel multi-task framework to simultaneously achieve ischemic lesion segmentation and TSS classification with limited labeled data. In addition, a multi-modal region-related feature fusion module is proposed to enable the feature correlation and synergy between multi-modal deep image features for more accurate TSS decision-making. Extensive experiments on the large-scale multi-center MRI dataset demonstrate the superiority of the proposed framework. Therefore, it is promising that it helps better stroke evaluation and treatment decision-making.

Intersection-union dual-stream cross-attention Lova-SwinUnet for skin cancer hair segmentation and image repair

Skin cancer images have hair occlusion problems, which greatly affects the accuracy of diagnosis and classification. Current dermoscopic hair removal methods use segmentation networks to locate hairs, and then uses repair networks to perform image repair. However, it is difficult to segment hair and capture the overall structure between hairs because of the hair being thin, unclear, and similar in color to the entire image. When conducting image restoration tasks, the only available images are those obstructed by hair, and there is no corresponding ground truth (supervised data) of the same scene without hair obstruction. In addition, the texture information and structural information used in existing repair methods are often insufficient, which leads to poor results in skin cancer image repair. To address these challenges, we propose the intersection-union dual-stream cross-attention Lova-SwinUnet (IUDC-LS). Firstly, we propose the Lova-SwinUnet module, which embeds Lovasz loss function into Swin-Unet, enabling the network to better capture features of various scales, thus obtaining better hair mask segmentation results. Secondly, we design the intersection-union (IU) module, which takes the mask results obtained in the previous step for pairwise intersection or union, and then overlays the results on the skin cancer image without hair to generate the labeled training data. This turns the unsupervised image repair task into the supervised one. Finally, we propose the dual-stream cross-attention (DC) module, which makes texture information and structure information interact with each other, and then uses cross-attention to make the network pay attention to the more important texture information and structure information in the fusion process of texture information and structure information, so as to improve the effect of image repair. The experimental results show that the PSNR index and SSIM index of the proposed method are increased by 5.4875 and 0.0401 compared with the other common methods. Experimental results unequivocally demonstrate the effectiveness of our approach, which serves as a potent tool for skin cancer detection, significantly surpassing the performance of comparable methods.

CHNet: A multi-task global-local Collaborative Hybrid Network for KRAS mutation status prediction in colorectal cancer

Accurate prediction of Kirsten rat sarcoma (KRAS) mutation status is crucial for personalized treatment of advanced colorectal cancer patients. However, despite the excellent performance of deep learning models in certain aspects, they often overlook the synergistic promotion among multiple tasks and the consideration of both global and local information, which can significantly reduce prediction accuracy. To address these issues, this paper proposes an innovative method called the Multi-task Global-Local Collaborative Hybrid Network (CHNet) aimed at more accurately predicting patients' KRAS mutation status. CHNet consists of two branches that can extract global and local features from segmentation and classification tasks, respectively, and exchange complementary information to collaborate in executing these tasks. Within the two branches, we have designed a Channel-wise Hybrid Transformer (CHT) and a Spatial-wise Hybrid Transformer (SHT). These transformers integrate the advantages of both Transformer and CNN, employing cascaded hybrid attention and convolution to capture global and local information from the two tasks. Additionally, we have created an Adaptive Collaborative Attention (ACA) module to facilitate the collaborative fusion of segmentation and classification features through guidance. Furthermore, we introduce a novel Class Activation Map (CAM) loss to encourage CHNet to learn complementary information between the two tasks. We evaluate CHNet on the T2-weighted MRI dataset, and achieve an accuracy of 88.93% in KRAS mutation status prediction, which outperforms the performance of representative KRAS mutation status prediction methods. The results suggest that our CHNet can more accurately predict KRAS mutation status in patients via a multi-task collaborative facilitation and considering global-local information way, which can assist doctors in formulating more personalized treatment strategies for patients.

Skin Type Diversity in Skin Lesion Datasets: A Review

Purpose of review

Skin type diversity in image datasets refers to the representation of various skin types. This diversity allows for the verification of comparable performance of a trained model across different skin types. A widespread problem in datasets involving human skin is the lack of verifiable diversity in skin types, making it difficult to evaluate whether the performance of the trained models generalizes across different skin types. For example, the diversity issues in skin lesion datasets, which are used to train deep learning-based models, often result in lower accuracy for darker skin types that are typically under-represented in these datasets. Under-representation in datasets results in lower performance in deep learning models for under-represented skin types.

Recent findings

This issue has been discussed in previous works; however, the reporting of skin types, and inherent diversity, have not been fully assessed. Some works report skin types but do not attempt to assess the representation of each skin type in datasets. Others, focusing on skin lesions, identify the issue but do not measure skin type diversity in the datasets examined.

Summary

Effort is needed to address these shortcomings and move towards facilitating verifiable diversity. Building on previous works in skin lesion datasets, this review explores the general issue of skin type diversity by investigating and evaluating skin lesion datasets specifically. The main contributions of this work are an evaluation of publicly available skin lesion datasets and their metadata to assess the frequency and completeness of reporting of skin type and an investigation into the diversity and representation of each skin type within these datasets.

Supplementary Information

The online version contains material available at 10.1007/s13671-024-00440-0.

Skin-CAD: Explainable deep learning classification of skin cancer from dermoscopic images by feature selection of dual high-level CNNs features and transfer learning

Skin cancer (SC) significantly impacts many individuals' health all over the globe. Hence, it is imperative to promptly identify and diagnose such conditions at their earliest stages using dermoscopic imaging. Computer-aided diagnosis (CAD) methods relying on deep learning techniques especially convolutional neural networks (CNN) can effectively address this issue with outstanding outcomes. Nevertheless, such black box methodologies lead to a deficiency in confidence as dermatologists are incapable of comprehending and verifying the predictions that were made by these models. This article presents an advanced an explainable artificial intelligence (XAI) based CAD system named "Skin-CAD" which is utilized for the classification of dermoscopic photographs of SC. The system accurately categorises the photographs into two categories: benign or malignant, and further classifies them into seven subclasses of SC. Skin-CAD employs four CNNs of different topologies and deep layers. It gathers features out of a pair of deep layers of every CNN, particularly the final pooling and fully connected layers, rather than merely depending on attributes from a single deep layer. Skin-CAD applies the principal component analysis (PCA) dimensionality reduction approach to minimise the dimensions of pooling layer features. This also reduces the complexity of the training procedure compared to using deep features from a CNN that has a substantial size. Furthermore, it combines the reduced pooling features with the fully connected features of each CNN. Additionally, Skin-CAD integrates the dual-layer features of the four CNNs instead of entirely depending on the features of a single CNN architecture. In the end, it utilizes a feature selection step to determine the most important deep attributes. This helps to decrease the general size of the feature set and streamline the classification process. Predictions are analysed in more depth using the local interpretable model-agnostic explanations (LIME) approach. This method is used to create visual interpretations that align with an already existing viewpoint and adhere to recommended standards for general clarifications. Two benchmark datasets are employed to validate the efficiency of Skin-CAD which are the Skin Cancer: Malignant vs. Benign and HAM10000 datasets. The maximum accuracy achieved using Skin-CAD is 97.2 % and 96.5 % for the Skin Cancer: Malignant vs. Benign and HAM10000 datasets respectively. The findings of Skin-CAD demonstrate its potential to assist professional dermatologists in detecting and classifying SC precisely and quickly.

Skin cancer classification based on an optimized convolutional neural network and multicriteria decision-making

Skin cancer is a type of cancer disease in which abnormal alterations in skin characteristics can be detected. It can be treated if it is detected early. Many artificial intelligence-based models have been developed for skin cancer detection and classification. Considering the development of numerous models according to various scenarios and selecting the optimum model was rarely considered in previous works. This study aimed to develop various models for skin cancer classification and select the optimum model. Convolutional neural networks (CNNs) in the form of AlexNet, Inception V3, MobileNet V2, and ResNet 50 were used for feature extraction. Feature reduction was carried out using two algorithms of the grey wolf optimizer (GWO) in addition to using the original features. Skin cancer images were classified into four classes based on six machine learning (ML) classifiers. As a result, 51 models were developed with different combinations of CNN algorithms, without GWO algorithms, with two GWO algorithms, and with six ML classifiers. To select the optimum model with the best results, the multicriteria decision-making approach was utilized to rank the alternatives by perimeter similarity (RAPS). Model training and testing were conducted using the International Skin Imaging Collaboration (ISIC) 2017 dataset. Based on nine evaluation metrics and according to the RAPS method, the AlexNet algorithm with a classical GWO yielded the optimum model, achieving a classification accuracy of 94.5%. This work presents the first study on benchmarking skin cancer classification with many models. Feature reduction not only reduces the time spent on training but also improves classification accuracy. The RAPS method has proven its robustness in the problem of selecting the best model for skin cancer classification.

ELA-Net: An Efficient Lightweight Attention Network for Skin Lesion Segmentation

In clinical conditions limited by equipment, attaining lightweight skin lesion segmentation is pivotal as it facilitates the integration of the model into diverse medical devices, thereby enhancing operational efficiency. However, the lightweight design of the model may face accuracy degradation, especially when dealing with complex images such as skin lesion images with irregular regions, blurred boundaries, and oversized boundaries. To address these challenges, we propose an efficient lightweight attention network (ELANet) for the skin lesion segmentation task. In ELANet, two different attention mechanisms of the bilateral residual module (BRM) can achieve complementary information, which enhances the sensitivity to features in spatial and channel dimensions, respectively, and then multiple BRMs are stacked for efficient feature extraction of the input information. In addition, the network acquires global information and improves segmentation accuracy by putting feature maps of different scales through multi-scale attention fusion (MAF) operations. Finally, we evaluate the performance of ELANet on three publicly available datasets, ISIC2016, ISIC2017, and ISIC2018, and the experimental results show that our algorithm can achieve 89.87%, 81.85%, and 82.87% of the mIoU on the three datasets with a parametric of 0.459 M, which is an excellent balance between accuracy and lightness and is superior to many existing segmentation methods.

Deep Learning Algorithms for the Detection of Suspicious Pigmented Skin Lesions in Primary Care Settings: A Systematic Review and Meta-Analysis

Early detection of suspicious pigmented skin lesions is crucial for improving the outcomes and survival rates of skin cancers. However, the accuracy of clinical diagnosis by primary care physicians (PCPs) is suboptimal, leading to unnecessary referrals and biopsies. In recent years, deep learning (DL) algorithms have shown promising results in the automated detection and classification of skin lesions. This systematic review and meta-analysis aimed to evaluate the diagnostic performance of DL algorithms for the detection of suspicious pigmented skin lesions in primary care settings. A comprehensive literature search was conducted using electronic databases, including PubMed, Scopus, IEEE Xplore, Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science. Data from eligible studies were extracted, including study characteristics, sample size, algorithm type, sensitivity, specificity, diagnostic odds ratio (DOR), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and receiver operating characteristic curve analysis. Three studies were included. The results showed that DL algorithms had a high sensitivity (90%, 95% CI: 90-91%) and specificity (85%, 95% CI: 84-86%) for detecting suspicious pigmented skin lesions in primary care settings. Significant heterogeneity was observed in both sensitivity (p = 0.0062, I2 = 80.3%) and specificity (p < 0.001, I2 = 98.8%). The analysis of DOR and PLR further demonstrated the strong diagnostic performance of DL algorithms. The DOR was 26.39, indicating a strong overall diagnostic performance of DL algorithms. The PLR was 4.30, highlighting the ability of these algorithms to influence diagnostic outcomes positively. The NLR was 0.16, indicating that a negative test result decreased the odds of misdiagnosis. The area under the curve of DL algorithms was 0.95, indicating excellent discriminative ability in distinguishing between benign and malignant pigmented skin lesions. DL algorithms have the potential to significantly improve the detection of suspicious pigmented skin lesions in primary care settings. Our analysis showed that DL exhibited promising performance in the early detection of suspicious pigmented skin lesions. However, further studies are needed.

Performance evaluation of E-VGG19 model: Enhancing real-time skin cancer detection and classification

Skin cancer is a pervasive and potentially life-threatening disease. Early detection plays a crucial role in improving patient outcomes. Machine learning (ML) techniques, particularly when combined with pre-trained deep learning models, have shown promise in enhancing the accuracy of skin cancer detection. In this paper, we enhanced the VGG19 pre-trained model with max pooling and dense layer for the prediction of skin cancer. Moreover, we also explored the pre-trained models such as Visual Geometry Group 19 (VGG19), Residual Network 152 version 2 (ResNet152v2), Inception-Residual Network version 2 (InceptionResNetV2), Dense Convolutional Network 201 (DenseNet201), Residual Network 50 (ResNet50), Inception version 3 (InceptionV3), For training, skin lesions dataset is used with malignant and benign cases. The models extract features and divide skin lesions into two categories: malignant and benign. The features are then fed into machine learning methods, including Linear Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Decision Tree (DT), Logistic Regression (LR) and Support Vector Machine (SVM), our results demonstrate that combining E-VGG19 model with traditional classifiers significantly improves the overall classification accuracy for skin cancer detection and classification. Moreover, we have also compared the performance of baseline classifiers and pre-trained models with metrics (recall, F1 score, precision, sensitivity, and accuracy). The experiment results provide valuable insights into the effectiveness of various models and classifiers for accurate and efficient skin cancer detection. This research contributes to the ongoing efforts to create automated technologies for detecting skin cancer that can help healthcare professionals and individuals identify potential skin cancer cases at an early stage, ultimately leading to more timely and effective treatments.

CSDNet: A Novel Deep Learning Framework for Improved Cataract State Detection

Cataracts, known for lens clouding and being a common cause of visual impairment, persist as a primary contributor to vision loss and blindness, presenting notable diagnostic and prognostic challenges. This work presents a novel framework called the Cataract States Detection Network (CSDNet), which utilizes deep learning methods to improve the detection of cataract states. The aim is to create a framework that is more lightweight and adaptable for use in environments or devices with limited memory or storage capacity. This involves reducing the number of trainable parameters while still allowing for effective learning of representations from data. Additionally, the framework is designed to be suitable for real-time or near-real-time applications where rapid inference is essential. This study utilizes cataract and normal images from the Ocular Disease Intelligent Recognition (ODIR) database. The suggested model employs smaller kernels, fewer training parameters, and layers to efficiently decrease the number of trainable parameters, thereby lowering computational costs and average running time compared to other pre-trained models such as VGG19, ResNet50, DenseNet201, MIRNet, Inception V3, Xception, and Efficient net B0. The experimental results illustrate that the proposed approach achieves a binary classification accuracy of 97.24% (normal or cataract) and an average cataract state detection accuracy of 98.17% (normal, grade 1-minimal cloudiness, grade 2-immature cataract, grade 3-mature cataract, and grade 4-hyper mature cataract), competing with state-of-the-art cataract detection methods. The resulting model is lightweight at 17 MB and has fewer trainable parameters (175, 617), making it suitable for deployment in environments or devices with constrained memory or storage capacity. With a runtime of 212 ms, it is well-suited for real-time or near-real-time applications requiring rapid inference.

Sample self-selection using dual teacher networks for pathological image classification with noisy labels

Deep neural networks (DNNs) involve advanced image processing but depend on large quantities of high-quality labeled data. The presence of noisy data significantly degrades the DNN model performance. In the medical field, where model accuracy is crucial and labels for pathological images are scarce and expensive to obtain, the need to handle noisy data is even more urgent. Deep networks exhibit a memorization effect, they tend to prioritize remembering clean labels initially. Therefore, early stopping is highly effective in managing learning with noisy labels. Previous research has often concentrated on developing robust loss functions or implementing training constraints to mitigate the impact of noisy labels; however, such approaches have frequently resulted in underfitting. We propose using knowledge distillation to slow the learning process of the target network rather than preventing late-stage training from being affected by noisy labels. In this paper, we introduce a data sample self-selection strategy based on early stopping to filter out most of the noisy data. Additionally, we employ the distillation training method with dual teacher networks to ensure the steady learning of the student network. The experimental results show that our method outperforms current state-of-the-art methods for handling noisy labels on both synthetic and real-world noisy datasets. In particular, on the real-world pathological image dataset Chaoyang, the highest classification accuracy increased by 2.39 %. Our method leverages the model's predictions based on training history to select cleaner datasets and retrains them using these cleaner datasets, significantly mitigating the impact of noisy labels on model performance.

LSCS-Net: A lightweight skin cancer segmentation network with densely connected multi-rate atrous convolution

The rising occurrence and notable public health consequences of skin cancer, especially of the most challenging form known as melanoma, have created an urgent demand for more advanced approaches to disease management. The integration of modern computer vision methods into clinical procedures offers the potential for enhancing the detection of skin cancer . The UNet model has gained prominence as a valuable tool for this objective, continuously evolving to tackle the difficulties associated with the inherent diversity of dermatological images. These challenges stem from diverse medical origins and are further complicated by variations in lighting, patient characteristics, and hair density. In this work, we present an innovative end-to-end trainable network crafted for the segmentation of skin cancer . This network comprises an encoder-decoder architecture, a novel feature extraction block, and a densely connected multi-rate Atrous convolution block. We evaluated the performance of the proposed lightweight skin cancer segmentation network (LSCS-Net) on three widely used benchmark datasets for skin lesion segmentation: ISIC 2016, ISIC 2017, and ISIC 2018. The generalization capabilities of LSCS-Net are testified by the excellent performance on breast cancer and thyroid nodule segmentation datasets. The empirical findings confirm that LSCS-net attains state-of-the-art results, as demonstrated by a significantly elevated Jaccard index.

The effect of hair removal and filtering on melanoma detection: a comparative deep learning study with AlexNet CNN

Melanoma is the most aggressive and prevalent form of skin cancer globally, with a higher incidence in men and individuals with fair skin. Early detection of melanoma is essential for the successful treatment and prevention of metastasis. In this context, deep learning methods, distinguished by their ability to perform automated and detailed analysis, extracting melanoma-specific features, have emerged. These approaches excel in performing large-scale analysis, optimizing time, and providing accurate diagnoses, contributing to timely treatments compared to conventional diagnostic methods. The present study offers a methodology to assess the effectiveness of an AlexNet-based convolutional neural network (CNN) in identifying early-stage melanomas. The model is trained on a balanced dataset of 10,605 dermoscopic images, and on modified datasets where hair, a potential obstructive factor, was detected and removed allowing for an assessment of how hair removal affects the model's overall performance. To perform hair removal, we propose a morphological algorithm combined with different filtering techniques for comparison: Fourier, Wavelet, average blur, and low-pass filters. The model is evaluated through 10-fold cross-validation and the metrics of accuracy, recall, precision, and the F1 score. The results demonstrate that the proposed model performs the best for the dataset where we implemented both a Wavelet filter and hair removal algorithm. It has an accuracy of 91.30%, a recall of 87%, a precision of 95.19%, and an F1 score of 90.91%.

CosSIF: Cosine similarity-based image filtering to overcome low inter-class variation in synthetic medical image datasets

Crafting effective deep learning models for medical image analysis is a complex task, particularly in cases where the medical image dataset lacks significant inter-class variation. This challenge is further aggravated when employing such datasets to generate synthetic images using generative adversarial networks (GANs), as the output of GANs heavily relies on the input data. In this research, we propose a novel filtering algorithm called Cosine Similarity-based Image Filtering (CosSIF). We leverage CosSIF to develop two distinct filtering methods: Filtering Before GAN Training (FBGT) and Filtering After GAN Training (FAGT). FBGT involves the removal of real images that exhibit similarities to images of other classes before utilizing them as the training dataset for a GAN. On the other hand, FAGT focuses on eliminating synthetic images with less discriminative features compared to real images used for training the GAN. The experimental results reveal that the utilization of either the FAGT or FBGT method reduces low inter-class variation in clinical image classification datasets and enables GANs to generate synthetic images with greater discriminative features. Moreover, modern transformer and convolutional-based models, trained with datasets that utilize these filtering methods, lead to less bias toward the majority class, more accurate predictions of samples in the minority class, and overall better generalization capabilities. Code and implementation details are available at: https://github.com/mominul-ssv/cossif.

An Efficient Multi-Task Synergetic Network for Polyp Segmentation and Classification

Colonoscopy is considered the best diagnostic tool for early detection and resection of polyps, which can effectively prevent consequential colorectal cancer. In clinical practice, segmenting and classifying polyps from colonoscopic images have a great significance since they provide precious information for diagnosis and treatment. In this study, we propose an efficient multi-task synergetic network (EMTS-Net) for concurrent polyp segmentation and classification, and we introduce a polyp classification benchmark for exploring the potential correlations of the above-mentioned two tasks. This framework is composed of an enhanced multi-scale network (EMS-Net) for coarse-grained polyp segmentation, an EMTS-Net (Class) for accurate polyp classification, and an EMTS-Net (Seg) for fine-grained polyp segmentation. Specifically, we first obtain coarse segmentation masks by using EMS-Net. Then, we concatenate these rough masks with colonoscopic images to assist EMTS-Net (Class) in locating and classifying polyps precisely. To further enhance the segmentation performance of polyps, we propose a random multi-scale (RMS) training strategy to eliminate the interference caused by redundant information. In addition, we design an offline dynamic class activation mapping (OFLD CAM) generated by the combined effect of EMTS-Net (Class) and RMS strategy, which optimizes bottlenecks between multi-task networks efficiently and elegantly and helps EMTS-Net (Seg) to perform more accurate polyp segmentation. We evaluate the proposed EMTS-Net on the polyp segmentation and classification benchmarks, and it achieves an average mDice of 0.864 in polyp segmentation and an average AUC of 0.913 with an average accuracy of 0.924 in polyp classification. Quantitative and qualitative evaluations on the polyp segmentation and classification benchmarks demonstrate that our EMTS-Net achieves the best performance and outperforms previous state-of-the-art methods in terms of both efficiency and generalization.

Deep learning approach for skin melanoma and benign classification using empirical wavelet decomposition

BACKGROUND

Melanoma is a malignant skin cancer that causes high mortality. Early detection of melanoma can save patients' lives. The features of the skin lesion images can be extracted using computer techniques to differentiate early between melanoma and benign skin lesions.

OBJECTIVE

A new model of empirical wavelet decomposition (EWD) based on tan hyperbolic modulated filter banks (THMFBs) (EWD-THMFBs) was used to obtain the features of skin lesion images by MATLAB software.

METHODS

The EWD-THMFBs model was compared with the empirical short-time Fourier decomposition method based on THMFBs (ESTFD-THMFBs) and the empirical Fourier decomposition method based on THMFBs (EFD-THMFBs).

RESULTS

The accuracy rates obtained for EWD-THMFBs, ESTFD-THMFBs, and EFD-THMFBs models were 100%, 98.89%, and 83.33%, respectively. The area under the curve (AUC) was 1, 0.97, and 0.91, respectively.

CONCLUSION

The EWD-THMFBs model performed best in extracting features from skin lesion images. This model can be programmed on a mobile to detect skin lesions in rural areas by a nurse before consulting a dermatologist.

Detection for melanoma skin cancer through ACCF, BPPF, and CLF techniques with machine learning approach

Intense sun exposure is a major risk factor for the development of melanoma, an abnormal proliferation of skin cells. Yet, this more prevalent type of skin cancer can also develop in less-exposed areas, such as those that are shaded. Melanoma is the sixth most common type of skin cancer. In recent years, computer-based methods for imaging and analyzing biological systems have made considerable strides. This work investigates the use of advanced machine learning methods, specifically ensemble models with Auto Correlogram Methods, Binary Pyramid Pattern Filter, and Color Layout Filter, to enhance the detection accuracy of Melanoma skin cancer. These results suggest that the Color Layout Filter model of the Attribute Selection Classifier provides the best overall performance. Statistics for ROC, PRC, Kappa, F-Measure, and Matthews Correlation Coefficient were as follows: 90.96% accuracy, 0.91 precision, 0.91 recall, 0.95 ROC, 0.87 PRC, 0.87 Kappa, 0.91 F-Measure, and 0.82 Matthews Correlation Coefficient. In addition, its margins of error are the smallest. The research found that the Attribute Selection Classifier performed well when used in conjunction with the Color Layout Filter to improve image quality.

A deep sift convolutional neural networks for total brain volume estimation from 3D ultrasound images

Preterm infants are a highly vulnerable population. The total brain volume (TBV) of these infants can be accurately estimated by brain ultrasound (US) imaging which enables a longitudinal study of early brain growth during Neonatal Intensive Care (NICU) admission. Automatic estimation of TBV from 3D images increases the diagnosis speed and evades the necessity for an expert to manually segment 3D images, which is a sophisticated and time consuming task. We develop a deep-learning approach to estimate TBV from 3D ultrasound images. It benefits from deep convolutional neural networks (CNN) with dilated residual connections and an additional layer, inspired by the fuzzy c-Means (FCM), to further separate the features into different regions, i.e. sift layer. Therefore, we call this method deep-sift convolutional neural networks (DSCNN). The proposed method is validated against three state-of-the-art methods including AlexNet-3D, ResNet-3D, and VGG-3D, for TBV estimation using two datasets acquired from two different ultrasound devices. The results highlight a strong correlation between the predictions and the observed TBV values. The regression activation maps are used to interpret DSCNN, allowing TBV estimation by exploring those pixels that are more consistent and plausible from an anatomical standpoint. Therefore, it can be used for direct estimation of TBV from 3D images without needing further image segmentation.

Consistent representation via contrastive learning for skin lesion diagnosis

BACKGROUND

Skin lesions are a prevalent ailment, with melanoma emerging as a particularly perilous variant. Encouragingly, artificial intelligence displays promising potential in early detection, yet its integration within clinical contexts, particularly involving multi-modal data, presents challenges. While multi-modal approaches enhance diagnostic efficacy, the influence of modal bias is often disregarded.

METHODS

In this investigation, a multi-modal feature learning technique termed "Contrast-based Consistent Representation Disentanglement" for dermatological diagnosis is introduced. This approach employs adversarial domain adaptation to disentangle features from distinct modalities, fostering a shared representation. Furthermore, a contrastive learning strategy is devised to incentivize the model to preserve uniformity in common lesion attributes across modalities. Emphasizing the learning of a uniform representation among models, this approach circumvents reliance on supplementary data.

RESULTS

Assessment of the proposed technique on a 7-point criteria evaluation dataset yields an average accuracy of 76.1% for multi-classification tasks, surpassing researched state-of-the-art methods. The approach tackles modal bias, enabling the acquisition of a consistent representation of common lesion appearances across diverse modalities, which transcends modality boundaries. This study underscores the latent potential of multi-modal feature learning in dermatological diagnosis.

CONCLUSION

In summation, a multi-modal feature learning strategy is posited for dermatological diagnosis. This approach outperforms other state-of-the-art methods, underscoring its capacity to enhance diagnostic precision for skin lesions.

Federated and Transfer Learning Methods for the Classification of Melanoma and Nonmelanoma Skin Cancers: A Prospective Study

Skin cancer is considered a dangerous type of cancer with a high global mortality rate. Manual skin cancer diagnosis is a challenging and time-consuming method due to the complexity of the disease. Recently, deep learning and transfer learning have been the most effective methods for diagnosing this deadly cancer. To aid dermatologists and other healthcare professionals in classifying images into melanoma and nonmelanoma cancer and enabling the treatment of patients at an early stage, this systematic literature review (SLR) presents various federated learning (FL) and transfer learning (TL) techniques that have been widely applied. This study explores the FL and TL classifiers by evaluating them in terms of the performance metrics reported in research studies, which include true positive rate (TPR), true negative rate (TNR), area under the curve (AUC), and accuracy (ACC). This study was assembled and systemized by reviewing well-reputed studies published in eminent fora between January 2018 and July 2023. The existing literature was compiled through a systematic search of seven well-reputed databases. A total of 86 articles were included in this SLR. This SLR contains the most recent research on FL and TL algorithms for classifying malignant skin cancer. In addition, a taxonomy is presented that summarizes the many malignant and non-malignant cancer classes. The results of this SLR highlight the limitations and challenges of recent research. Consequently, the future direction of work and opportunities for interested researchers are established that help them in the automated classification of melanoma and nonmelanoma skin cancers.

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities

BACKGROUND

Machine learning (ML) is considered an important technology for future data analysis in health care.

METHODS

The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers.

RESULTS AND CONCLUSION

In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future.

KEY POINTS

· ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET..

PMJAF-Net: Pyramidal multi-scale joint attention and adaptive fusion network for explainable skin lesion segmentation

Traditional convolutional neural networks have achieved remarkable success in skin lesion segmentation. However, the successive pooling operations and convolutional spans reduce the feature resolution and hinder the dense prediction for spatial information, resulting in blurred boundaries, low accuracy and poor interpretability for irregular lesion segmentation under low contrast. To solve the above issues, a pyramidal multi-scale joint attention and adaptive fusion network for explainable (PMJAF-Net) skin lesion segmentation is proposed. Firstly, an adaptive spatial attention module is designed to establish the long-term correlation between pixels, enrich the global and local contextual information, and refine the detailed features. Subsequently, an efficient pyramidal multi-scale channel attention module is proposed to capture the multi-scale information and edge features by using the pyramidal module. Meanwhile, a channel attention module is devised to establish the long-term correlation between channels and highlight the most related feature channels to capture the multi-scale key information on each channel. Thereafter, a multi-scale adaptive fusion attention module is put forward to efficiently fuse the scale features at different decoding stages. Finally, a novel hybrid loss function based on region salient features and boundary quality is presented to guide the network to learn from map-level, patch-level and pixel-level and to accurately predict the lesion regions with clear boundaries. In addition, visualizing attention weight maps are utilized to visually enhance the interpretability of our proposed model. Comprehensive experiments are conducted on four public skin lesion datasets, and the results demonstrate that the proposed network outperforms the state-of-the-art methods, with the segmentation assessment evaluation metrics Dice, JI, and ACC improved to 92.65%, 87.86% and 96.26%, respectively.

Dermo-Seg: ResNet-UNet Architecture and Hybrid Loss Function for Detection of Differential Patterns to Diagnose Pigmented Skin Lesions

Convolutional neural network (CNN) models have been extensively applied to skin lesions segmentation due to their information discrimination capabilities. However, CNNs' struggle to capture the connection between long-range contexts when extracting deep semantic features from lesion images, resulting in a semantic gap that causes segmentation distortion in skin lesions. Therefore, detecting the presence of differential structures such as pigment networks, globules, streaks, negative networks, and milia-like cysts becomes difficult. To resolve these issues, we have proposed an approach based on semantic-based segmentation (Dermo-Seg) to detect differential structures of lesions using a UNet model with a transfer-learning-based ResNet-50 architecture and a hybrid loss function. The Dermo-Seg model uses ResNet-50 backbone architecture as an encoder in the UNet model. We have applied a combination of focal Tversky loss and IOU loss functions to handle the dataset's highly imbalanced class ratio. The obtained results prove that the intended model performs well compared to the existing models. The dataset was acquired from various sources, such as ISIC18, ISBI17, and HAM10000, to evaluate the Dermo-Seg model. We have dealt with the data imbalance present within each class at the pixel level using our hybrid loss function. The proposed model achieves a mean IOU score of 0.53 for streaks, 0.67 for pigment networks, 0.66 for globules, 0.58 for negative networks, and 0.53 for milia-like-cysts. Overall, the Dermo-Seg model is efficient in detecting different skin lesion structures and achieved 96.4% on the IOU index. Our Dermo-Seg system improves the IOU index compared to the most recent network.

An Efficient Artificial Rabbits Optimization Based on Mutation Strategy For Skin Cancer Prediction

Accurate skin lesion diagnosis is critical for the early detection of melanoma. However, the existing approaches are unable to attain substantial levels of accuracy. Recently, pre-trained Deep Learning (DL) models have been applied to tackle and improve efficiency on tasks such as skin cancer detection instead of training models from scratch. Therefore, we develop a robust model for skin cancer detection with a DL-based model as a feature extraction backbone, which is achieved using MobileNetV3 architecture. In addition, a novel algorithm called the Improved Artificial Rabbits Optimizer (IARO) is introduced, which uses the Gaussian mutation and crossover operator to ignore the unimportant features from those features extracted using MobileNetV3. The PH2, ISIC-2016, and HAM10000 datasets are used to validate the developed approach's efficiency. The empirical results show that the developed approach yields outstanding accuracy results of 87.17% on the ISIC-2016 dataset, 96.79% on the PH2 dataset, and 88.71 % on the HAM10000 dataset. Experiments show that the IARO can significantly improve the prediction of skin cancer.

Early Melanoma Detection Based on a Hybrid YOLOv5 and ResNet Technique

Skin cancer, specifically melanoma, is a serious health issue that arises from the melanocytes, the cells that produce melanin, the pigment responsible for skin color. With skin cancer on the rise, the timely identification of skin lesions is crucial for effective treatment. However, the similarity between some skin lesions can result in misclassification, which is a significant problem. It is important to note that benign skin lesions are more prevalent than malignant ones, which can lead to overly cautious algorithms and incorrect results. As a solution, researchers are developing computer-assisted diagnostic tools to detect malignant tumors early. First, a new model based on the combination of "you only look once" (YOLOv5) and "ResNet50" is proposed for melanoma detection with its degree using humans against a machine with 10,000 training images (HAM10000). Second, feature maps integrate gradient change, which allows rapid inference, boosts precision, and reduces the number of hyperparameters in the model, making it smaller. Finally, the current YOLOv5 model is changed to obtain the desired outcomes by adding new classes for dermatoscopic images of typical lesions with pigmented skin. The proposed approach improves melanoma detection with a real-time speed of 0.4 MS of non-maximum suppression (NMS) per image. The performance metrics average is 99.0%, 98.6%, 98.8%, 99.5, 98.3%, and 98.7% for the precision, recall, dice similarity coefficient (DSC), accuracy, mean average precision (MAP) from 0.0 to 0.5, and MAP from 0.5 to 0.95, respectively. Compared to current melanoma detection approaches, the provided approach is more efficient in using deep features.

Enhancing Skin Cancer Detection and Classification in Dermoscopic Images through Concatenated MobileNetV2 and Xception Models

One of the most promising research initiatives in the healthcare field is focused on the rising incidence of skin cancer worldwide and improving early discovery methods for the disease. The most significant factor in the fatalities caused by skin cancer is the late identification of the disease. The likelihood of human survival may be significantly improved by performing an early diagnosis followed by appropriate therapy. It is not a simple process to extract the elements from the photographs of the tumors that may be used for the prospective identification of skin cancer. Several deep learning models are widely used to extract efficient features for a skin cancer diagnosis; nevertheless, the literature demonstrates that there is still room for additional improvements in various performance metrics. This study proposes a hybrid deep convolutional neural network architecture for identifying skin cancer by adding two main heuristics. These include Xception and MobileNetV2 models. Data augmentation was introduced to balance the dataset, and the transfer learning technique was utilized to resolve the challenges of the absence of labeled datasets. It has been detected that the suggested method of employing Xception in conjunction with MobileNetV2 attains the most excellent performance, particularly concerning the dataset that was evaluated: specifically, it produced 97.56% accuracy, 97.00% area under the curve, 100% sensitivity, 93.33% precision, 96.55% F1 score, and 0.0370 false favorable rates. This research has implications for clinical practice and public health, offering a valuable tool for dermatologists and healthcare professionals in their fight against skin cancer.

Deep Hyperspherical Clustering for Skin Lesion Medical Image Segmentation

Diagnosis of skin lesions based on imaging techniques remains a challenging task because data (knowledge) uncertainty may reduce accuracy and lead to imprecise results. This paper investigates a new deep hyperspherical clustering (DHC) method for skin lesion medical image segmentation by combining deep convolutional neural networks and the theory of belief functions (TBF). The proposed DHC aims to eliminate the dependence on labeled data, improve segmentation performance, and characterize the imprecision caused by data (knowledge) uncertainty. First, the SLIC superpixel algorithm is employed to group the image into multiple meaningful superpixels, aiming to maximize the use of context without destroying the boundary information. Second, an autoencoder network is designed to transform the superpixels' information into potential features. Third, a hypersphere loss is developed to train the autoencoder network. The loss is defined to map the input to a pair of hyperspheres so that the network can perceive tiny differences. Finally, the result is redistributed to characterize the imprecision caused by data (knowledge) uncertainty based on the TBF. The proposed DHC method can well characterize the imprecision between skin lesions and non-lesions, which is particularly important for the medical procedures. A series of experiments on four dermoscopic benchmark datasets demonstrate that the proposed DHC yields better segmentation performance, increasing the accuracy of the predictions while can perceive imprecise regions compared to other typical methods.