Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities

Hair removal in dermoscopy images using variational autoencoders

BACKGROUND

In recent years, melanoma is rising at a faster rate compared to other cancers. Although it is the most serious type of skin cancer, the diagnosis at early stages makes it curable. Dermoscopy is a reliable medical technique used to detect melanoma by using a dermoscope to examine the skin. In the last few decades, digital imaging devices have made great progress which allowed capturing and storing high-quality images from these examinations. The stored images are now being standardized and used for the automatic detection of melanoma. However, when the hair covers the skin, this makes the task challenging. Therefore, it is important to eliminate the hair to get accurate results.

METHODS

In this paper, we propose a simple yet efficient method for hair removal using a variational autoencoder without the need for paired samples. The encoder takes as input a dermoscopy image and builds a latent distribution that ignores hair as it is considered noise, while the decoder reconstructs a hair-free image. Both encoder and decoder use a decent convolutional neural networks architecture that provides high performance. The construction of our model comprises two stages of training. In the first stage, the model has trained on hair-occluded images to output hair-free images, and in the second stage, it is optimized using hair-free images to preserve the image textures. Although the variational autoencoder produces hair-free images, it does not maintain the quality of the generated images. Thus, we explored the use of three-loss functions including the structural similarity index (SSIM), L1-norm, and L2-norm to improve the visual quality of the generated images.

RESULTS

The evaluation of the hair-free reconstructed images is carried out using t-distributed stochastic neighbor embedding (SNE) feature mapping by visualizing the distribution of the real hair-free images and the synthesized hair-free images. The conducted experiments on the publicly available dataset HAM10000 show that our method is very efficient.

Applying the Digital Health Social Justice Guide

Introduction

Digital health, the use of apps, text-messaging, and online interventions, can revolutionize healthcare and make care more equitable. Currently, digital health interventions are often not designed for those who could benefit most and may have unintended consequences. In this paper, we explain how privacy vulnerabilities and power imbalances, including racism and sexism, continue to influence health app design and research. We provide guidelines for researchers to design, report and evaluate digital health studies to maximize social justice in health.

Methods

From September 2020 to April 2021, we held five discussion and brainstorming sessions with researchers, students, and community partners to develop the guide and the key questions. We additionally conducted an informal literature review, invited experts to review our guide, and identified examples from our own digital health study and other studies.

Results

We identified five overarching topics with key questions and subquestions to guide researchers in designing or evaluating a digital health research study. The overarching topics are: 1. Equitable distribution; 2. Equitable design; 3. Privacy and data return; 4. Stereotype and bias; 5. Structural racism.

Conclusion

We provide a guide with five key topics and questions for social justice digital health research. Encouraging researchers and practitioners to ask these questions will help to spark a transformation in digital health toward more equitable and ethical research. Future work needs to determine if the quality of studies can improve when researchers use this guide.

Machine Learning and Deep Learning Algorithms for Skin Cancer Classification from Dermoscopic Images

We carry out a critical assessment of machine learning and deep learning models for the classification of skin tumors. Machine learning (ML) algorithms tested in this work include logistic regression, linear discriminant analysis, k-nearest neighbors classifier, decision tree classifier and Gaussian naive Bayes, while deep learning (DL) models employed are either based on a custom Convolutional Neural Network model, or leverage transfer learning via the use of pre-trained models (VGG16, Xception and ResNet50). We find that DL models, with accuracies up to 0.88, all outperform ML models. ML models exhibit accuracies below 0.72, which can be increased to up to 0.75 with ensemble learning. To further assess the performance of DL models, we test them on a larger and more imbalanced dataset. Metrics, such as the F-score and accuracy, indicate that, after fine-tuning, pre-trained models perform extremely well for skin tumor classification. This is most notably the case for VGG16, which exhibits an F-score of 0.88 and an accuracy of 0.88 on the smaller database, and metrics of 0.70 and 0.88, respectively, on the larger database.

Imtidad: A Reference Architecture and a Case Study on Developing Distributed AI Services for Skin Disease Diagnosis over Cloud, Fog and Edge

Several factors are motivating the development of preventive, personalized, connected, virtual, and ubiquitous healthcare services. These factors include declining public health, increase in chronic diseases, an ageing population, rising healthcare costs, the need to bring intelligence near the user for privacy, security, performance, and costs reasons, as well as COVID-19. Motivated by these drivers, this paper proposes, implements, and evaluates a reference architecture called Imtidad that provides Distributed Artificial Intelligence (AI) as a Service (DAIaaS) over cloud, fog, and edge using a service catalog case study containing 22 AI skin disease diagnosis services. These services belong to four service classes that are distinguished based on software platforms (containerized gRPC, gRPC, Android, and Android Nearby) and are executed on a range of hardware platforms (Google Cloud, HP Pavilion Laptop, NVIDIA Jetson nano, Raspberry Pi Model B, Samsung Galaxy S9, and Samsung Galaxy Note 4) and four network types (Fiber, Cellular, Wi-Fi, and Bluetooth). The AI models for the diagnosis include two standard Deep Neural Networks and two Tiny AI deep models to enable their execution at the edge, trained and tested using 10,015 real-life dermatoscopic images. The services are evaluated using several benchmarks including model service value, response time, energy consumption, and network transfer time. A DL service on a local smartphone provides the best service in terms of both energy and speed, followed by a Raspberry Pi edge device and a laptop in fog. The services are designed to enable different use cases, such as patient diagnosis at home or sending diagnosis requests to travelling medical professionals through a fog device or cloud. This is the pioneering work that provides a reference architecture and such a detailed implementation and treatment of DAIaaS services, and is also expected to have an extensive impact on developing smart distributed service infrastructures for healthcare and other sectors.

Deep Learning-Based Fully Automated Diagnosis of Melanocytic Lesions by Using Whole Slide Images

Background Erroneous diagnoses of melanocytic lesions (benign, atypical, and malignant types) result in inappropriate surgical treatment plans.Objective To propose a deep learning (DL)-based fully automated diagnostic method using whole slide images (WSIs) for melanocytic lesions.Methods The method consisted of patch prediction using a DL model and patient diagnosis using an aggregation module. The method was developed with 745 WSIs, and evaluated using internal and external testing sets comprising 182 WSIs and 54 WSIs, respectively. The results were compared with those of the classification by one junior and two senior pathologists. Furthermore, we compared the performance of the three pathologists in the classification of melanocytic lesions with and without the assistance of our method.Results The method achieved an accuracy of 0.963 and 0.930 on the internal and external testing sets, respectively, which was significantly higher than that of the junior pathologist (0.419 and 0.535). With assistance from the method, all three pathologists achieved higher accuracy on the internal and external testing sets; the accuracy of the junior pathologist increased by 39.0% and 30.2%, respectively (p < 0.05).Conclusion This generalizable method can accurately classify melanocytic lesions and effectively improve the diagnostic accuracy of pathologists.

Artificial Intelligence and Melanoma: A Comprehensive Review of Clinical, Dermoscopic, and Histologic Applications

Melanoma detection, prognosis, and treatment represent challenging and complex areas of cutaneous oncology with considerable impact on patient outcomes and healthcare economics. Artificial intelligence (AI) applications in these tasks are rapidly developing. Neural networks with increasing levels of sophistication are being implemented in clinical image, dermoscopic image, and histopathologic specimen classification of pigmented lesions. These efforts hold promise of earlier and highly accurate melanoma detection, as well as reliable prognostication and prediction of therapeutic response. Herein, we provide a brief introduction to AI, discuss contemporary investigational applications of AI in melanoma, and summarize challenges encountered with AI.

Artificial Intelligence-Based Approaches to Reflectance Confocal Microscopy Image Analysis in Dermatology

Reflectance confocal microscopy (RCM) is a non-invasive imaging method designed to identify various skin diseases. Confocal based diagnosis may be subjective due to the learning curve of the method, the scarcity of training programs available for RCM, and the lack of clearly defined diagnostic criteria for all skin conditions. Given that in vivo RCM is becoming more widely used in dermatology, numerous deep learning technologies have been developed in recent years to provide a more objective approach to RCM image analysis. Machine learning-based algorithms are used in RCM image quality assessment to reduce the number of artifacts the operator has to view, shorten evaluation times, and decrease the number of patient visits to the clinic. However, the current visual method for identifying the dermal-epidermal junction (DEJ) in RCM images is subjective, and there is a lot of variation. The delineation of DEJ on RCM images could be automated through artificial intelligence, saving time and assisting novice RCM users in studying the key DEJ morphological structure. The purpose of this paper is to supply a current summary of machine learning and artificial intelligence’s impact on the quality control of RCM images, key morphological structures identification, and detection of different skin lesion types on static RCM images.

InSiNet: a deep convolutional approach to skin cancer detection and segmentation

Cancer is among the common causes of death around the world. Skin cancer is one of the most lethal types of cancer. Early diagnosis and treatment are vital in skin cancer. In addition to traditional methods, method such as deep learning is frequently used to diagnose and classify the disease. Expert experience plays a major role in diagnosing skin cancer. Therefore, for more reliable results in the diagnosis of skin lesions, deep learning algorithms can help in the correct diagnosis. In this study, we propose InSiNet, a deep learning-based convolutional neural network to detect benign and malignant lesions. The performance of the method is tested on International Skin Imaging Collaboration HAM10000 images (ISIC 2018), ISIC 2019, and ISIC 2020, under the same conditions. The computation time and accuracy comparison analysis was performed between the proposed algorithm and other machine learning techniques (GoogleNet, DenseNet-201, ResNet152V2, EfficientNetB0, RBF-support vector machine, logistic regression, and random forest). The results show that the developed InSiNet architecture outperforms the other methods achieving an accuracy of 94.59%, 91.89%, and 90.54% in ISIC 2018, 2019, and 2020 datasets, respectively. Since the deep learning algorithms eliminate the human factor during diagnosis, they can give reliable results in addition to traditional methods.

Machine Learning and Its Application in Skin Cancer

Artificial intelligence (AI) has wide applications in healthcare, including dermatology. Machine learning (ML) is a subfield of AI involving statistical models and algorithms that can progressively learn from data to predict the characteristics of new samples and perform a desired task. Although it has a significant role in the detection of skin cancer, dermatology skill lags behind radiology in terms of AI acceptance. With continuous spread, use, and emerging technologies, AI is becoming more widely available even to the general population. AI can be of use for the early detection of skin cancer. For example, the use of deep convolutional neural networks can help to develop a system to evaluate images of the skin to diagnose skin cancer. Early detection is key for the effective treatment and better outcomes of skin cancer. Specialists can accurately diagnose the cancer, however, considering their limited numbers, there is a need to develop automated systems that can diagnose the disease efficiently to save lives and reduce health and financial burdens on the patients. ML can be of significant use in this regard. In this article, we discuss the fundamentals of ML and its potential in assisting the diagnosis of skin cancer.

Non-invasive diagnostic techniques in pigmentary skin disorders and skin cancer

BACKGROUND

Diagnosis of pigmentary skin disorders, pre-cancerous and cancerous skin diseases is traditionally relied on visual assessment. The most widely applied invasive diagnostic technique is the skin biopsy. There have been significant technological advances in non-invasive diagnostic methods for skin disorders.

OBJECTIVE

The objective of this article is to discuss different non-invasive diagnostic modalities, used in the diagnosis of pigmentary skin disorders and cutaneous cancers.

METHODS

Comprehensive literature search was performed to screen articles related to non-invasive diagnostic techniques in pigmentary skin disorders and cutaneous cancers. Articles published in journals indexed in PubMed were searched along with those in Google Scholar. Clinical trials, review articles, case series, case reports and other relevant articles were considered for review. References of relevant articles were also considered for review.

RESULTS

Dermoscopy and ultrasonography were the only non-invasive diagnostic and imaging techniques available to dermatologists for many years. The advent of computed tomography (CT) and magnetic resonance imaging (MRI) augmented the visualization of deeper structures. Confocal laser microscopy (CLM) and reflectance spectrophotometers have showed promising results in the non-invasive detection of pigmented lesions. Optical coherence tomography (OCT), electrical impedance spectroscopy (EIS), multispectral imaging, high frequency ultrasonography (HFUS) and adhesive patch biopsy aid in the accurate diagnosis of benign, as well as neoplastic skin diseases.

CONCLUSION

There have been significant advancements in non-invasive methods for diagnosis of dermatological diseases. These techniques can be repeatedly used in a comfort manner for the patient, and may offer an objective way to follow the course of a disease.

Intelligent Dermatologist Tool for Classifying Multiple Skin Cancer Subtypes by Incorporating Manifold Radiomics Features Categories

The rates of skin cancer (SC) are rising every year and becoming a critical health issue worldwide. SC's early and accurate diagnosis is the key procedure to reduce these rates and improve survivability. However, the manual diagnosis is exhausting, complicated, expensive, prone to diagnostic error, and highly dependent on the dermatologist's experience and abilities. Thus, there is a vital need to create automated dermatologist tools that are capable of accurately classifying SC subclasses. Recently, artificial intelligence (AI) techniques including machine learning (ML) and deep learning (DL) have verified the success of computer-assisted dermatologist tools in the automatic diagnosis and detection of SC diseases. Previous AI-based dermatologist tools are based on features which are either high-level features based on DL methods or low-level features based on handcrafted operations. Most of them were constructed for binary classification of SC. This study proposes an intelligent dermatologist tool to accurately diagnose multiple skin lesions automatically. This tool incorporates manifold radiomics features categories involving high-level features such as ResNet-50, DenseNet-201, and DarkNet-53 and low-level features including discrete wavelet transform (DWT) and local binary pattern (LBP). The results of the proposed intelligent tool prove that merging manifold features of different categories has a high influence on the classification accuracy. Moreover, these results are superior to those obtained by other related AI-based dermatologist tools. Therefore, the proposed intelligent tool can be used by dermatologists to help them in the accurate diagnosis of the SC subcategory. It can also overcome manual diagnosis limitations, reduce the rates of infection, and enhance survival rates.

A Cloud Approach for Melanoma Detection based on Deep Learning Networks

In the era of digitized images, the goal is to be able to extract information from them and create new knowledge thanks to the use of Computer Vision techniques, Machine Learning and Deep Learning. This allows their use for early diagnosis and subsequent determination of the treatment of many pathologies. In the specific case treated here, deep neural networks are used in the dermatological field to distinguish between melanoma and non-melanoma images. In this work we have underlined two essential points of melanoma detection research. The first aspect taken into consideration is how even a simple modification of the parameters in the dataset determines a change of the accuracy of the classifiers, while working on the same original dataset. The second point is the need to have a system architecture that can be more flexible in updating the training datasets for the classification of this pathology. In this context, the proposed architecture reserves the goal of developing and implementing a hybrid architecture based on Cloud, Fog and Edge Computing in order to provide a Melanoma Detection service based on clinical and/or dermoscopic images. At the same time, this architecture must be able to interface with the amount of data to be analyzed by reducing the running time of the necessary computational operations. This has been highlighted with experiments carried out on a single machine and on different distribution systems, highlighting how a distributed approach guarantees the achievement of an output in a much more acceptable time without the need to fully rely on data scientists skills.

Skin lesion image retrieval using transfer learning-based approach for query-driven distance recommendation

Content-Based Dermatological Lesion Retrieval (CBDLR) systems retrieve similar skin lesion images, with a pathology-confirmed diagnosis, for a given query image of a skin lesion. By producing an intuitive support to both inexperienced and experienced dermatologists, the early diagnosis through CBDLR screening can significantly enhance the patients' survival, while reducing the treatment cost. To deal with this issue, a CBDLR system is proposed in this study. This system integrates a similarity measure recommender which allows a dynamic selection of the adequate distance metric for each query image. The main contributions of this work reside in (i) the adoption of deep-learned features according to their performances for the classification of skin lesions into seven classes; and (ii) the automatic generation of ground truth that was investigated within the framework of transfer learning in order to recommend the most appropriate distance for any new query image. The proposed CBDLR system has been exhaustively evaluated using the challenging ISIC2018 and ISIC2019 datasets, and the obtained results show that the proposed system can provide a useful aided-decision while offering superior performances. Indeed, it outperforms similar CBDLR systems that adopt standard distances by at least 9% in terms of mAP@K.

Incorporating clinical knowledge with constrained classifier chain into a multimodal deep network for melanoma detection

In recent years, vast developments in Computer-Aided Diagnosis (CAD) for skin diseases have generated much interest from clinicians and other eventual end-users of this technology. Introducing clinical domain knowledge to these machine learning strategies can help dispel the black box nature of these tools, strengthening clinician trust. Clinical domain knowledge also provides new information channels which can improve CAD diagnostic performance. In this paper, we propose a novel framework for malignant melanoma (MM) detection by fusing clinical images and dermoscopic images. The proposed method combines a multi-labeled deep feature extractor and clinically constrained classifier chain (CC). This allows the 7-point checklist, a clinician diagnostic algorithm, to be included in the decision level while maintaining the clinical importance of the major and minor criteria in the checklist. Our proposed framework achieved an average accuracy of 81.3% for detecting all criteria and melanoma when testing on a publicly available 7-point checklist dataset. This is the highest reported results, outperforming state-of-the-art methods in the literature by 6.4% or more. Analyses also show that the proposed system surpasses the single modality system of using either clinical images or dermoscopic images alone and the systems without adopting the approach of multi-label and clinically constrained classifier chain. Our carefully designed system demonstrates a substantial improvement over melanoma detection. By keeping the familiar major and minor criteria of the 7-point checklist and their corresponding weights, the proposed system may be more accepted by physicians as a human-interpretable CAD tool for automated melanoma detection.

Artificial Intelligence in Dermatopathology: New Insights and Perspectives

In recent years, an increasing enthusiasm has been observed towards artificial intelligence and machine learning, involving different areas of medicine. Among these, although still in the embryonic stage, the dermatopathological field has also been partially involved, with the attempt to develop and train algorithms that could assist the pathologist in the differential diagnosis of complex melanocytic lesions. In this article, we face this new challenge of the modern era, carry out a review of the literature regarding the state of the art and try to determine promising future perspectives.

Correlation of manual semi-quantitative and automated quantitative Ki-67 proliferative index with OncotypeDXTM recurrence score in invasive breast carcinoma

BACKGROUND

Ki-67 immunohistochemistry (IHC) staining is a widely used cancer proliferation assay; however, its limitations could be improved with automated scoring. The OncotypeDXTM Recurrence Score (ORS), which primarily evaluates cancer proliferation genes, is a prognostic indicator for breast cancer chemotherapy response; however, it is more expensive and slower than Ki-67.

OBJECTIVE

To compare manual Ki-67 (mKi-67) with automated Ki-67 (aKi-67) algorithm results based on manually selected Ki-67 "hot spots" in breast cancer, and correlate both with ORS.

METHODS

105 invasive breast carcinoma cases from 100 patients at our institution (2011-2013) with available ORS were evaluated. Concordance was assessed via Cohen's Kappa (κ).

RESULTS

57/105 cases showed agreement between mKi-67 and aKi-67 (κ 0.31, 95% CI 0.18-0.45), with 41 cases overestimated by aKi-67. Concordance was higher when estimated on the same image (κ 0.53, 95% CI 0.37-0.69). Concordance between mKi-67 score and ORS was fair (κ 0.27, 95% CI 0.11-0.42), and concordance between aKi-67 and ORS was poor (κ 0.10, 95% CI -0.03-0.23).

CONCLUSIONS

These results highlight the limits of Ki-67 algorithms that use manual "hot spot" selection. Due to suboptimal concordance, Ki-67 is likely most useful as a complement to, rather than a surrogate for ORS, regardless of scoring method.

DNA methylation association with stage progression of head and neck squamous cell carcinoma

Head and Neck Squamous Cell Carcinoma (HNSCC) is the sixth most common cancer worldwide, which accounts for approximately 6% of all cases and is responsible for an estimated 2% of all cancer deaths. Despite progress in the treatment of squamous cell carcinomas, survival rates remain low. It is a fact that epigenetic modifications have numerous associations with biological processes and complex diseases such as cancer. Hence, a more systematic approach is needed to provide potential screening targets and have an effective therapy method. This study developed a workflow to analyze HM450 methylation arrays with mRNA expression profiles that identified novel signatures of epigenetic regulators for tumor progression. We identified differentially expressed genes and differentially methylated regions and the correlation between associated genes to identify epigenetic modifications underlying regulation roles. We have taken the differentiation direction of expressions into account during the integration of gene expression and DNA methylation modification to detect epigenetic regulators of core genes of tumor-stage progression. Enrichment analysis of selected key genes provides better insight into their functionality. Thus, we have investigated gene copy number alteration and mutations to filter differentially expressed genes, including some members of the fibroblast growth factor family and cyclin-dependent kinase inhibitor family with other potential known regulators. Our analysis has revealed the list of 61 commercial methylation probes positively correlated with 31 differentially expressed genes, which can be associated with HNSC metastasis stages. Most of these genes have already reported potential epigenetic regulators, and their role in cancer progression was studied. We suggest these selected probes of DNA methylation as potential targets of the epigenetic regulators in revealed genes that have displayed significant genetic and epigenetic modification behavior during cancer stage progression and tumor metastasis.

Developing an Artificial Intelligence (A.I)-based descriptor of facial appearance that fits with the assessments of makeup experts

OBJECTIVE

To develop an A.I-based automatic descriptor that detects and grades, from selfie pictures, 23 facial signs, hairs included, as a help to making-up procedures.

MATERIAL AND METHODS

The selfie images taken in very different conditions by 3326 women and men were used to create (90% of dataset) and validate (10% of dataset) a new algorithm architecture to appraise and grade 23 different facial signs such as lips, nose, eye color, eyebrows, eyelashes, and hair color as defined by makeup artists. Each selfie image was annotated by 12 experts and defined references to train Artificial Intelligence (A.I)-based algorithm.

RESULTS

As some the 23 signs present a continuous or discontinuous feature, these were analyzed by two different statistical approaches. The results provided by the automatic descriptor system were not only in good agreement with the expert's assessments but were even found of a better precision and reproducibility. This automatic descriptor system has proven a good and robust accuracy despite the very variable conditions in the acquisition of selfie pictures.

CONCLUSION

Such automatic descriptor system seems providing a valuable help in making-up procedures and may extend to other activities such as Skincare or Haircare. As such it should allow large investigations to better evaluate the consumers' needs of esthetical improvements.

Artificial Intelligence Applications to Improve the Treatment of Locally Advanced Non-Small Cell Lung Cancers

Locally advanced non-small cell lung cancer patients represent around one third of newly diagnosed lung cancer patients. There remains a large unmet need to find treatment strategies that can improve the survival of these patients while minimizing therapeutical side effects. Increasing the availability of patients' data (imaging, electronic health records, patients' reported outcomes, and genomics) will enable the application of AI algorithms to improve therapy selections. In this review, we discuss how artificial intelligence (AI) can be integral to improving clinical decision support systems. To realize this, a roadmap for AI must be defined. We define six milestones involving a broad spectrum of stakeholders, from physicians to patients, that we feel are necessary for an optimal transition of AI into the clinic.

Cutaneous Malignant Melanoma: A Review of Early Diagnosis and Management

Cutaneous melanoma (CM) is a malignant tumor formed from pigment-producing cells called melanocytes. It is one of the most aggressive and fatal forms of skin malignancy. In the last decades, CM's incidence has gradually risen, with 351,880 new cases in 2015. Since the 1960s, its incidence has increased steadily, in 2019, with approximately 96,000 new cases. A greater understanding of early diagnosis and management of CM is urgently needed because of the high mortality rates due to metastatic melanoma. Timely detection of melanoma is crucial for successful treatment, but diagnosis with histopathology may also pose a significant challenge to this objective. Early diagnosis and management are essential and contribute to better survival rates of the patient. To better control this malignancy, such information is expected to be particularly useful in the early detection of possible metastatic lesions and the development of new therapeutic approaches. This article reviews the available information on the early diagnosis and management of CM and discusses such information's potential in facilitating the future prospective.