Reply to the letter to the Editor "Reply to 'Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists' by H. A. Haenssle et al. " by L. Oakden-Rayner

Transformer-based framework for multi-class segmentation of skin cancer from histopathology images

Introduction

Non-melanoma skin cancer comprising Basal cell carcinoma (BCC), Squamous cell carcinoma (SCC), and Intraepidermal carcinoma (IEC) has the highest incidence rate among skin cancers. Intelligent decision support systems may address the issue of the limited number of subject experts and help in mitigating the parity of health services between urban centers and remote areas.

Method

In this research, we propose a transformer-based model for the segmentation of histopathology images not only into inflammation and cancers such as BCC, SCC, and IEC but also to identify skin tissues and boundaries that are important in decision-making. Accurate segmentation of these tissue types will eventually lead to accurate detection and classification of non-melanoma skin cancer. The segmentation according to tissue types and their visual representation before classification enhances the trust of pathologists and doctors being relatable to how most pathologists approach this problem. The visualization of the confidence of the model in its prediction through uncertainty maps is also what distinguishes this study from most deep learning methods.

Results

The evaluation of proposed system is carried out using publicly available dataset. The application of our proposed segmentation system demonstrated good performance with an F1 score of 0.908, mean intersection over union (mIoU) of 0.653, and average accuracy of 83.1%, advocating that the system can be used as a decision support system successfully and has the potential of subsequently maturing into a fully automated system.

Discussion

This study is an attempt to automate the segmentation of the most occurring non-melanoma skin cancer using a transformer-based deep learning technique applied to histopathology skin images. Highly accurate segmentation and visual representation of histopathology images according to tissue types by the proposed system implies that the system can be used for skin-related routine pathology tasks including cancer and other anomaly detection, their classification, and measurement of surgical margins in the case of cancer cases.

Ability to Predict Melanoma Within 5 Years Using Registry Data and a Convolutional Neural Network: A Proof of Concept Study

Research relating to machine learning algorithms, including convolutional neural networks, has increased during the past 5 years. The aim of this pilot study was to investigate how accurately a convolutional neural network, trained on Swedish registry data, could perform in predicting cutaneous invasive and in situ melanoma (CMM) within 5 years. A cohort of 1,208,393 individuals was used. Registry data ranged from 4 July 2005 to 31 December 2011, predicting CMM between 1 January 2012 and 31 December 2016. A convolutional neural network with one-dimensional convolutions with respect to time was trained using healthcare databases and registers. The algorithm was trained on 23,886 individuals. Validation was performed on a holdout validation set including 6,000 individuals. After training and validation, the convolutional neural network was evaluated on a test set (1,000 individuals with an CMM occurring within 5 years and 5,000 without). The area under the receiver-operating characteristic curve was 0.59 (95% confidence interval (95% CI) 0.57–0.61). The point on the receiver-operating characteristic curve where sensitivity equalled specificity had a value of 56% (sensitivity 95% CI 53–60% and specificity 95% CI 55–58%). Albeit at an early stage, this pilot investigation demonstrates potential usefulness for machine learning algorithms in predicting melanoma risk.

Deep Learning-Based Fully Automated Diagnosis of Melanocytic Lesions by Using Whole Slide Images

Background Erroneous diagnoses of melanocytic lesions (benign, atypical, and malignant types) result in inappropriate surgical treatment plans.Objective To propose a deep learning (DL)-based fully automated diagnostic method using whole slide images (WSIs) for melanocytic lesions.Methods The method consisted of patch prediction using a DL model and patient diagnosis using an aggregation module. The method was developed with 745 WSIs, and evaluated using internal and external testing sets comprising 182 WSIs and 54 WSIs, respectively. The results were compared with those of the classification by one junior and two senior pathologists. Furthermore, we compared the performance of the three pathologists in the classification of melanocytic lesions with and without the assistance of our method.Results The method achieved an accuracy of 0.963 and 0.930 on the internal and external testing sets, respectively, which was significantly higher than that of the junior pathologist (0.419 and 0.535). With assistance from the method, all three pathologists achieved higher accuracy on the internal and external testing sets; the accuracy of the junior pathologist increased by 39.0% and 30.2%, respectively (p < 0.05).Conclusion This generalizable method can accurately classify melanocytic lesions and effectively improve the diagnostic accuracy of pathologists.

A Deep Learning Based Framework for Diagnosing Multiple Skin Diseases in a Clinical Environment

Background: Numerous studies have attempted to apply artificial intelligence (AI) in the dermatological field, mainly on the classification and segmentation of various dermatoses. However, researches under real clinical settings are scarce. Objectives: This study was aimed to construct a novel framework based on deep learning trained by a dataset that represented the real clinical environment in a tertiary class hospital in China, for better adaptation of the AI application in clinical practice among Asian patients. Methods: Our dataset was composed of 13,603 dermatologist-labeled dermoscopic images, containing 14 categories of diseases, namely lichen planus (LP), rosacea (Rosa), viral warts (VW), acne vulgaris (AV), keloid and hypertrophic scar (KAHS), eczema and dermatitis (EAD), dermatofibroma (DF), seborrheic dermatitis (SD), seborrheic keratosis (SK), melanocytic nevus (MN), hemangioma (Hem), psoriasis (Pso), port wine stain (PWS), and basal cell carcinoma (BCC). In this study, we applied Google's EfficientNet-b4 with pre-trained weights on ImageNet as the backbone of our CNN architecture. The final fully-connected classification layer was replaced with 14 output neurons. We added seven auxiliary classifiers to each of the intermediate layer groups. The modified model was retrained with our dataset and implemented using Pytorch. We constructed saliency maps to visualize our network's attention area of input images for its prediction. To explore the visual characteristics of different clinical classes, we also examined the internal image features learned by the proposed framework using t-SNE (t-distributed Stochastic Neighbor Embedding). Results: Test results showed that the proposed framework achieved a high level of classification performance with an overall accuracy of 0.948, a sensitivity of 0.934 and a specificity of 0.950. We also compared the performance of our algorithm with three most widely used CNN models which showed our model outperformed existing models with the highest area under curve (AUC) of 0.985. We further compared this model with 280 board-certificated dermatologists, and results showed a comparable performance level in an 8-class diagnostic task. Conclusions: The proposed framework retrained by the dataset that represented the real clinical environment in our department could accurately classify most common dermatoses that we encountered during outpatient practice including infectious and inflammatory dermatoses, benign and malignant cutaneous tumors.

Artificial intelligence (AI) in medicine as a strategic valuable tool

Humans' creativity led to machines that outperform human capabilities in terms of workload, effectiveness, precision, endurance, strength, and repetitiveness. It has always been a vision and a way to transcend the existence and to give more sense to life, which is precious. The common denominator of all these creations was that they were meant to replace, enhance or go beyond the mechanical capabilities of the human body. The story takes another bifurcation when Alan Turing introduced the concept of a machine that could think, in 1950. Artificial intelligence, presented as a term in 1956, describes the use of computers to imitate intelligence and critical thinking comparable to humans. However, the revolution began in 1943, when artificial neural networks was an attempt to exploit the architecture of the human brain to perform tasks that conventional algorithms had little success with. Artificial intelligence is becoming a research focus and a tool of strategic value. The same observations apply in the field of healthcare, too. In this manuscript, we try to address key questions regarding artificial intelligence in medicine, such as what artificial intelligence is and how it works, what is its value in terms of application in medicine, and what are the prospects?

Case report: removal of a proliferating pilomatricoma with a CO2 laser

Tumors of the skin/skin adnexa are astonishingly diverse. The diagnostic algorithms for skin neoplasms includes history taking, the assessment of clinical data, dermoscopy and a histopathological examination. Literature descriptions of a histologically confirmed pilomatricoma are scarce. If the lesion is localized to an esthetically sensitive body area, it is important to minimize the postoperative cosmetic defect. In the case described below, we were able to achieve a positive esthetic outcome in a patient with a facial pilomatricoma sized < 2 cm2 using a СО2 laser.

The Possibility of Deep Learning-Based, Computer-Aided Skin Tumor Classifiers

The incidence of skin tumors has steadily increased. Although most are benign and do not affect survival, some of the more malignant skin tumors present a lethal threat if a delay in diagnosis permits them to become advanced. Ideally, an inspection by an expert dermatologist would accurately detect malignant skin tumors in the early stage; however, it is not practical for every single patient to receive intensive screening by dermatologists. To overcome this issue, many studies are ongoing to develop dermatologist-level, computer-aided diagnostics. Whereas, many systems that can classify dermoscopic images at this dermatologist-equivalent level have been reported, a much fewer number of systems that can classify conventional clinical images have been reported thus far. Recently, the introduction of deep-learning technology, a method that automatically extracts a set of representative features for further classification has dramatically improved classification efficacy. This new technology has the potential to improve the computer classification accuracy of conventional clinical images to the level of skilled dermatologists. In this review, this new technology and present development of computer-aided skin tumor classifiers will be summarized.