Diagnostic assessment of deep learning for melanocytic lesions using whole-slide pathological images

Artificial intelligence and skin cancer

Artificial intelligence is poised to rapidly reshape many fields, including that of skin cancer screening and diagnosis, both as a disruptive and assistive technology. Together with the collection and availability of large medical data sets, artificial intelligence will become a powerful tool that can be leveraged by physicians in their diagnoses and treatment plans for patients. This comprehensive review focuses on current progress toward AI applications for patients, primary care providers, dermatologists, and dermatopathologists, explores the diverse applications of image and molecular processing for skin cancer, and highlights AI's potential for patient self-screening and improving diagnostic accuracy for non-dermatologists. We additionally delve into the challenges and barriers to clinical implementation, paths forward for implementation and areas of active research.

Artificial Intelligence in Melanoma Dermatopathology: A Review of Literature

ABSTRACT

Pathology serves as a promising field to integrate artificial intelligence into clinical practice as a powerful screening tool. Melanoma is a common skin cancer with high mortality and morbidity, requiring timely and accurate histopathologic diagnosis. This review explores applications of artificial intelligence in melanoma dermatopathology, including differential diagnostics, prognosis prediction, and personalized medicine decision-making.

Recent advances of artificial intelligence in melanoma clinical practice

Skin melanoma is a lethal cancer. The incidence of melanoma is increasing rapidly in all regions of the world. Despite significant breakthroughs in melanoma treatment in recent years, precise diagnosis of melanoma is still a challenge in some cases. Even specialized physicians may need time and effort to make accurate judgments. As artificial intelligence (AI) technology advances into medical practice, it may bring new solutions to this problem based on its efficiency, accuracy, and speed. This paper summarizes the recent progress of AI in melanoma-related applications, including melanoma diagnosis and classification, the discovery of new medication, guiding treatment, and prognostic assessment. The paper also compares the effectiveness of various algorithms in melanoma application and suggests future research directions for AI in melanoma clinical practice.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

Self-supervised clustering analysis of colorectal cancer biomarkers based on multi-scale whole slides image and mass spectrometry imaging fused images

Mass spectrometry imaging (MSI) is widely used for unlabeled molecular co-localization in biological samples and is also commonly used for screening cancer biomarkers. The main issues affecting the screening of cancer biomarkers are: 1) low-resolution MSI and pathological slices cannot be accurately matched; 2) a large amount of MSI data cannot be directly analyzed without manual annotation. This paper proposes a self-supervised cluster analysis method for colorectal cancer biomarkers based on multi-scale whole slide images (WSI) and MSI fusion images without manual annotation, which can accurately determine the correlation between molecules and lesion areas. This paper uses the combination of WSI multi-scale high-resolution and MSI high-dimensional data to obtain high-resolution fusion images. This method can observe the spatial distribution of molecules in pathological slices and use this method as an evaluation index for self-supervised screening of cancer biomarkers. The experimental results show that the method proposed in this chapter can train the image fusion model with a small amount of MSI and WSI data, and the mean Pixel Accuracy (mPA) and mean Intersection over Union (mIoU) evaluation metrics of the fused images can reach 0.9587 and 0.8745. And self-supervised clustering using MSI features and fused image features can obtain good classification results, and the precision, recall, and F1-score values of the self-supervised model reach 0.9074, 0.9065, and 0.9069, respectively. This method effectively combines the advantages of WSI and MSI, which will significantly expand the application scenarios of MSI and facilitate the screening of disease markers.

An artificial intelligence model for the pathological diagnosis of invasion depth and histologic grade in bladder cancer

BACKGROUND

Accurate pathological diagnosis of invasion depth and histologic grade is key for clinical management in patients with bladder cancer (BCa), but it is labour-intensive, experience-dependent and subject to interobserver variability. Here, we aimed to develop a pathological artificial intelligence diagnostic model (PAIDM) for BCa diagnosis.

METHODS

A total of 854 whole slide images (WSIs) from 692 patients were included and divided into training and validation sets. The PAIDM was developed using the training set based on the deep learning algorithm ScanNet, and the performance was verified at the patch level in validation set 1 and at the WSI level in validation set 2. An independent validation cohort (validation set 3) was employed to compare the PAIDM and pathologists. Model performance was evaluated using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value and negative predictive value.

RESULTS

The AUCs of the PAIDM were 0.878 (95% CI 0.875-0.881) at the patch level in validation set 1 and 0.870 (95% CI 0.805-0.923) at the WSI level in validation set 2. In comparing the PAIDM and pathologists, the PAIDM achieved an AUC of 0.847 (95% CI 0.779-0.905), which was non-inferior to the average diagnostic level of pathologists. There was high consistency between the model-predicted and manually annotated areas, improving the PAIDM's interpretability.

CONCLUSIONS

We reported an artificial intelligence-based diagnostic model for BCa that performed well in identifying invasion depth and histologic grade. Importantly, the PAIDM performed admirably in patch-level recognition, with a promising application for transurethral resection specimens.

Deep Learning for Skin Melanocytic Tumors in Whole-Slide Images: A Systematic Review

The rise of Artificial Intelligence (AI) has shown promising performance as a support tool in clinical pathology workflows. In addition to the well-known interobserver variability between dermatopathologists, melanomas present a significant challenge in their histological interpretation. This study aims to analyze all previously published studies on whole-slide images of melanocytic tumors that rely on deep learning techniques for automatic image analysis. Embase, Pubmed, Web of Science, and Virtual Health Library were used to search for relevant studies for the systematic review, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Articles from 2015 to July 2022 were included, with an emphasis placed on the used artificial intelligence methods. Twenty-eight studies that fulfilled the inclusion criteria were grouped into four groups based on their clinical objectives, including pathologists versus deep learning models (n = 10), diagnostic prediction (n = 7); prognosis (n = 5), and histological features (n = 6). These were then analyzed to draw conclusions on the general parameters and conditions of AI in pathology, as well as the necessary factors for better performance in real scenarios.

Diagnostic and Prognostic Deep Learning Applications for Histological Assessment of Cutaneous Melanoma

Melanoma is among the most devastating human malignancies. Accurate diagnosis and prognosis are essential to offer optimal treatment. Histopathology is the gold standard for establishing melanoma diagnosis and prognostic features. However, discrepancies often exist between pathologists, and analysis is costly and time-consuming. Deep-learning algorithms are deployed to improve melanoma diagnosis and prognostication from histological images of melanoma. In recent years, the development of these machine-learning tools has accelerated, and machine learning is poised to become a clinical tool to aid melanoma histology. Nevertheless, a review of the advances in machine learning in melanoma histology was lacking. We performed a comprehensive literature search to provide a complete overview of the recent advances in machine learning in the assessment of melanoma based on hematoxylin eosin digital pathology images. In our work, we review 37 recent publications, compare the methods and performance of the reviewed studies, and highlight the variety of promising machine-learning applications in melanoma histology.

Deep learning for the detection of anatomical tissue structures and neoplasms of the skin on scanned histopathological tissue sections

Basal cell carcinoma (BCC), squamous cell carcinoma (SqCC) and melanoma are among the most common cancer types. Correct diagnosis based on histological evaluation after biopsy or excision is paramount for adequate therapy stratification. Deep learning on histological slides has been suggested to complement and improve routine diagnostics, but publicly available curated and annotated data and usable models trained to distinguish common skin tumors are rare and often lack heterogeneous non-tumor categories. A total of 16 classes from 386 cases were manually annotated on scanned histological slides, 129,364 100 x 100 µm (~395 x 395 px) image tiles were extracted and split into a training, validation and test set. An EfficientV2 neuronal network was trained and optimized to classify image categories. Cross entropy loss, balanced accuracy and Matthews correlation coefficient were used for model evaluation. Image and patient data were assessed with confusion matrices. Application of the model to an external set of whole slides facilitated localization of melanoma and non-tumor tissue. Automated differentiation of BCC, SqCC, melanoma, naevi and non-tumor tissue structures was possible, and a high diagnostic accuracy was achieved in the validation (98%) and test (97%) set. In summary, we provide a curated dataset including the most common neoplasms of the skin and various anatomical compartments to enable researchers to train, validate and improve deep learning models. Automated classification of skin tumors by deep learning techniques is possible with high accuracy, facilitates tumor localization and has the potential to support and improve routine diagnostics.

Clinical-grade endometrial cancer detection system via whole-slide images using deep learning

The accurate pathological diagnosis of endometrial cancer (EC) improves the curative effect and reduces the mortality rate. Deep learning has demonstrated expert-level performance in pathological diagnosis of a variety of organ systems using whole-slide images (WSIs). It is urgent to build the deep learning system for endometrial cancer detection using WSIs. The deep learning model was trained and validated using a dataset of 601 WSIs from PUPH. The model performance was tested on three independent datasets containing a total of 1,190 WSIs. For the retrospective test, we evaluated the model performance on 581 WSIs from PUPH. In the prospective study, 317 consecutive WSIs from PUPH were collected from April 2022 to May 2022. To further evaluate the generalizability of the model, 292 WSIs were gathered from PLAHG as part of the external test set. The predictions were thoroughly analyzed by expert pathologists. The model achieved an area under the receiver operating characteristic curve (AUC), sensitivity, and specificity of 0.928, 0.924, and 0.801, respectively, on 1,190 WSIs in classifying EC and non-EC. On the retrospective dataset from PUPH/PLAGH, the model achieved an AUC, sensitivity, and specificity of 0.948/0.971, 0.928/0.947, and 0.80/0.938, respectively. On the prospective dataset, the AUC, sensitivity, and specificity were, in order, 0.933, 0.934, and 0.837. Falsely predicted results were analyzed to further improve the pathologists’ confidence in the model. The deep learning model achieved a high degree of accuracy in identifying EC using WSIs. By pre-screening the suspicious EC regions, it would serve as an assisted diagnostic tool to improve working efficiency for pathologists.

Deep Learning-Based Fully Automated Diagnosis of Melanocytic Lesions by Using Whole Slide Images

Background Erroneous diagnoses of melanocytic lesions (benign, atypical, and malignant types) result in inappropriate surgical treatment plans.Objective To propose a deep learning (DL)-based fully automated diagnostic method using whole slide images (WSIs) for melanocytic lesions.Methods The method consisted of patch prediction using a DL model and patient diagnosis using an aggregation module. The method was developed with 745 WSIs, and evaluated using internal and external testing sets comprising 182 WSIs and 54 WSIs, respectively. The results were compared with those of the classification by one junior and two senior pathologists. Furthermore, we compared the performance of the three pathologists in the classification of melanocytic lesions with and without the assistance of our method.Results The method achieved an accuracy of 0.963 and 0.930 on the internal and external testing sets, respectively, which was significantly higher than that of the junior pathologist (0.419 and 0.535). With assistance from the method, all three pathologists achieved higher accuracy on the internal and external testing sets; the accuracy of the junior pathologist increased by 39.0% and 30.2%, respectively (p < 0.05).Conclusion This generalizable method can accurately classify melanocytic lesions and effectively improve the diagnostic accuracy of pathologists.

Assessment of deep learning assistance for the pathological diagnosis of gastric cancer

Previous studies on deep learning (DL) applications in pathology have focused on pathologist-versus-algorithm comparisons. However, DL will not replace the breadth and contextual knowledge of pathologists; rather, only through their combination may the benefits of DL be achieved. A fully crossed multireader multicase study was conducted to evaluate DL assistance with pathologists' diagnosis of gastric cancer. A total of 110 whole-slide images (WSI) (50 malignant and 60 benign) were interpreted by 16 board-certified pathologists with or without DL assistance, with a washout period between sessions. DL-assisted pathologists achieved a higher area under receiver operating characteristic curve (ROC-AUC) (0.911 vs. 0.863, P = 0.003) than unassisted in interpreting the 110 WSIs. Pathologists with DL assistance demonstrated higher sensitivity in detection of gastric cancer than without (90.63% vs. 82.75%, P = 0.010). No significant difference was observed in specificity with or without deep learning assistance (78.23% vs. 79.90%, P = 0.468). The average review time per WSI was shortened with DL assistance than without (22.68 vs. 26.37 second, P = 0.033). Our results demonstrated that DL assistance indeed improved pathologists' accuracy and efficiency in gastric cancer diagnosis and further boosted the acceptance of this new technique.