Automated segmentation of cell membranes to evaluate HER2 status in whole slide images using a modified deep learning network

Self-HER2Net: A generative self-supervised framework for HER2 classification in IHC histopathology of breast cancer

Breast cancer is a significant global health concern, where precise identification of proteins like Human Epidermal Growth Factor Receptor 2 (HER2) in cancer cells via Immunohistochemistry (IHC) is pivotal for treatment decisions. HER2 overexpression is evaluated through HER2 scoring on a scale from 0 to 3 + based on staining patterns and intensity. Recent efforts have been made to automate HER2 scoring using image processing and AI techniques. However, existing methods require large manually annotated datasets as these follow supervised learning paradigms. Therefore, we proposed a generative self-supervised learning (SSL) framework "Self-HER2Net" for the classification of HER2 scoring, to reduce dependence on large manually annotated data by leveraging one of best performing four novel generative self-supervised tasks, that we proposed. The first two SSL tasks HER2hsl and HER2hsv are domain-agnostic and the other two HER2dab and HER2hae are domain-specific SSL tasks focusing on domain-agnostic and domain-specific staining patterns and intensity representation. Our approach is evaluated under different budget scenarios (2 %, 15 %, & 100 % labeled datasets) and also out distribution test. For tile-level assessment, HER2hsv achieved the best performance with AUC-ROC of 0.965 ± 0.037. Our self-supervised learning approach shows potential for application in scenarios with limited annotated data for HER2 analysis.

Fully automatic HER2 tissue segmentation for interpretable HER2 scoring

Breast cancer is the most common cancer in women, with HER2 (human epidermal growth factor receptor 2) overexpression playing a critical role in regulating cell growth and division. HER2 status, assessed according to established scoring guidelines, offers important information for treatment selection. However, the complexity of the task leads to variability in human rater assessments. In this work, we propose a fully automated, interpretable HER2 scoring pipeline based on pixel-level semantic segmentations, designed to align with clinical guidelines. Using polygon annotations, our method balances annotation effort with the ability to capture fine-grained details and larger structures, such as non-invasive tumor tissue. To enhance HER2 segmentation, we propose the use of a Wasserstein Dice loss to model class relationships, ensuring robust segmentation and HER2 scoring performance. Additionally, based on observations of pathologists' behavior in clinical practice, we propose a calibration step to the scoring rules, which positively impacts the accuracy and consistency of automated HER2 scoring. Our approach achieves an F1 score of 0.832 on HER2 scoring, demonstrating its effectiveness. This work establishes a potent segmentation pipeline that can be further leveraged to analyze HER2 expression in breast cancer tissue.

Insights into AI advances in immunohistochemistry for effective breast cancer treatment: a literature review of ER, PR, and HER2 scoring

Breast cancer is a significant health challenge, with accurate and timely diagnosis being critical to effective treatment. Immunohistochemistry (IHC) staining is a widely used technique for the evaluation of breast cancer markers, but manual scoring is time-consuming and can be subject to variability. With the rise of Artificial Intelligence (AI), there is an increasing interest in using machine learning and deep learning approaches to automate the scoring of ER, PR, and HER2 biomarkers in IHC-stained images for effective treatment. This narrative literature review focuses on AI-based techniques for the automated scoring of breast cancer markers in IHC-stained images, specifically Allred, Histochemical (H-Score) and HER2 scoring. We aim to identify the current state-of-the-art approaches, challenges, and potential future research prospects for this area of study. By conducting a comprehensive review of the existing literature, we aim to contribute to the ultimate goal of improving the accuracy and efficiency of breast cancer diagnosis and treatment.

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Breast cancer is the most prevalent cancer among women, and its diagnosis requires the accurate identification and classification of histological features for effective patient management. Artificial intelligence, particularly through deep learning, represents the next frontier in cancer diagnosis and management. Notably, the use of convolutional neural networks and emerging Vision Transformers (ViT) has been reported to automate pathologists' tasks, including tumor detection and classification, in addition to improving the efficiency of pathology services. Deep learning applications have also been extended to the prediction of protein expression, molecular subtype, mutation status, therapeutic efficacy, and outcome prediction directly from hematoxylin and eosin-stained slides, bypassing the need for immunohistochemistry or genetic testing. This review explores the current status and prospects of deep learning in breast cancer diagnosis with a focus on whole-slide image analysis. Artificial intelligence applications are increasingly applied to many tasks in breast pathology ranging from disease diagnosis to outcome prediction, thus serving as valuable tools for assisting pathologists and supporting breast cancer management.

Fully Automated Artificial Intelligence Solution for Human Epidermal Growth Factor Receptor 2 Immunohistochemistry Scoring in Breast Cancer: A Multireader Study

PURPOSE

The proven efficacy of human epidermal growth factor receptor 2 (HER2) antibody-drug conjugate therapy for treating HER2-low breast cancers necessitates more accurate and reproducible HER2 immunohistochemistry (IHC) scoring. We aimed to validate performance and utility of a fully automated artificial intelligence (AI) solution for interpreting HER2 IHC in breast carcinoma.

MATERIALS AND METHODS

A two-arm multireader study of 120 HER2 IHC whole-slide images from four sites assessed HER2 scoring by four surgical pathologists without and with the aid of an AI HER2 solution. Both arms were compared with high-confidence ground truth (GT) established by agreement of at least four of five breast pathology subspecialists according to ASCO/College of American Pathologists (CAP) 2018/2023 guidelines.

RESULTS

The mean interobserver agreement among GT pathologists across all HER2 scores was 72.4% (N = 120). The AI solution demonstrated high accuracy for HER2 scoring, with 92.1% agreement on slides with high confidence GT (n = 92). The use of the AI tool led to improved performance by readers, interobserver agreement increased from 75.0% for digital manual read to 83.7% for AI-assisted review, and scoring accuracy improved from 85.3% to 88.0%. For the distinction of HER2 0 from 1+ cases (n = 58), pathologists supported by AI showed significantly higher interobserver agreement (69.8% without AI v 87.4% with AI) and accuracy (81.9% without AI v 88.8% with AI).

CONCLUSION

This study demonstrated utility of a fully automated AI solution to aid in scoring HER2 IHC accurately according to ASCO/CAP 2018/2023 guidelines. Pathologists supported by AI showed improvements in HER2 IHC scoring consistency and accuracy, especially for distinguishing HER2 0 from 1+ cases. This AI solution could be used by pathologists as a decision support tool for enhancing reproducibility and consistency of HER2 scoring and particularly for identifying HER2-low breast cancers.

Best practices for achieving consensus in HER2-low expression in breast cancer: current perspectives from practising pathologists

AIMS

Human epidermal growth factor receptor 2 (HER2) expression is an important biomarker in breast cancer (BC). Most BC cases categorised as HER2-negative (HER2-) express low levels of HER2 [immunohistochemistry (IHC) 1+ or IHC 2+/in-situ hybridisation not amplified (ISH-)] and represent a clinically relevant therapeutic category that is amenable to targeted therapy using a recently approved HER2-directed antibody-drug conjugate. A group of practising pathologists, with expertise in breast pathology and BC biomarker testing, outline best practices and guidance for achieving consensus in HER2 IHC scoring for BC.

METHODS AND RESULTS

The authors describe current knowledge and challenges of IHC testing and scoring of HER2-low expressing BC and provide best practices and guidance for accurate identification of BCs expressing low levels of HER2. These expert pathologists propose an algorithm for assessing HER2 expression with validated IHC assays and incorporate the 2023 American Society of Clinical Oncology and College of American Pathologist guideline update. The authors also provide guidance on when to seek consensus for HER2 IHC scoring, how to incorporate HER2-low into IHC reporting and present examples of HER2 IHC staining, including challenging cases.

CONCLUSIONS

Awareness of BC cases that are negative for HER protein overexpression/gene amplification and the related clinical relevance for targeted therapy highlight the importance of accurate HER2 IHC scoring for optimal treatment selection.

The Performance and Clinical Applicability of HER2 Digital Image Analysis in Breast Cancer: A Systematic Review

This systematic review aims to address the research gap in the performance of computational algorithms for the digital image analysis of HER2 images in clinical settings. While numerous studies have explored various aspects of these algorithms, there is a lack of comprehensive evaluation regarding their effectiveness in real-world clinical applications. We conducted a search of the Web of Science and PubMed databases for studies published from 31 December 2013 to 30 June 2024, focusing on performance effectiveness and components such as dataset size, diversity and source, ground truth, annotation, and validation methods. The study was registered with PROSPERO (CRD42024525404). Key questions guiding this review include the following: How effective are current computational algorithms at detecting HER2 status in digital images? What are the common validation methods and dataset characteristics used in these studies? Is there standardization of algorithm evaluations of clinical applications that can improve the clinical utility and reliability of computational tools for HER2 detection in digital image analysis? We identified 6833 publications, with 25 meeting the inclusion criteria. The accuracy rate with clinical datasets varied from 84.19% to 97.9%. The highest accuracy was achieved on the publicly available Warwick dataset at 98.8% in synthesized datasets. Only 12% of studies used separate datasets for external validation; 64% of studies used a combination of accuracy, precision, recall, and F1 as a set of performance measures. Despite the high accuracy rates reported in these studies, there is a notable absence of direct evidence supporting their clinical application. To facilitate the integration of these technologies into clinical practice, there is an urgent need to address real-world challenges and overreliance on internal validation. Standardizing study designs on real clinical datasets can enhance the reliability and clinical applicability of computational algorithms in improving the detection of HER2 cancer.

Automated HER2 Scoring in Breast Cancer Images Using Deep Learning and Pyramid Sampling

Objective and Impact Statement: Human epidermal growth factor receptor 2 (HER2) is a critical protein in cancer cell growth that signifies the aggressiveness of breast cancer (BC) and helps predict its prognosis. Here, we introduce a deep learning-based approach utilizing pyramid sampling for the automated classification of HER2 status in immunohistochemically (IHC) stained BC tissue images. Introduction: Accurate assessment of IHC-stained tissue slides for HER2 expression levels is essential for both treatment guidance and understanding of cancer mechanisms. Nevertheless, the traditional workflow of manual examination by board-certified pathologists encounters challenges, including inter- and intra-observer inconsistency and extended turnaround times. Methods: Our deep learning-based method analyzes morphological features at various spatial scales, efficiently managing the computational load and facilitating a detailed examination of cellular and larger-scale tissue-level details. Results: This approach addresses the tissue heterogeneity of HER2 expression by providing a comprehensive view, leading to a blind testing classification accuracy of 84.70%, on a dataset of 523 core images from tissue microarrays. Conclusion: This automated system, proving reliable as an adjunct pathology tool, has the potential to enhance diagnostic precision and evaluation speed, and might substantially impact cancer treatment planning.

Vision Transformers for Breast Cancer Human Epidermal Growth Factor Receptor 2 Expression Staging without Immunohistochemical Staining

Accurate staging of human epidermal growth factor receptor 2 (HER2) expression is vital for evaluating breast cancer treatment efficacy. However, it typically involves costly and complex immunohistochemical staining, along with hematoxylin and eosin staining. This work presents customized vision transformers for staging HER2 expression in breast cancer using only hematoxylin and eosin-stained images. The proposed algorithm comprised three modules: a localization module for weakly localizing critical image features using spatial transformers, an attention module for global learning via vision transformers, and a loss module to determine proximity to a HER2 expression level based on input images by calculating ordinal loss. Results, reported with 95% CIs, reveal the proposed approach's success in HER2 expression staging: area under the receiver operating characteristic curve, 0.9202 ± 0.01; precision, 0.922 ± 0.01; sensitivity, 0.876 ± 0.01; and specificity, 0.959 ± 0.02 over fivefold cross-validation. Comparatively, this approach significantly outperformed conventional vision transformer models and state-of-the-art convolutional neural network models (P < 0.001). Furthermore, it surpassed existing methods when evaluated on an independent test data set. This work holds great importance, aiding HER2 expression staging in breast cancer treatment while circumventing the costly and time-consuming immunohistochemical staining procedure, thereby addressing diagnostic disparities in low-resource settings and low-income countries.

Invasive carcinoma segmentation in whole slide images using MS-ResMTUNet

Identifying the invasive cancer area is a crucial step in the automated diagnosis of digital pathology slices of the breast. When examining the pathological sections of patients with invasive ductal carcinoma, several evaluations are required specifically for the invasive cancer area. However, currently there is little work that can effectively distinguish the invasive cancer area from the ductal carcinoma in situ in whole slide images. To address this issue, we propose a novel architecture named ResMTUnet that combines the strengths of vision transformer and CNN, and uses multi-task learning to achieve accurate invasive carcinoma recognition and segmentation in breast cancer. Furthermore, we introduce a multi-scale input model based on ResMTUnet with conditional random field, named MS-ResMTUNet, to perform segmentation on WSIs. Our systematic experimentation has shown that the proposed network outperforms other competitive methods and effectively segments invasive carcinoma regions in WSIs. This lays a solid foundation for subsequent analysis of breast pathological slides in the future. The code is available at: https://github.com/liuyiqing2018/MS-ResMTUNet.

Emerging Landscape of Targeted Therapy of Breast Cancers With Low Human Epidermal Growth Factor Receptor 2 Protein Expression

CONTEXT.—

Human epidermal growth factor receptor 2 (HER2) status in breast cancer is currently classified as negative or positive for selecting patients for anti-HER2 targeted therapy. The evolution of the HER2 status has included a new HER2-low category defined as an HER2 immunohistochemistry score of 1+ or 2+ without gene amplification. This new category opens the door to a targetable HER2-low breast cancer population for which new treatments may be effective.

OBJECTIVE.—

To review the current literature on the emerging category of breast cancers with low HER2 protein expression, including the clinical, histopathologic, and molecular features, and outline the clinical trials and best practice recommendations for identifying HER2-low-expressing breast cancers by immunohistochemistry.

DATA SOURCES.—

We conducted a literature review based on peer-reviewed original articles, review articles, regulatory communications, ongoing and past clinical trials identified through ClinicalTrials.gov, and the authors' practice experience.

CONCLUSIONS.—

The availability of new targeted therapy potentially effective for patients with breast cancers with low HER2 protein expression requires multidisciplinary recognition. In particular, pathologists need to recognize and identify this category to allow the optimal selection of patients for targeted therapy.

Deep learning-based immunohistochemical estimation of breast cancer via ultrasound image applications

Background

Breast cancer is the key global menace to women's health, which ranks first by mortality rate. The rate reduction and early diagnostics of breast cancer are the mainstream of medical research. Immunohistochemical examination is the most important link in the process of breast cancer treatment, and its results directly affect physicians' decision-making on follow-up medical treatment.

Purpose

This study aims to develop a computer-aided diagnosis (CAD) method based on deep learning to classify breast ultrasound (BUS) images according to immunohistochemical results.

Methods

A new depth learning framework guided by BUS image data analysis was proposed for the classification of breast cancer nodes in BUS images. The proposed CAD classification network mainly comprised three innovation points. First, a multilevel feature distillation network (MFD-Net) based on CNN, which could extract feature layers of different scales, was designed. Then, the image features extracted at different depths were fused to achieve multilevel feature distillation using depth separable convolution and reverse depth separable convolution to increase convolution depths. Finally, a new attention module containing two independent submodules, the channel attention module (CAM) and the spatial attention module (SAM), was introduced to improve the model classification ability in channel and space.

Results

A total of 500 axial BUS images were retrieved from 294 patients who underwent BUS examination, and these images were detected and cropped, resulting in breast cancer node BUS image datasets, which were classified according to immunohistochemical findings, and the datasets were randomly subdivided into a training set (70%) and a test set (30%) in the classification process, with the results of the four immune indices output simultaneously from training and testing, in the model comparison experiment. Taking ER immune indicators as an example, the proposed model achieved a precision of 0.8933, a recall of 0.7563, an F1 score of 0.8191, and an accuracy of 0.8386, significantly outperforming the other models. The results of the designed ablation experiment also showed that the proposed multistage characteristic distillation structure and attention module were key in improving the accuracy rate.

Conclusion

The extensive experiments verify the high efficiency of the proposed method. It is considered the first classification of breast cancer by immunohistochemical results in breast cancer image processing, and it provides an effective aid for postoperative breast cancer treatment, greatly reduces the difficulty of diagnosis for doctors, and improves work efficiency.

HER2GAN: Overcome the Scarcity of HER2 Breast Cancer Dataset Based on Transfer Learning and GAN Model

INTRODUCTION

Immunohistochemistry (IHC) is crucial for breast cancer diagnosis, classification, and individualized treatment. IHC is used to measure the levels of expression of hormone receptors (estrogen and progesterone receptors), human epidermal growth factor receptor 2 (HER2), and other biomarkers, which are used to make treatment decisions and predict how well a patient will do. The evaluation of the breast cancer score on IHC slides, taking into account structural and morphological features as well as a scarcity of relevant data, is one of the most important issues in the IHC debate. Several recent studies have utilized machine learning and deep learning techniques to resolve these issues.

MATERIALS AND METHODS

This paper introduces a new approach for addressing the issue based on supervised deep learning. A GAN-based model is proposed for generating high-quality HER2 images and identifying and classifying HER2 levels. Using transfer learning methodologies, the original and generated images were evaluated.

RESULTS AND CONCLUSION

All of the models have been trained and evaluated using publicly accessible and private data sets, respectively. The InceptionV3 and InceptionResNetV2 models achieved a high accuracy of 93% with the combined generated and original images used for training and testing, demonstrating the exceptional quality of the details in the synthesized images.

Deep Transfer Learning-Based Approach for Glucose Transporter-1 (GLUT1) Expression Assessment

Glucose transporter-1 (GLUT-1) expression level is a biomarker of tumour hypoxia condition in immunohistochemistry (IHC)-stained images. Thus, the GLUT-1 scoring is a routine procedure currently employed for predicting tumour hypoxia markers in clinical practice. However, visual assessment of GLUT-1 scores is subjective and consequently prone to inter-pathologist variability. Therefore, this study proposes an automated method for assessing GLUT-1 scores in IHC colorectal carcinoma images. For this purpose, we leverage deep transfer learning methodologies for evaluating the performance of six different pre-trained convolutional neural network (CNN) architectures: AlexNet, VGG16, GoogleNet, ResNet50, DenseNet-201 and ShuffleNet. The target CNNs are fine-tuned as classifiers or adapted as feature extractors with support vector machine (SVM) to classify GLUT-1 scores in IHC images. Our experimental results show that the winning model is the trained SVM classifier on the extracted deep features fusion Feat-Concat from DenseNet201, ResNet50 and GoogLeNet extractors. It yields the highest prediction accuracy of 98.86%, thus outperforming the other classifiers on our dataset. We also conclude, from comparing the methodologies, that the off-the-shelf feature extraction is better than the fine-tuning model in terms of time and resources required for training.

MGGAN: A multi-generator generative adversarial network for breast cancer immunohistochemical image generation

The immunohistochemical technique (IHC) is widely used for evaluating diagnostic markers, but it can be expensive to obtain IHC-stained section. Translating the cheap and easily available hematoxylin and eosin (HE) images into IHC images provides a solution to this challenge. In this paper, we propose a multi-generator generative adversarial network (MGGAN) that can generate high-quality IHC images based on the HE of breast cancer. Our MGGAN approach combines the low-frequency and high-frequency components of the HE image to improve the translation of breast cancer image details. We use the multi-generator to extract semantic information and a U-shaped architecture and patch-based discriminator to collect and optimize the low-frequency and high-frequency components of an image. We also include a cross-entropy loss as a regularization term in the loss function to ensure consistency between the synthesized image and the real image. Our experimental and visualization results demonstrate that our method outperforms other state-of-the-art image synthesis methods in terms of both quantitative and qualitative analysis. Our approach provides a cost-effective and efficient solution for obtaining high-quality IHC images.

Artificial Intelligence for Image Analysis in Oral Squamous Cell Carcinoma: A Review

Head and neck tumor differential diagnosis and prognosis have always been a challenge for oral pathologists due to their similarities and complexity. Artificial intelligence novel applications can function as an auxiliary tool for the objective interpretation of histomorphological digital slides. In this review, we present digital histopathological image analysis applications in oral squamous cell carcinoma. A literature search was performed in PubMed MEDLINE with the following keywords: "artificial intelligence" OR "deep learning" OR "machine learning" AND "oral squamous cell carcinoma". Artificial intelligence has proven to be a helpful tool in histopathological image analysis of tumors and other lesions, even though it is necessary to continue researching in this area, mainly for clinical validation.

The Role of Artificial Intelligence in Accurate Interpretation of HER2 Immunohistochemical Scores 0 and 1+ in Breast Cancer

The new human epidermal growth factor receptor (HER)2-targeting antibody-drug conjugate offers the opportunity to treat patients with HER2-low breast cancer. Distinguishing HER2 immunohistochemical (IHC) scores of 0 and 1+ is not only critical but also challenging owing to HER2 heterogeneity and variability of observers. In this study, we aimed to increase the interpretation accuracy and consistency of HER2 IHC 0 and 1+ evaluation through assistance from an artificial intelligence (AI) algorithm. In addition, we examined the value of our AI algorithm in evaluating HER2 IHC scores in tumors with heterogeneity. AI-assisted interpretation consisted of AI algorithms and an augmenting reality module with a microscope. Fifteen pathologists (5 junior, 5 midlevel, and 5 senior) participated in this multi-institutional 2-round ring study that included 246 infiltrating duct carcinoma cases that were not otherwise specified. In round 1, pathologists analyzed 246 HER2 IHC slides by microscope without AI assistance. After a 2-week washout period, the pathologists read the same slides with AI algorithm assistance and rendered the definitive results by adjusting to the AI algorithm. The accuracy of interpretation accuracy with AI assistance (0.93 vs 0.80), thereby the evaluation precision of HER2 0 and the recall of HER2 1+. In addition, the AI algorithm improved the total consistency (intraclass correlation coefficient = 0.542-0.812), especially in HER2 1+ cases. In cases with heterogeneity, accuracy improved significantly (0.68 to 0.89) and to a similar level as in cases without heterogeneity (accuracy, 0.97). Both accuracy and consistency improved more for junior pathologists than those for the midlevel and senior pathologists. To the best of our knowledge, this is the first study to show that the accuracy and consistency of HER2 IHC 0 and 1+ evaluation and the accuracy of HER2 IHC evaluation in breast cancers with heterogeneity can be significantly improved using AI-assisted interpretation.

Immunohistochemical HER2 Recognition and Analysis of Breast Cancer Based on Deep Learning

Breast cancer is one of the common malignant tumors in women. It seriously endangers women's life and health. The human epidermal growth factor receptor 2 (HER2) protein is responsible for the division and growth of healthy breast cells. The overexpression of the HER2 protein is generally evaluated by immunohistochemistry (IHC). The IHC evaluation criteria mainly includes three indexes: staining intensity, circumferential membrane staining pattern, and proportion of positive cells. Manually scoring HER2 IHC images is an error-prone, variable, and time-consuming work. To solve these problems, this study proposes an automated predictive method for scoring whole-slide images (WSI) of HER2 slides based on a deep learning network. A total of 95 HER2 pathological slides from September 2021 to December 2021 were included. The average patch level precision and f1 score were 95.77% and 83.09%, respectively. The overall accuracy of automated scoring for slide-level classification was 97.9%. The proposed method showed excellent specificity for all IHC 0 and 3+ slides and most 1+ and 2+ slides. The evaluation effect of the integrated method is better than the effect of using the staining result only.

Deep Learning Based Methods for Breast Cancer Diagnosis: A Systematic Review and Future Direction

Breast cancer is one of the precarious conditions that affect women, and a substantive cure has not yet been discovered for it. With the advent of Artificial intelligence (AI), recently, deep learning techniques have been used effectively in breast cancer detection, facilitating early diagnosis and therefore increasing the chances of patients' survival. Compared to classical machine learning techniques, deep learning requires less human intervention for similar feature extraction. This study presents a systematic literature review on the deep learning-based methods for breast cancer detection that can guide practitioners and researchers in understanding the challenges and new trends in the field. Particularly, different deep learning-based methods for breast cancer detection are investigated, focusing on the genomics and histopathological imaging data. The study specifically adopts the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), which offer a detailed analysis and synthesis of the published articles. Several studies were searched and gathered, and after the eligibility screening and quality evaluation, 98 articles were identified. The results of the review indicated that the Convolutional Neural Network (CNN) is the most accurate and extensively used model for breast cancer detection, and the accuracy metrics are the most popular method used for performance evaluation. Moreover, datasets utilized for breast cancer detection and the evaluation metrics are also studied. Finally, the challenges and future research direction in breast cancer detection based on deep learning models are also investigated to help researchers and practitioners acquire in-depth knowledge of and insight into the area.

Using Whole Slide Gray Value Map to Predict HER2 Expression and FISH Status in Breast Cancer

Accurate detection of HER2 expression through immunohistochemistry (IHC) is of great clinical significance in the treatment of breast cancer. However, manual interpretation of HER2 is challenging, due to the interobserver variability among pathologists. We sought to explore a deep learning method to predict HER2 expression level and gene status based on a Whole Slide Image (WSI) of the HER2 IHC section. When applied to 228 invasive breast carcinoma of no special type (IBC-NST) DAB-stained slides, our GrayMap+ convolutional neural network (CNN) model accurately classified HER2 IHC level with mean accuracy 0.952 ± 0.029 and predicted HER2 FISH status with mean accuracy 0.921 ± 0.029. Our result also demonstrated strong consistency in HER2 expression score between our system and experienced pathologists (intraclass correlation coefficient (ICC) = 0.903, Cohen's κ = 0.875). The discordant cases were found to be largely caused by high intra-tumor staining heterogeneity in the HER2 IHC group and low copy number in the HER2 FISH group.

Deep Learning Approaches in Histopathology

The revolution of artificial intelligence and its impacts on our daily life has led to tremendous interest in the field and its related subtypes: machine learning and deep learning. Scientists and developers have designed machine learning- and deep learning-based algorithms to perform various tasks related to tumor pathologies, such as tumor detection, classification, grading with variant stages, diagnostic forecasting, recognition of pathological attributes, pathogenesis, and genomic mutations. Pathologists are interested in artificial intelligence to improve the diagnosis precision impartiality and to minimize the workload combined with the time consumed, which affects the accuracy of the decision taken. Regrettably, there are already certain obstacles to overcome connected to artificial intelligence deployments, such as the applicability and validation of algorithms and computational technologies, in addition to the ability to train pathologists and doctors to use these machines and their willingness to accept the results. This review paper provides a survey of how machine learning and deep learning methods could be implemented into health care providers' routine tasks and the obstacles and opportunities for artificial intelligence application in tumor morphology.

Using Deep Learning to Predict Final HER2 Status in Invasive Breast Cancers That are Equivocal (2+) by Immunohistochemistry

Invasive breast carcinomas are routinely tested for HER2 using immunohistochemistry (IHC), with reflex in situ hybridization (ISH) for those scored as equivocal (2+). ISH testing is expensive, time-consuming, and not universally available. In this study, we trained a deep learning algorithm to directly predict HER2 gene amplification status from HER2 2+ IHC slides. Data included 115 consecutive cases of invasive breast carcinoma scored as 2+ by IHC that had follow-up HER2 ISH testing. An external validation data set was created from 36 HER2 IHC slides prepared at an outside institution. All internal IHC slides were digitized and divided into training (80%), and test (20%) sets with 5-fold cross-validation. Small patches (256×256 pixels) were randomly extracted and used to train convolutional neural networks with EfficientNet B0 architecture using a transfer learning approach. Predictions for slides in the test set were made on individual patches, and these predictions were aggregated to generate an overall prediction for each slide. This resulted in a receiver operating characteristic area under the curve of 0.83 with an overall accuracy of 79% (sensitivity=0.70, specificity=0.82). Analysis of external validation slides resulted in a receiver operating characteristic area under the curve of 0.79 with an overall accuracy of 81% (sensitivity=0.50, specificity=0.82). Although the sensitivity and specificity are not high enough to negate the need for reflexive ISH testing entirely, this approach may be useful for triaging cases more likely to be HER2 positive and initiating treatment planning in centers where HER2 ISH testing is not readily available.

HEROHE Challenge: Predicting HER2 Status in Breast Cancer from Hematoxylin–Eosin Whole-Slide Imaging

Breast cancer is the most common malignancy in women worldwide, and is responsible for more than half a million deaths each year. The appropriate therapy depends on the evaluation of the expression of various biomarkers, such as the human epidermal growth factor receptor 2 (HER2) transmembrane protein, through specialized techniques, such as immunohistochemistry or in situ hybridization. In this work, we present the HER2 on hematoxylin and eosin (HEROHE) challenge, a parallel event of the 16th European Congress on Digital Pathology, which aimed to predict the HER2 status in breast cancer based only on hematoxylin–eosin-stained tissue samples, thus avoiding specialized techniques. The challenge consisted of a large, annotated, whole-slide images dataset (509), specifically collected for the challenge. Models for predicting HER2 status were presented by 21 teams worldwide. The best-performing models are presented by detailing the network architectures and key parameters. Methods are compared and approaches, core methodologies, and software choices contrasted. Different evaluation metrics are discussed, as well as the performance of the presented models for each of these metrics. Potential differences in ranking that would result from different choices of evaluation metrics highlight the need for careful consideration at the time of their selection, as the results show that some metrics may misrepresent the true potential of a model to solve the problem for which it was developed. The HEROHE dataset remains publicly available to promote advances in the field of computational pathology.

Automatic CD30 scoring method for whole slide images of primary cutaneous CD30+ lymphoproliferative diseases

AIMS

Deep-learning methods for scoring biomarkers are an active research topic. However, the superior performance of many studies relies on large datasets collected from clinical samples. In addition, there are fewer studies on immunohistochemical marker assessment for dermatological diseases. Accordingly, we developed a method for scoring CD30 based on convolutional neural networks for a few primary cutaneous CD30+ lymphoproliferative disorders and used this method to evaluate other biomarkers.

METHODS

A multipatch spatial attention mechanism and conditional random field algorithm were used to fully fuse tumour tissue characteristics on immunohistochemical slides and alleviate the few sample feature deficits. We trained and tested 28 CD30+ immunohistochemical whole slide images (WSIs), evaluated them with a performance index, and compared them with the diagnoses of senior dermatologists. Finally, the model's performance was further demonstrated on the publicly available Yale HER2 cohort.

RESULTS

Compared with the diagnoses by senior dermatologists, this method can better locate the tumour area and reduce the misdiagnosis rate. The prediction of CD3 and Ki-67 validated the model's ability to identify other biomarkers.

CONCLUSIONS

In this study, using a few immunohistochemical WSIs, our model can accurately identify CD30, CD3 and Ki-67 markers. In addition, the model could be applied to additional tumour identification tasks to aid pathologists in diagnosis and benefit clinical evaluation.

Artificial Intelligence-Assisted Score Analysis for Predicting the Expression of the Immunotherapy Biomarker PD-L1 in Lung Cancer

Programmed cell death ligand 1 (PD-L1) is a critical biomarker for predicting the response to immunotherapy. However, traditional quantitative evaluation of PD-L1 expression using immunohistochemistry staining remains challenging for pathologists. Here we developed a deep learning (DL)-based artificial intelligence (AI) model to automatically analyze the immunohistochemical expression of PD-L1 in lung cancer patients. A total of 1,288 patients with lung cancer were included in the study. The diagnostic ability of three different AI models (M1, M2, and M3) was assessed in both PD-L1 (22C3) and PD-L1 (SP263) assays. M2 and M3 showed improved performance in the evaluation of PD-L1 expression in the PD-L1 (22C3) assay, especially at 1% cutoff. Highly accurate performance in the PD-L1 (SP263) was also achieved, with accuracy and specificity of 96.4 and 96.8% in both M2 and M3, respectively. Moreover, the diagnostic results of these three AI-assisted models were highly consistent with those from the pathologist. Similar performances of M1, M2, and M3 in the 22C3 dataset were also obtained in lung adenocarcinoma and lung squamous cell carcinoma in both sampling methods. In conclusion, these results suggest that AI-assisted diagnostic models in PD-L1 expression are a promising tool for improving the efficiency of clinical pathologists.

A Deep Learning Quantification Algorithm for HER2 Scoring of Gastric Cancer

Gastric cancer is the third most common cause of cancer-related death in the world. Human epidermal growth factor receptor 2 (HER2) positive is an important subtype of gastric cancer, which can provide significant diagnostic information for gastric cancer pathologists. However, pathologists usually use a semi-quantitative assessment method to assign HER2 scores for gastric cancer by repeatedly comparing hematoxylin and eosin (H&E) whole slide images (WSIs) with their HER2 immunohistochemical WSIs one by one under the microscope. It is a repetitive, tedious, and highly subjective process. Additionally, WSIs have billions of pixels in an image, which poses computational challenges to Computer-Aided Diagnosis (CAD) systems. This study proposed a deep learning algorithm for HER2 quantification evaluation of gastric cancer. Different from other studies that use convolutional neural networks for extracting feature maps or pre-processing on WSIs, we proposed a novel automatic HER2 scoring framework in this study. In order to accelerate the computational process, we proposed to use the re-parameterization scheme to separate the training model from the deployment model, which significantly speedup the inference process. To the best of our knowledge, this is the first study to provide a deep learning quantification algorithm for HER2 scoring of gastric cancer to assist the pathologist's diagnosis. Experiment results have demonstrated the effectiveness of our proposed method with an accuracy of 0.94 for the HER2 scoring prediction.

A novel ensemble feature selection method for pixel-level segmentation of HER2 overexpression

Classifying histopathology images on a pixel-level requires sets of features able to capture the complex characteristics of the images, like the irregular cell morphology and the color heterogeneity on the tissue aspect. In this context, feature selection becomes a crucial step in the classification process such that it reduces model complexity and computational costs, avoids overfitting, and thereby it improves the model performance. In this study, we propose a new ensemble feature selection method by combining a set of base selectors, classifiers, and rank aggregation methods, aiming to determine from any initial set of handcrafted features, a smaller set of relevant color and texture pixel-level features, subsequently used for segmenting HER2 overexpression on a pixel-level, in breast cancer tissue images. We have been able to significantly reduce the set of initial features, using the proposed ensemble feature selection method. The best results are obtained using $$\chi ^2$$ χ 2 , Random Forest, and Runoff as the based selector, classifier, and aggregation method, respectively. The classification performance of the best model trained on the selected features set results in 0.939 recall, 0.866 specificity, 0.903 accuracy, 0.875 precision, and 0.906 F1-score.

Progress on deep learning in digital pathology of breast cancer: a narrative review

BACKGROUND AND OBJECTIVE

Pathology is the gold standard criteria for breast cancer diagnosis and has important guiding value in formulating the clinical treatment plan and predicting the prognosis. However, traditional microscopic examinations of tissue sections are time consuming and labor intensive, with unavoidable subjective variations. Deep learning (DL) can evaluate and extract the most important information from images with less need for human instruction, providing a promising approach to assist in the pathological diagnosis of breast cancer. To provide an informative and up-to-date summary on the topic of DL-based diagnostic systems for breast cancer pathology image analysis and discuss the advantages and challenges to the routine clinical application of digital pathology.

METHODS

A PubMed search with keywords ("breast neoplasm" or "breast cancer") and ("pathology" or "histopathology") and ("artificial intelligence" or "deep learning") was conducted. Relevant publications in English published from January 2000 to October 2021 were screened manually for their title, abstract, and even full text to determine their true relevance. References from the searched articles and other supplementary articles were also studied.

KEY CONTENT AND FINDINGS

DL-based computerized image analysis has obtained impressive achievements in breast cancer pathology diagnosis, classification, grading, staging, and prognostic prediction, providing powerful methods for faster, more reproducible, and more precise diagnoses. However, all artificial intelligence (AI)-assisted pathology diagnostic models are still in the experimental stage. Improving their economic efficiency and clinical adaptability are still required to be developed as the focus of further researches.

CONCLUSIONS

Having searched PubMed and other databases and summarized the application of DL-based AI models in breast cancer pathology, we conclude that DL is undoubtedly a promising tool for assisting pathologists in routines, but further studies are needed to realize the digitization and automation of clinical pathology.

L’intelligence artificielle pourrait-elle intervenir dans l’aide au diagnostic des cancers du sein ? – L’exemple de HER2

HER2 is an important prognostic and predictive biomarker in breast cancer. Its detection makes it possible to define which patients will benefit from a targeted treatment. While assessment of HER2 status by immunohistochemistry in positive vs negative categories is well implemented and reproducible, the introduction of a new "HER2-low" category could raise some concerns about its scoring and reproducibility. We herein described the current HER2 testing methods and the application of innovative machine learning techniques to improve these determinations, as well as the main challenges and opportunities related to the implementation of digital pathology in the up-and-coming AI era.

Digital Technology in Diagnostic Breast Pathology and Immunohistochemistry

Digital technology has been used in the field of diagnostic breast pathology and immunohistochemistry (IHC) for decades. Examples include automated tissue processing and staining, digital data processing, storing and management, voice recognition systems, and digital technology-based production of antibodies and other IHC reagents. However, the recent application of whole slide imaging technology and artificial intelligence (AI)-based tools has attracted a lot of attention. The use of AI tools in breast pathology is discussed briefly as it is covered in other reviews. Here, we present the main application of digital technology in IHC. This includes automation of IHC staining, using image analysis systems and computer vision technology to interpret IHC staining, and the use of AI-based tools to predict marker expression from haematoxylin and eosin-stained digitalized images.

Development of a tracking error prediction system for the CyberKnife Synchrony Respiratory Tracking System with use of support vector regression

PURPOSE

The accuracy of the CyberKnife Synchrony Respiratory Tracking System is dependent on the breathing pattern of a patient. Therefore, the tracking error in each patient must be determined. Support vector regression (SVR) can be used to easily identify the tracking error in each patient. This study aimed to develop a system with SVR that can predict tracking error according to a patient's respiratory waveform.

METHODS

Datasets of the respiratory waveforms of 93 patients were obtained. The feature variables were variation in respiration amplitude, tumor velocity, and phase shift between tumor and the chest wall, and the target variable was tracking error. A learning model was evaluated with tenfold cross-validation. We documented the difference between the predicted and actual tracking errors and assessed the correlation coefficient and coefficient of determination.

RESULTS

The average difference and maximum difference between the actual and predicted tracking errors were 0.57 ± 0.63 mm and 2.1 mm, respectively. The correlation coefficient and coefficient of determination were 0.86 and 0.74, respectively.

CONCLUSION

We developed a system for obtaining tracking error by using SVR. The accuracy of such a system is clinically useful. Moreover, the system can easily evaluate tracking error. We developed a system that can be used to predict the tracking error of SRTS in the CyberKnife Robotic Radiosurgery System using machine learning. The feature variables were the breathing parameters, and the target variable was the tracking error. We used support vector regression algorithm.

HER2 Molecular Marker Scoring Using Transfer Learning and Decision Level Fusion

In prognostic evaluation of breast cancer, immunohistochemical (IHC) marker human epidermal growth factor receptor 2 (HER2) is used for prognostic evaluation. Accurate assessment of HER2-stained tissue sample is essential in therapeutic decision making for the patients. In regular clinical settings, expert pathologists assess the HER2-stained tissue slide under microscope for manual scoring based on prior experience. Manual scoring is time consuming, tedious, and often prone to inter-observer variation among group of pathologists. With the recent advancement in the area of computer vision and deep learning, medical image analysis has got significant attention. A number of deep learning architectures have been proposed for classification of different image groups. These networks are also used for transfer learning to classify other image classes. In the presented study, a number of transfer learning architectures are used for HER2 scoring. Five pre-trained architectures viz. VGG16, VGG19, ResNet50, MobileNetV2, and NASNetMobile with decimating the fully connected layers to get 3-class classification have been used for the comparative assessment of the networks as well as further scoring of stained tissue sample image based on statistical voting using mode operator. HER2 Challenge dataset from Warwick University is used in this study. A total of 2130 image patches were extracted to generate the training dataset from 300 training images corresponding to 30 training cases. The output model is then tested on 800 new test image patches from 100 test images acquired from 10 test cases (different from training cases) to report the outcome results. The transfer learning models have shown significant accuracy with VGG19 showing the best accuracy for the test images. The accuracy is found to be 93%, which increases to 98% on the image-based scoring using statistical voting mechanism. The output shows a capable quantification pipeline in automated HER2 score generation.

High throughput assessment of biomarkers in tissue microarrays using artificial intelligence: PTEN loss as a proof-of-principle in multi-center prostate cancer cohorts

Phosphatase and tensin homolog (PTEN) loss is associated with adverse outcomes in prostate cancer and has clinical potential as a prognostic biomarker. The objective of this work was to develop an artificial intelligence (AI) system for automated detection and localization of PTEN loss on immunohistochemically (IHC) stained sections. PTEN loss was assessed using IHC in two prostate tissue microarrays (TMA) (internal cohort, n = 272 and external cohort, n = 129 patients). TMA cores were visually scored for PTEN loss by pathologists and, if present, spatially annotated. Cores from each patient within the internal TMA cohort were split into 90% cross-validation (N = 2048) and 10% hold-out testing (N = 224) sets. ResNet-101 architecture was used to train core-based classification using a multi-resolution ensemble approach (×5, ×10, and ×20). For spatial annotations, single resolution pixel-based classification was trained from patches extracted at ×20 resolution, interpolated to ×40 resolution, and applied in a sliding-window fashion. A final AI-based prediction model was created from combining multi-resolution and pixel-based models. Performance was evaluated in 428 cores of external cohort. From both cohorts, a total of 2700 cores were studied, with a frequency of PTEN loss of 14.5% in internal (180/1239) and external 13.5% (43/319) cancer cores. The final AI-based prediction of PTEN status demonstrated 98.1% accuracy (95.0% sensitivity, 98.4% specificity; median dice score = 0.811) in internal cohort cross-validation set and 99.1% accuracy (100% sensitivity, 99.0% specificity; median dice score = 0.804) in internal cohort test set. Overall core-based classification in the external cohort was significantly improved in the external cohort (area under the curve = 0.964, 90.6% sensitivity, 95.7% specificity) when further trained (fine-tuned) using 15% of cohort data (19/124 patients). These results demonstrate a robust and fully automated method for detection and localization of PTEN loss in prostate cancer tissue samples. AI-based algorithms have potential to streamline sample assessment in research and clinical laboratories.

Artificial intelligence in dermatopathology: Diagnosis, education, and research

Artificial intelligence (AI) utilizes computer algorithms to carry out tasks with human-like intelligence. Convolutional neural networks, a type of deep learning AI, can classify basal cell carcinoma, seborrheic keratosis, and conventional nevi, highlighting the potential for deep learning algorithms to improve diagnostic workflow in dermatopathology of highly routine diagnoses. Additionally, convolutional neural networks can support the diagnosis of melanoma and may help predict disease outcomes. Capabilities of machine learning in dermatopathology can extend beyond clinical diagnosis to education and research. Intelligent tutoring systems can teach visual diagnoses in inflammatory dermatoses, with measurable cognitive effects on learners. Natural language interfaces can instruct dermatopathology trainees to produce diagnostic reports that capture relevant detail for diagnosis in compliance with guidelines. Furthermore, deep learning can power computation- and population-based research. However, there are many limitations of deep learning that need to be addressed before broad incorporation into clinical practice. The current potential of AI in dermatopathology is to supplement diagnosis, and dermatopathologist guidance is essential for the development of useful deep learning algorithms. Herein, the recent progress of AI in dermatopathology is reviewed with emphasis on how deep learning can influence diagnosis, education, and research.

Multimodal medical image fusion via laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy

BACKGROUND

In recent years, numerous fusion algorithms have been proposed for multimodal medical images. The Laplacian pyramid is one type of multiscale fusion method. Although the pyramid-based fusion algorithm can fuse images well, it has the disadvantages of edge degradation, detail loss and image smoothing as the number of decomposition layers increase, which is harmful for medical diagnosis and analysis.

METHOD

This paper proposes a medical image fusion algorithm based on the Laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy, which can greatly improve the edge quality. First, multimodal medical images are reconstructed through convolutional neural network. Then, the Laplacian pyramid is applied in the decomposition and fusion process. The optimal number of decomposition layers is determined by experiments. In addition, a local gradient energy fusion strategy is utilized to fuse the coefficients in each layer. Finally, the fused image is output through Laplacian inverse transformation.

RESULTS

Compared with existing algorithms, our fusion results represent better vision quality performance. Furthermore, our algorithm is considerably superior to the compared algorithms in objective indicators. In addition, in our fusion results of Alzheimer and Glioma, the disease details are much clearer than those of compared algorithms, which can provide a reliable basis for doctors to analyze disease and make pathological diagnoses.

Recursive Additive Complement Networks for Cell Membrane Segmentation in Histological Images

A recursive additive complement network (RacNet) is introduced to segment cell membranes in histological images as closed lines. Segmenting cell membranes as closed lines is necessary to calculate cell areas and to estimate N/C ratio, which is useful to diagnose early hepatocellular carcinoma. The RacNet is composed of a complement network and an element-wise maximization (EWM) process and is recursively applied to the network output. The complement network complements the lacking parts of cell membranes. The network, however, has a tendency to mistakenly delete some parts of the segmented cell membranes. The EWM process eliminates this unwanted effect.Experiments carried out using unstained hepatic sections showed that the accuracy for segmenting cell membranes as closed lines was significantly improved by using the RacNet.Three imaging methods, bright-field, dark-field, and phase-contrast, were used, as unstained sections show very low contrast in the bright-field imaging commonly used in pathological diagnosis. These imaging methods are available in optical microscopes used by pathologists. Among the three methods, phase-contrast imaging showed the highest accuracy.

AutoIHC-Analyzer: computer-assisted microscopy for automated membrane extraction/scoring in HER2 molecular markers

Human epidermal growth factor receptor 2 (HER2) is one of the widely used Immunohistochemical (IHC) markers for prognostic evaluation amongst the patient of breast cancer. Accurate quantification of cell membrane is essential for HER2 scoring in therapeutic decision making. In modern laboratory practice, expert pathologist visually assesses the HER2-stained tissue sample under the bright field microscope for cell membrane assessment. This manual assessment is time consuming, tedious and quite often results in interobserver variability. Further, the burden of increasing number of patients is a challenge for the pathologists. To address these challenges, there is an urgent need with a rapid HER2 cell membrane extraction method. The proposed study aims at developing an automated IHC scoring system, termed as AutoIHC-Analyzer, for automated cell membrane extraction followed by HER2 molecular expression assessment from stained tissue images. A series of image processing approaches have been used to automatically extract the stained cells and membrane region, followed by automatic assessment of complete and broken membrane. Finally, a set of features are used to automatically classify the tissue under observation for the quantitative scoring as 0/1+, 2+ and 3+. In a set of surgically extracted cases of HER2-stained tissues, obtained from collaborative hospital for the testing and validation of the proposed approach AutoIHC-Analyzer and publicly available open source ImmunoMembrane software are compared for 90 set of randomly acquired images with the scores by expert pathologist where significant correlation is observed [(r = 0.9448; p < 0.001) and (r = 0.8521; p < 0.001)] respectively. The output shows promising quantification in automated scoring. LAY DESCRIPTION: In cancer prognosis amongst the patient of breast cancer, human epidermal growth factor receptor 2 (HER2) is used as Immunohistochemical (IHC) biomarker. The correct assessment of HER2 leads to the therapeutic decision making. In regular practice, the stained tissue sample is observed under a bright microscope and the expert pathologists score the sample as negative (0/1+), equivocal (2+) and positive (3+) case. The scoring is based on the standard guidelines relating the complete and broken cell membrane as well as intensity of staining in the membrane boundary. Such evaluation is time consuming, tedious and quite often results in interobserver variability. To assist in rapid HER2 cell membrane assessment, the proposed study aims at developing an automated IHC scoring system, termed as AutoIHC-Analyzer, for automated cell membrane extraction followed by HER2 molecular expression assessment from stained tissue images. The input image is preprocessed using modified white patch and CMYK and RGB colour space were used in extracting the haematoxylin (negatively stained cells) and diaminobenzidine (DAB) stain observed in the tumour cell membrane. Segmentation and postprocessing are applied to create the masks for each of the stain channels. The membrane mask is then quantified as complete or broken using skeletonisation and morphological operations. Six set of features were assessed for the classification from a set of 180 training images. These features are: complete to broken membrane ratio, amount of stain using area of Blue and Saturation channels to the image size, DAB to haematoxylin ratio from segmented masks and average R, G and B from five largest blobs in segmented DAB-masked image. These features are then used in training the SVM classifier with Gaussian kernel using 5-fold cross-validation. The accuracy in the training sample is found to be 88.3%. The model is then used for 90 set of unknown test sample images and the final labelling of stained cells and HER2 scores (as 0/1+, 2+ and 3+) are compared with the ground truth, that is expert pathologists' score from the collaborative hospital. The test sample images were also fed to ImmunoMembrane software for a comparative assessment. The results from the proposed AutoIHC-Analyzer and ImmunoMembrane software were compared with the expert pathologists' score where significant agreement using Pearson's correlation coefficient [(r = 0.9448; p < 0.001) and (r = 0.8521; p < 0.001) respectively] is observed. The results from AutoIHC-Analyzer show promising quantitative assessment of HER2 scoring.

Emerging role of deep learning-based artificial intelligence in tumor pathology

The development of digital pathology and progression of state-of-the-art algorithms for computer vision have led to increasing interest in the use of artificial intelligence (AI), especially deep learning (DL)-based AI, in tumor pathology. The DL-based algorithms have been developed to conduct all kinds of work involved in tumor pathology, including tumor diagnosis, subtyping, grading, staging, and prognostic prediction, as well as the identification of pathological features, biomarkers and genetic changes. The applications of AI in pathology not only contribute to improve diagnostic accuracy and objectivity but also reduce the workload of pathologists and subsequently enable them to spend additional time on high-level decision-making tasks. In addition, AI is useful for pathologists to meet the requirements of precision oncology. However, there are still some challenges relating to the implementation of AI, including the issues of algorithm validation and interpretability, computing systems, the unbelieving attitude of pathologists, clinicians and patients, as well as regulators and reimbursements. Herein, we present an overview on how AI-based approaches could be integrated into the workflow of pathologists and discuss the challenges and perspectives of the implementation of AI in tumor pathology.