1
|
Budginaite E, Magee DR, Kloft M, Woodruff HC, Grabsch HI. Computational methods for metastasis detection in lymph nodes and characterization of the metastasis-free lymph node microarchitecture: A systematic-narrative hybrid review. J Pathol Inform 2024; 15:100367. [PMID: 38455864 PMCID: PMC10918266 DOI: 10.1016/j.jpi.2024.100367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 03/09/2024] Open
Abstract
Background Histological examination of tumor draining lymph nodes (LNs) plays a vital role in cancer staging and prognostication. However, as soon as a LN is classed as metastasis-free, no further investigation will be performed and thus, potentially clinically relevant information detectable in tumor-free LNs is currently not captured. Objective To systematically study and critically assess methods for the analysis of digitized histological LN images described in published research. Methods A systematic search was conducted in several public databases up to December 2023 using relevant search terms. Studies using brightfield light microscopy images of hematoxylin and eosin or immunohistochemically stained LN tissue sections aiming to detect and/or segment LNs, their compartments or metastatic tumor using artificial intelligence (AI) were included. Dataset, AI methodology, cancer type, and study objective were compared between articles. Results A total of 7201 articles were collected and 73 articles remained for detailed analyses after article screening. Of the remaining articles, 86% aimed at LN metastasis identification, 8% aimed at LN compartment segmentation, and remaining focused on LN contouring. Furthermore, 78% of articles used patch classification and 22% used pixel segmentation models for analyses. Five out of six studies (83%) of metastasis-free LNs were performed on publicly unavailable datasets, making quantitative article comparison impossible. Conclusions Multi-scale models mimicking multiple microscopy zooms show promise for computational LN analysis. Large-scale datasets are needed to establish the clinical relevance of analyzing metastasis-free LN in detail. Further research is needed to identify clinically interpretable metrics for LN compartment characterization.
Collapse
Affiliation(s)
- Elzbieta Budginaite
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | | | - Maximilian Kloft
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Department of Internal Medicine, Justus-Liebig-University, Giessen, Germany
| | - Henry C. Woodruff
- Department of Precision Medicine, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Heike I. Grabsch
- Department of Pathology, GROW - Research Institute for Oncology and Reproduction, Maastricht University Medical Center+, Maastricht, The Netherlands
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds, UK
| |
Collapse
|
2
|
Wu S, Wang Y, Hong G, Luo Y, Lin Z, Shen R, Zeng H, Xu A, Wu P, Xiao M, Li X, Rao P, Yang Q, Feng Z, He Q, Jiang F, Xie Y, Liao C, Huang X, Chen R, Lin T. An artificial intelligence model for detecting pathological lymph node metastasis in prostate cancer using whole slide images: a retrospective, multicentre, diagnostic study. EClinicalMedicine 2024; 71:102580. [PMID: 38618206 PMCID: PMC11015342 DOI: 10.1016/j.eclinm.2024.102580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Revised: 03/17/2024] [Accepted: 03/18/2024] [Indexed: 04/16/2024] Open
Abstract
Background The pathological examination of lymph node metastasis (LNM) is crucial for treating prostate cancer (PCa). However, the limitations with naked-eye detection and pathologist workload contribute to a high missed-diagnosis rate for nodal micrometastasis. We aimed to develop an artificial intelligence (AI)-based, time-efficient, and high-precision PCa LNM detector (ProCaLNMD) and evaluate its clinical application value. Methods In this multicentre, retrospective, diagnostic study, consecutive patients with PCa who underwent radical prostatectomy and pelvic lymph node dissection at five centres between Sep 2, 2013 and Apr 28, 2023 were included, and histopathological slides of resected lymph nodes were collected and digitised as whole-slide images for model development and validation. ProCaLNMD was trained at a dataset from a single centre (the Sun Yat-sen Memorial Hospital of Sun Yat-sen University [SYSMH]), and externally validated in the other four centres. A bladder cancer dataset from SYSMH was used to further validate ProCaLNMD, and an additional validation (human-AI comparison and collaboration study) containing consecutive patients with PCa from SYSMH was implemented to evaluate the application value of integrating ProCaLNMD into the clinical workflow. The primary endpoint was the area under the receiver operating characteristic curve (AUROC) of ProCaLNMD. In addition, the performance measures for pathologists with ProCaLNMD assistance was also assessed. Findings In total, 8225 slides from 1297 patients with PCa were collected and digitised. Overall, 8158 slides (18,761 lymph nodes) from 1297 patients with PCa (median age 68 years [interquartile range 64-73]; 331 [26%] with LNM) were used to train and validate ProCaLNMD. The AUROC of ProCaLNMD ranged from 0.975 (95% confidence interval 0.953-0.998) to 0.992 (0.982-1.000) in the training and validation datasets, with sensitivities > 0.955 and specificities > 0.921. ProCaLNMD also demonstrated an AUROC of 0.979 in the cross-cancer dataset. ProCaLNMD use triggered true reclassification in 43 (4.3%) slides in which micrometastatic tumour regions were initially missed by pathologists, thereby correcting 28 (8.5%) missed-diagnosed cases of previous routine pathological reports. In the human-AI comparison and collaboration study, the sensitivity of ProCaLNMD (0.983 [0.908-1.000]) surpassed that of two junior pathologists (0.862 [0.746-0.939], P = 0.023; 0.879 [0.767-0.950], P = 0.041) by 10-12% and showed no difference to that of two senior pathologists (both 0.983 [0.908-1.000], both P > 0.99). Furthermore, ProCaLNMD significantly boosted the diagnostic sensitivity of two junior pathologists (both P = 0.041) to the level of senior pathologists (both P > 0.99), and substantially reduced the four pathologists' slide reviewing time (-31%, P < 0.0001; -34%, P < 0.0001; -29%, P < 0.0001; and -27%, P = 0.00031). Interpretation ProCaLNMD demonstrated high diagnostic capabilities for identifying LNM in prostate cancer, reducing the likelihood of missed diagnoses by pathologists and decreasing the slide reviewing time, highlighting its potential for clinical application. Funding National Natural Science Foundation of China, the Science and Technology Planning Project of Guangdong Province, the National Key Research and Development Programme of China, the Guangdong Provincial Clinical Research Centre for Urological Diseases, and the Science and Technology Projects in Guangzhou.
Collapse
Affiliation(s)
- Shaoxu Wu
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
- Guangdong Provincial Key Laboratory of Malignant Tumour Epigenetics and Gene Regulation, Guangdong-Hong Kong Joint Laboratory for RNA Medicine, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
- Guangdong Provincial Clinical Research Centre for Urological Diseases, Guangzhou, China
| | - Yun Wang
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Guibin Hong
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yun Luo
- Department of Urology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Zhen Lin
- CellsVision Medical Technology Services Co., Ltd., Guangzhou, China
| | - Runnan Shen
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Hong Zeng
- Department of Pathology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Abai Xu
- Department of Urology, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Peng Wu
- Department of Urology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Mingzhao Xiao
- Department of Urology, First Affiliated Hospital of Chongqing Medical University, Chongqing Medical University, Chongqing, China
| | - Xiaoyang Li
- Department of Urology, The Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Peng Rao
- Department of Urology, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Qishen Yang
- Department of Urology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Zhengyuan Feng
- Department of Urology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Quanhao He
- Department of Urology, First Affiliated Hospital of Chongqing Medical University, Chongqing Medical University, Chongqing, China
| | - Fan Jiang
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Ye Xie
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Chengxiao Liao
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Xiaowei Huang
- CellsVision Medical Technology Services Co., Ltd., Guangzhou, China
| | - Rui Chen
- CellsVision Medical Technology Services Co., Ltd., Guangzhou, China
| | - Tianxin Lin
- Department of Urology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
- Guangdong Provincial Key Laboratory of Malignant Tumour Epigenetics and Gene Regulation, Guangdong-Hong Kong Joint Laboratory for RNA Medicine, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
- Guangdong Provincial Clinical Research Centre for Urological Diseases, Guangzhou, China
| |
Collapse
|
3
|
van Diest PJ, Flach RN, van Dooijeweert C, Makineli S, Breimer GE, Stathonikos N, Pham P, Nguyen TQ, Veta M. Pros and cons of artificial intelligence implementation in diagnostic pathology. Histopathology 2024; 84:924-934. [PMID: 38433288 DOI: 10.1111/his.15153] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 12/29/2023] [Accepted: 01/19/2024] [Indexed: 03/05/2024]
Abstract
The rapid introduction of digital pathology has greatly facilitated development of artificial intelligence (AI) models in pathology that have shown great promise in assisting morphological diagnostics and quantitation of therapeutic targets. We are now at a tipping point where companies have started to bring algorithms to the market, and questions arise whether the pathology community is ready to implement AI in routine workflow. However, concerns also arise about the use of AI in pathology. This article reviews the pros and cons of introducing AI in diagnostic pathology.
Collapse
Affiliation(s)
- Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Rachel N Flach
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
| | | | - Seher Makineli
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Surgical Oncology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Gerben E Breimer
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Nikolas Stathonikos
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Paul Pham
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Tri Q Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Mitko Veta
- Department of Oncological Urology, University Medical Center Utrecht, Utrecht, the Netherlands
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| |
Collapse
|
4
|
Giammanco A, Bychkov A, Schallenberg S, Tsvetkov T, Fukuoka J, Pryalukhin A, Mairinger F, Seper A, Hulla W, Klein S, Quaas A, Büttner R, Tolkach Y. Fast-track development and multi-institutional clinical validation of an artificial intelligence algorithm for detection of lymph node metastasis in colorectal cancer. Mod Pathol 2024:100496. [PMID: 38636778 DOI: 10.1016/j.modpat.2024.100496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Revised: 03/24/2024] [Accepted: 04/10/2024] [Indexed: 04/20/2024]
Abstract
Lymph node metastasis (LNM) detection can be automated using artificial intelligence-based diagnostic tools. Only limited studies have addressed this task for colorectal cancer. The aim of this study was to develop of a clinical-grade digital pathology tool for LNM detection in colorectal cancer (CRC) using the original fast-track framework. The training cohort included 432 slides from one department. A segmentation algorithm detecting 8 relevant tissue classes was trained. The test cohorts consisted of materials from five pathology departments digitized by four different scanning systems. A high-quality, large training dataset was generated within 7 days, and a minimal amount of annotation work using fast-track principles. The AI tool showed very high accuracy for LNM detection in all cohorts, with sensitivity, negative predictive value, and specificity ranges of 0.980-1.000, 0.997-1.000, and 0.913-0.990, correspondingly. Only 5 of 14460 analyzed test slides with tumor cells over all cohorts were classified as false negative (3/5 representing clusters of tumor cells in lymphatic vessels). A clinical-grade tool was trained in a short time using fast-track development principles and validated using the largest international, multi-institutional, multi-scanner cohort of cases to date, showing very high precision for LNM detection in CRC. We are releasing a part of the test datasets to facilitate academic research.
Collapse
Affiliation(s)
- Avri Giammanco
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki, Japan
| | | | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Junya Fukuoka
- Department of Pathology, Kameda Medical Center, Kamogawa, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki, Japan
| | - Alexey Pryalukhin
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria
| | - Fabian Mairinger
- Institute of Pathology, University Hospital Essen, Essen, Germany
| | - Alexander Seper
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria; Danube Private University, Wien, Austria
| | - Wolfgang Hulla
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|
5
|
Kreager BC, Wu H, Chang WY, Moon S, Mitchell J, Peng C, Huang CC, Muller M, Tian J, Jiang X. High-Performance PMN-PT Single-Crystal-Based 1-3 Composite Transducer Integrated with a Biopsy Needle. Biosensors (Basel) 2024; 14:74. [PMID: 38391993 PMCID: PMC10887013 DOI: 10.3390/bios14020074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 01/25/2024] [Accepted: 01/29/2024] [Indexed: 02/24/2024]
Abstract
To address the need for high-resolution imaging in lung nodule detection and overcome the limitations of the shallow imaging depth associated with high-frequency ultrasound and the complex structure of lung tissue, we successfully integrated 50 MHz ultrasound transducers with 18-gauge biopsy needles. Featuring a miniaturized size of 0.6 × 0.5 × 0.5 mm3, the 50 MHz micromachined 1-3 composite transducer was tested to perform mechanical scanning of a nodule within a lung-tissue-mimicking phantom in vitro. The high-frequency transducer demonstrated the ability to achieve imaging with an axial resolution of 30 μm for measuring nodule edges. Moreover, the integrated biopsy needle prototype exhibited high accuracy (1.74% discrepancy) in estimating nodule area compared to actual dimensions in vitro. These results underscore the promising potential of biopsy-needle-integrated transducers in enhancing the accuracy of endoscopic ultrasound-guided fine needle aspiration biopsy (EUS-FNA) for clinical applications.
Collapse
Affiliation(s)
- Benjamin C. Kreager
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| | - Huaiyu Wu
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| | - Wei-Yi Chang
- CTS Advanced Materials, 4925 Indiana Ave, Lisle, IL 604532, USA; (W.-Y.C.); (J.T.)
| | - Sunho Moon
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| | - Josh Mitchell
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| | - Chang Peng
- School of Biomedical Engineering, ShanghaiTech University, Shanghai 201210, China;
| | - Chih-Chung Huang
- Department of Biomedical Engineering, National Cheng Kung University, Tainan 701, Taiwan;
| | - Marie Muller
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| | - Jian Tian
- CTS Advanced Materials, 4925 Indiana Ave, Lisle, IL 604532, USA; (W.-Y.C.); (J.T.)
| | - Xiaoning Jiang
- Department of Mechanical and Aerospace Engineering, North Carolina State University, Raleigh, NC 27695, USA; (B.C.K.); (H.W.); (S.M.); (J.M.); (M.M.)
| |
Collapse
|
6
|
Tavolara TE, Su Z, Gurcan MN, Niazi MKK. One label is all you need: Interpretable AI-enhanced histopathology for oncology. Semin Cancer Biol 2023; 97:70-85. [PMID: 37832751 DOI: 10.1016/j.semcancer.2023.09.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Revised: 09/06/2023] [Accepted: 09/25/2023] [Indexed: 10/15/2023]
Abstract
Artificial Intelligence (AI)-enhanced histopathology presents unprecedented opportunities to benefit oncology through interpretable methods that require only one overall label per hematoxylin and eosin (H&E) slide with no tissue-level annotations. We present a structured review of these methods organized by their degree of verifiability and by commonly recurring application areas in oncological characterization. First, we discuss morphological markers (tumor presence/absence, metastases, subtypes, grades) in which AI-identified regions of interest (ROIs) within whole slide images (WSIs) verifiably overlap with pathologist-identified ROIs. Second, we discuss molecular markers (gene expression, molecular subtyping) that are not verified via H&E but rather based on overlap with positive regions on adjacent tissue. Third, we discuss genetic markers (mutations, mutational burden, microsatellite instability, chromosomal instability) that current technologies cannot verify if AI methods spatially resolve specific genetic alterations. Fourth, we discuss the direct prediction of survival to which AI-identified histopathological features quantitatively correlate but are nonetheless not mechanistically verifiable. Finally, we discuss in detail several opportunities and challenges for these one-label-per-slide methods within oncology. Opportunities include reducing the cost of research and clinical care, reducing the workload of clinicians, personalized medicine, and unlocking the full potential of histopathology through new imaging-based biomarkers. Current challenges include explainability and interpretability, validation via adjacent tissue sections, reproducibility, data availability, computational needs, data requirements, domain adaptability, external validation, dataset imbalances, and finally commercialization and clinical potential. Ultimately, the relative ease and minimum upfront cost with which relevant data can be collected in addition to the plethora of available AI methods for outcome-driven analysis will surmount these current limitations and achieve the innumerable opportunities associated with AI-driven histopathology for the benefit of oncology.
Collapse
Affiliation(s)
- Thomas E Tavolara
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Ziyu Su
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - Metin N Gurcan
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA
| | - M Khalid Khan Niazi
- Center for Artificial Intelligence Research, Wake Forest University School of Medicine, Winston-Salem, NC, USA.
| |
Collapse
|
7
|
Liu Q, Jiang N, Hao Y, Hao C, Wang W, Bian T, Wang X, Li H, zhang Y, Kang Y, Xie F, Li Y, Jiang X, Feng Y, Mao Z, Wang Q, Gao Q, Zhang W, Cui B, Dong T. Identification of lymph node metastasis in pre-operation cervical cancer patients by weakly supervised deep learning from histopathological whole-slide biopsy images. Cancer Med 2023; 12:17952-17966. [PMID: 37559500 PMCID: PMC10523985 DOI: 10.1002/cam4.6437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Revised: 07/28/2023] [Accepted: 07/31/2023] [Indexed: 08/11/2023] Open
Abstract
BACKGROUND Lymph node metastasis (LNM) significantly impacts the prognosis of individuals diagnosed with cervical cancer, as it is closely linked to disease recurrence and mortality, thereby impacting therapeutic schedule choices for patients. However, accurately predicting LNM prior to treatment remains challenging. Consequently, this study seeks to utilize digital pathological features extracted from histopathological slides of primary cervical cancer patients to preoperatively predict the presence of LNM. METHODS A deep learning (DL) model was trained using the Vision transformer (ViT) and recurrent neural network (RNN) frameworks to predict LNM. This prediction was based on the analysis of 554 histopathological whole-slide images (WSIs) obtained from Qilu Hospital of Shandong University. To validate the model's performance, an external test was conducted using 336 WSIs from four other hospitals. Additionally, the efficiency of the DL model was evaluated using 190 cervical biopsies WSIs in a prospective set. RESULTS In the internal test set, our DL model achieved an area under the curve (AUC) of 0.919, with sensitivity and specificity values of 0.923 and 0.905, respectively, and an accuracy (ACC) of 0.909. The performance of the DL model remained strong in the external test set. In the prospective cohort, the AUC was 0.91, and the ACC was 0.895. Additionally, the DL model exhibited higher accuracy compared to imaging examination in the evaluation of LNM. By utilizing the transformer visualization method, we generated a heatmap that illustrates the local pathological features in primary lesions relevant to LNM. CONCLUSION DL-based image analysis has demonstrated efficiency in predicting LNM in early operable cervical cancer through the utilization of biopsies WSI. This approach has the potential to enhance therapeutic decision-making for patients diagnosed with cervical cancer.
Collapse
Affiliation(s)
- Qingqing Liu
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Nan Jiang
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Yiping Hao
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Chunyan Hao
- Department of Pathology, School of Basic Medical Science, Cheeloo College of MedicineShandong UniversityJinan CityChina
- Department of PathologyQilu Hospital of Shandong UniversityJinan CityChina
| | - Wei Wang
- Department of PathologyAffiliated Hospital of Jining Medical UniversityJining CityChina
| | - Tingting Bian
- Department of Medical ImagingAffiliated Hospital of Jining Medical UniversityJining CityChina
| | - Xiaohong Wang
- Department of Obstetrics and GynecologyJinan People's HospitalJinan CityChina
| | - Hua Li
- Department of Obstetrics and GynecologyTai'an City Central HospitalTai'an CityChina
| | - Yan zhang
- Department of Obstetrics and GynecologyWeifang People's HospitalWeifang CityChina
| | - Yanjun Kang
- Department of Obstetrics and GynecologyWomen and Children's Hospital, Qingdao UniversityQingdao CityChina
| | - Fengxiang Xie
- Department of PathologyKingMed DiagnosticsJinan CityChina
| | - Yawen Li
- Department of PathologyQilu Hospital of Shandong UniversityJinan CityChina
| | - XuJi Jiang
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Yuan Feng
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Zhonghao Mao
- Cheeloo College of MedicineShandong UniversityJinan CityChina
| | - Qi Wang
- Department of Obstetrics and Gynecology, Shandong Provincial Qianfoshan HospitalShandong UniversityJinan CityChina
| | - Qun Gao
- Department of Obstetrics and GynecologyThe Affiliated Hospital of Qingdao UniversityQingdao CityChina
| | - Wenjing Zhang
- Department of Obstetrics and GynecologyQilu Hospital of Shandong UniversityJinan CityChina
| | - Baoxia Cui
- Department of Obstetrics and GynecologyQilu Hospital of Shandong UniversityJinan CityChina
| | - Taotao Dong
- Department of Obstetrics and GynecologyQilu Hospital of Shandong UniversityJinan CityChina
| |
Collapse
|
8
|
Zidane M, Makky A, Bruhns M, Rochwarger A, Babaei S, Claassen M, Schürch CM. A review on deep learning applications in highly multiplexed tissue imaging data analysis. Front Bioinform 2023; 3:1159381. [PMID: 37564726 PMCID: PMC10410935 DOI: 10.3389/fbinf.2023.1159381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2023] [Accepted: 07/12/2023] [Indexed: 08/12/2023] Open
Abstract
Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of DL-based pipelines used in preprocessing highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients. Since its introduction into the field of oncology, deep learning (DL) has impacted clinical discoveries and biomarker predictions. DL-driven discoveries and predictions in oncology are based on a variety of biological data such as genomics, proteomics, and imaging data. DL-based computational frameworks can predict genetic variant effects on gene expression, as well as protein structures based on amino acid sequences. Furthermore, DL algorithms can capture valuable mechanistic biological information from several spatial "omics" technologies, such as spatial transcriptomics and spatial proteomics. Here, we review the impact that the combination of artificial intelligence (AI) with spatial omics technologies has had on oncology, focusing on DL and its applications in biomedical image analysis, encompassing cell segmentation, cell phenotype identification, cancer prognostication, and therapy prediction. We highlight the advantages of using highly multiplexed images (spatial proteomics data) compared to single-stained, conventional histopathological ("simple") images, as the former can provide deep mechanistic insights that cannot be obtained by the latter, even with the aid of explainable AI. Furthermore, we provide the reader with the advantages/disadvantages of the DL-based pipelines used in preprocessing the highly multiplexed images (cell segmentation, cell type annotation). Therefore, this review also guides the reader to choose the DL-based pipeline that best fits their data. In conclusion, DL continues to be established as an essential tool in discovering novel biological mechanisms when combined with technologies such as highly multiplexed tissue imaging data. In balance with conventional medical data, its role in clinical routine will become more important, supporting diagnosis and prognosis in oncology, enhancing clinical decision-making, and improving the quality of care for patients.
Collapse
Affiliation(s)
- Mohammed Zidane
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Ahmad Makky
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Matthias Bruhns
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
- Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Alexander Rochwarger
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Sepideh Babaei
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
| | - Manfred Claassen
- Department of Internal Medicine I, University Hospital Tübingen, Tübingen, Germany
- Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Christian M. Schürch
- Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| |
Collapse
|
9
|
Olatunji I, Cui F. Multimodal AI for prediction of distant metastasis in carcinoma patients. Front Bioinform 2023; 3:1131021. [PMID: 37228671 PMCID: PMC10203594 DOI: 10.3389/fbinf.2023.1131021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Accepted: 04/24/2023] [Indexed: 05/27/2023] Open
Abstract
Metastasis of cancer is directly related to death in almost all cases, however a lot is yet to be understood about this process. Despite advancements in the available radiological investigation techniques, not all cases of Distant Metastasis (DM) are diagnosed at initial clinical presentation. Also, there are currently no standard biomarkers of metastasis. Early, accurate diagnosis of DM is however crucial for clinical decision making, and planning of appropriate management strategies. Previous works have achieved little success in attempts to predict DM from either clinical, genomic, radiology, or histopathology data. In this work we attempt a multimodal approach to predict the presence of DM in cancer patients by combining gene expression data, clinical data and histopathology images. We tested a novel combination of Random Forest (RF) algorithm with an optimization technique for gene selection, and investigated if gene expression pattern in the primary tissues of three cancer types (Bladder Carcinoma, Pancreatic Adenocarcinoma, and Head and Neck Squamous Carcinoma) with DM are similar or different. Gene expression biomarkers of DM identified by our proposed method outperformed Differentially Expressed Genes (DEGs) identified by the DESeq2 software package in the task of predicting presence or absence of DM. Genes involved in DM tend to be more cancer type specific rather than general across all cancers. Our results also indicate that multimodal data is more predictive of metastasis than either of the three unimodal data tested, and genomic data provides the highest contribution by a wide margin. The results re-emphasize the importance for availability of sufficient image data when a weakly supervised training technique is used. Code is made available at: https://github.com/rit-cui-lab/Multimodal-AI-for-Prediction-of-Distant-Metastasis-in-Carcinoma-Patients.
Collapse
|
10
|
Caldonazzi N, Rizzo PC, Eccher A, Girolami I, Fanelli GN, Naccarato AG, Bonizzi G, Fusco N, d'Amati G, Scarpa A, Pantanowitz L, Marletta S. Value of Artificial Intelligence in Evaluating Lymph Node Metastases. Cancers (Basel) 2023; 15:cancers15092491. [PMID: 37173958 PMCID: PMC10177013 DOI: 10.3390/cancers15092491] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Revised: 04/23/2023] [Accepted: 04/25/2023] [Indexed: 05/15/2023] Open
Abstract
One of the most relevant prognostic factors in cancer staging is the presence of lymph node (LN) metastasis. Evaluating lymph nodes for the presence of metastatic cancerous cells can be a lengthy, monotonous, and error-prone process. Owing to digital pathology, artificial intelligence (AI) applied to whole slide images (WSIs) of lymph nodes can be exploited for the automatic detection of metastatic tissue. The aim of this study was to review the literature regarding the implementation of AI as a tool for the detection of metastases in LNs in WSIs. A systematic literature search was conducted in PubMed and Embase databases. Studies involving the application of AI techniques to automatically analyze LN status were included. Of 4584 retrieved articles, 23 were included. Relevant articles were labeled into three categories based upon the accuracy of AI in evaluating LNs. Published data overall indicate that the application of AI in detecting LN metastases is promising and can be proficiently employed in daily pathology practice.
Collapse
Affiliation(s)
- Nicolò Caldonazzi
- Department of Diagnostics and Public Health, Section of Pathology, University of Verona, 37134 Verona, Italy
| | - Paola Chiara Rizzo
- Department of Diagnostics and Public Health, Section of Pathology, University of Verona, 37134 Verona, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, 37126 Verona, Italy
| | - Ilaria Girolami
- Department of Pathology, Lehrkrankenhaus der Paracelsus Medizinischen Privatuniversität, Provincial Hospital of Bolzano (SABES-ASDAA), 39100 Bolzano-Bozen, Italy
| | - Giuseppe Nicolò Fanelli
- Division of Pathology, Department of Translational Research, New Technologies in Medicine and Surgery, University of Pisa, 56126 Pisa, Italy
| | - Antonio Giuseppe Naccarato
- Division of Pathology, Department of Translational Research, New Technologies in Medicine and Surgery, University of Pisa, 56126 Pisa, Italy
| | - Giuseppina Bonizzi
- Division of Pathology, IEO, Europefan Institute of Oncology IRCCS, University of Milan, 20122 Milan, Italy
| | - Nicola Fusco
- Division of Pathology, IEO, Europefan Institute of Oncology IRCCS, University of Milan, 20122 Milan, Italy
- Department of Oncology and Hemato-Oncology, University of Milan, 20122 Milan, Italy
| | - Giulia d'Amati
- Department of Radiology, Oncology and Pathology, Sapienza, University of Rome, 00185 Rome, Italy
| | - Aldo Scarpa
- Department of Diagnostics and Public Health, Section of Pathology, University of Verona, 37134 Verona, Italy
| | - Liron Pantanowitz
- Department of Pathology, University of Michigan, Ann Arbor, MI 48104, USA
| | - Stefano Marletta
- Department of Diagnostics and Public Health, Section of Pathology, University of Verona, 37134 Verona, Italy
- Department of Pathology, Pederzoli Hospital, 37019 Peschiera del Garda, Italy
| |
Collapse
|
11
|
Tsai PC, Lee TH, Kuo KC, Su FY, Lee TLM, Marostica E, Ugai T, Zhao M, Lau MC, Väyrynen JP, Giannakis M, Takashima Y, Kahaki SM, Wu K, Song M, Meyerhardt JA, Chan AT, Chiang JH, Nowak J, Ogino S, Yu KH. Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients. Nat Commun 2023; 14:2102. [PMID: 37055393 PMCID: PMC10102208 DOI: 10.1038/s41467-023-37179-4] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 03/03/2023] [Indexed: 04/15/2023] Open
Abstract
Histopathologic assessment is indispensable for diagnosing colorectal cancer (CRC). However, manual evaluation of the diseased tissues under the microscope cannot reliably inform patient prognosis or genomic variations crucial for treatment selections. To address these challenges, we develop the Multi-omics Multi-cohort Assessment (MOMA) platform, an explainable machine learning approach, to systematically identify and interpret the relationship between patients' histologic patterns, multi-omics, and clinical profiles in three large patient cohorts (n = 1888). MOMA successfully predicts the overall survival, disease-free survival (log-rank test P-value<0.05), and copy number alterations of CRC patients. In addition, our approaches identify interpretable pathology patterns predictive of gene expression profiles, microsatellite instability status, and clinically actionable genetic alterations. We show that MOMA models are generalizable to multiple patient populations with different demographic compositions and pathology images collected from distinctive digitization methods. Our machine learning approaches provide clinically actionable predictions that could inform treatments for colorectal cancer patients.
Collapse
Affiliation(s)
- Pei-Chen Tsai
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Tsung-Hua Lee
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Kun-Chi Kuo
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Fang-Yi Su
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC
| | - Tsung-Lu Michael Lee
- Department of Computer Science and Information Engineering, Southern Taiwan University of Science and Technology, Tainan, Taiwan ROC
| | - Eliana Marostica
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Division of Health Sciences and Technology, Harvard-Massachusetts Institute of Technology, Boston, MA, USA
| | - Tomotaka Ugai
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Melissa Zhao
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Mai Chan Lau
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Juha P Väyrynen
- Cancer and Translational Medicine Research Unit, Medical Research Center Oulu, Oulu University Hospital and University of Oulu, Oulu, Finland
| | - Marios Giannakis
- Department of Medicine, Dana Farber Cancer Institute, Boston, MA, USA
| | | | | | - Kana Wu
- Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Mingyang Song
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | | | - Andrew T Chan
- Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
| | - Jung-Hsien Chiang
- Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan ROC.
| | - Jonathan Nowak
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
| | - Shuji Ogino
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA
- Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Kun-Hsing Yu
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Brigham and Women's Hospital, Boston, MA, USA.
| |
Collapse
|
12
|
Kang H, Yang M, Zhang F, Xu H, Ren S, Li J, Chen D, Wang F, Li D, Chen X. Identification lymph node metastasis in esophageal squamous cell carcinoma using whole slide images and a hybrid network of multiple instance and transfer learning. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
|
13
|
de Souza LL, Fonseca FP, Araújo ALD, Lopes MA, Vargas PA, Khurram SA, Kowalski LP, Dos Santos HT, Warnakulasuriya S, Dolezal J, Pearson AT, Santos-Silva AR. Machine learning for detection and classification of oral potentially malignant disorders: A conceptual review. J Oral Pathol Med 2023; 52:197-205. [PMID: 36792771 DOI: 10.1111/jop.13414] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 12/09/2022] [Indexed: 02/17/2023]
Abstract
Oral potentially malignant disorders represent precursor lesions that may undergo malignant transformation to oral cancer. There are many known risk factors associated with the development of oral potentially malignant disorders, and contribute to the risk of malignant transformation. Although many advances have been reported to understand the biological behavior of oral potentially malignant disorders, their clinical features that indicate the characteristics of malignant transformation are not well established. Early diagnosis of malignancy is the most important factor to improve patients' prognosis. The integration of machine learning into routine diagnosis has recently emerged as an adjunct to aid clinical examination. Increased performances of artificial intelligence AI-assisted medical devices are claimed to exceed the human capability in the clinical detection of early cancer. Therefore, the aim of this narrative review is to introduce artificial intelligence terminology, concepts, and models currently used in oncology to familiarize oral medicine scientists with the language skills, best research practices, and knowledge for developing machine learning models applied to the clinical detection of oral potentially malignant disorders.
Collapse
Affiliation(s)
- Lucas Lacerda de Souza
- Oral Diagnosis, Piracicaba Dental School, University of Campinas (UNICAMP), São Paulo, Brazil
| | - Felipe Paiva Fonseca
- Oral Diagnosis, Piracicaba Dental School, University of Campinas (UNICAMP), São Paulo, Brazil
- Department of Oral Surgery and Pathology, School of Dentistry, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
| | | | - Marcio Ajudarte Lopes
- Oral Diagnosis, Piracicaba Dental School, University of Campinas (UNICAMP), São Paulo, Brazil
| | - Pablo Agustin Vargas
- Oral Diagnosis, Piracicaba Dental School, University of Campinas (UNICAMP), São Paulo, Brazil
| | - Syed Ali Khurram
- Unit of Oral & Maxillofacial Pathology, School of Clinical Dentistry, University of Sheffield, Sheffield, UK
| | - Luiz Paulo Kowalski
- Department of Head and Neck Surgery, University of Sao Paulo Medical School and Department of Head and Neck Surgery and Otorhinolaryngology, AC Camargo Cancer Center, Sao Paulo, Brazil
| | - Harim Tavares Dos Santos
- Department of Otolaryngology-Head and Neck Surgery, University of Missouri, Columbia, Missouri, USA
- Department of Bond Life Sciences Center, University of Missouri, Columbia, Missouri, USA
| | - Saman Warnakulasuriya
- King's College London, London, UK
- WHO Collaborating Centre for Oral Cancer, London, UK
| | - James Dolezal
- Section of Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, Illinois, USA
| | - Alexander T Pearson
- Section of Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, Illinois, USA
| | - Alan Roger Santos-Silva
- Oral Diagnosis, Piracicaba Dental School, University of Campinas (UNICAMP), São Paulo, Brazil
| |
Collapse
|
14
|
Tan L, Li H, Yu J, Zhou H, Wang Z, Niu Z, Li J, Li Z. Colorectal cancer lymph node metastasis prediction with weakly supervised transformer-based multi-instance learning. Med Biol Eng Comput 2023; 61:1565-1580. [PMID: 36809427 PMCID: PMC10182132 DOI: 10.1007/s11517-023-02799-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 01/31/2023] [Indexed: 02/23/2023]
Abstract
Lymph node metastasis examined by the resected lymph nodes is considered one of the most important prognostic factors for colorectal cancer (CRC). However, it requires careful and comprehensive inspection by expert pathologists. To relieve the pathologists' burden and speed up the diagnostic process, in this paper, we develop a deep learning system with the binary positive/negative labels of the lymph nodes to solve the CRC lymph node classification task. The multi-instance learning (MIL) framework is adopted in our method to handle the whole slide images (WSIs) of gigapixels in size at once and get rid of the labor-intensive and time-consuming detailed annotations. First, a transformer-based MIL model, DT-DSMIL, is proposed in this paper based on the deformable transformer backbone and the dual-stream MIL (DSMIL) framework. The local-level image features are extracted and aggregated with the deformable transformer, and the global-level image features are obtained with the DSMIL aggregator. The final classification decision is made based on both the local and the global-level features. After the effectiveness of our proposed DT-DSMIL model is demonstrated by comparing its performance with its predecessors, a diagnostic system is developed to detect, crop, and finally identify the single lymph nodes within the slides based on the DT-DSMIL and the Faster R-CNN model. The developed diagnostic model is trained and tested on a clinically collected CRC lymph node metastasis dataset composed of 843 slides (864 metastasis lymph nodes and 1415 non-metastatic lymph nodes), achieving the accuracy of 95.3% and the area under the receiver operating characteristic curve (AUC) of 0.9762 (95% confidence interval [CI]: 0.9607-0.9891) for the single lymph node classification. As for the lymph nodes with micro-metastasis and macro-metastasis, our diagnostic system achieves the AUC of 0.9816 (95% CI: 0.9659-0.9935) and 0.9902 (95% CI: 0.9787-0.9983), respectively. Moreover, the system shows reliable diagnostic region localizing performance: the model can always identify the most likely metastases, no matter the model's predictions or manual labels, showing great potential in avoiding false negatives and discovering incorrectly labeled slides in actual clinical use.
Collapse
Affiliation(s)
- Luxin Tan
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Department of Pathology, Peking University Cancer Hospital & Institute, Beijing, 100142, China
| | - Huan Li
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Department of Pathology, Peking University Cancer Hospital & Institute, Beijing, 100142, China
| | - Jinze Yu
- Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China.,School of Computer Science and Engineering, Beihang University, Beijing, 100191, China.,Shenyuan Honors College, Beihang University, Beijing, 100191, China
| | - Haoyi Zhou
- Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China.,College of Software, Beihang University, Beijing, 100191, China
| | - Zhi Wang
- Blot Info & Tech (Beijing) Co. Ltd, Beijing, China
| | - Zhiyong Niu
- Blot Info & Tech (Beijing) Co. Ltd, Beijing, China.
| | - Jianxin Li
- Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China. .,School of Computer Science and Engineering, Beihang University, Beijing, 100191, China.
| | - Zhongwu Li
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Department of Pathology, Peking University Cancer Hospital & Institute, Beijing, 100142, China.
| |
Collapse
|
15
|
Khan A, Brouwer N, Blank A, Müller F, Soldini D, Noske A, Gaus E, Brandt S, Nagtegaal I, Dawson H, Thiran JP, Perren A, Lugli A, Zlobec I. Computer-Assisted Diagnosis of Lymph Node Metastases in Colorectal Cancers Using Transfer Learning With an Ensemble Model. Mod Pathol 2023; 36:100118. [PMID: 36805793 DOI: 10.1016/j.modpat.2023.100118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 12/14/2022] [Accepted: 01/21/2023] [Indexed: 02/05/2023]
Abstract
Screening of lymph node metastases in colorectal cancer (CRC) can be a cumbersome task, but it is amenable to artificial intelligence (AI)-assisted diagnostic solution. Here, we propose a deep learning-based workflow for the evaluation of CRC lymph node metastases from digitized hematoxylin and eosin-stained sections. A segmentation model was trained on 100 whole-slide images (WSIs). It achieved a Matthews correlation coefficient of 0.86 (±0.154) and an acceptable Hausdorff distance of 135.59 μm (±72.14 μm), indicating a high congruence with the ground truth. For metastasis detection, 2 models (Xception and Vision Transformer) were independently trained first on a patch-based breast cancer lymph node data set and were then fine-tuned using the CRC data set. After fine-tuning, the ensemble model showed significant improvements in the F1 score (0.797-0.949; P <.00001) and the area under the receiver operating characteristic curve (0.959-0.978; P <.00001). Four independent cohorts (3 internal and 1 external) of CRC lymph nodes were used for validation in cascading segmentation and metastasis detection models. Our approach showed excellent performance, with high sensitivity (0.995, 1.0) and specificity (0.967, 1.0) in 2 validation cohorts of adenocarcinoma cases (n = 3836 slides) when comparing slide-level labels with the ground truth (pathologist reports). Similarly, an acceptable performance was achieved in a validation cohort (n = 172 slides) with mucinous and signet-ring cell histology (sensitivity, 0.872; specificity, 0.936). The patch-based classification confidence was aggregated to overlay the potential metastatic regions within each lymph node slide for visualization. We also applied our method to a consecutive case series of lymph nodes obtained over the past 6 months at our institution (n = 217 slides). The overlays of prediction within lymph node regions matched 100% when compared with a microscope evaluation by an expert pathologist. Our results provide the basis for a computer-assisted diagnostic tool for easy and efficient lymph node screening in patients with CRC.
Collapse
|
16
|
Kindler C, Elfwing S, Öhrvik J, Nikberg M. A Deep Neural Network-Based Decision Support Tool for the Detection of Lymph Node Metastases in Colorectal Cancer Specimens. Mod Pathol 2023; 36:100015. [PMID: 36853787 DOI: 10.1016/j.modpat.2022.100015] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 09/13/2022] [Accepted: 09/16/2022] [Indexed: 01/11/2023]
Abstract
The identification of lymph node metastases in colorectal cancer (CRC) specimens is crucial for the planning of postoperative treatment and can be a time-consuming task for pathologists. In this study, we developed a deep neural network (DNN) algorithm for the detection of metastatic CRC in digitized histologic sections of lymph nodes and evaluated its performance as a diagnostic support tool. First, the DNN algorithm was trained using pixel-level annotations of cancerous areas on 758 whole slide images (360 with cancerous areas). The algorithm's performance was evaluated on 74 whole slide images (43 with cancerous areas). Second, the algorithm was evaluated as a decision support tool on 288 whole slide images covering 1517 lymph node sections, randomized in 16 batches. Two senior pathologists (C.K. and C.O.) assessed each batch with and without the help of the algorithm in a 2 × 2 crossover design, with a washout period of 1 month in between. The time needed for the evaluation of each node section was recorded. The DNN algorithm achieved a median pixel-level accuracy of 0.952 on slides with cancerous areas and 0.996 on slides with benign samples. N+ disease (metastases, micrometastases, or tumor deposits) was present in 103 of the 1517 sections. The algorithm highlighted cancerous areas in 102 of these sections, with a sensitivity of 0.990. Assisted by the algorithm, the median time needed for evaluation was significantly shortened for both pathologists (median time for pathologist 1, 26 vs 14 seconds; P < .001; 95% CI, 11.0-12.0; median time for pathologist 2, 25 vs 23 seconds; P < .001; 95% CI, 2.0-4.0). Our DNN showed high accuracy for detecting metastatic CRC in digitized histologic sections of lymph nodes. This decision support tool has the potential to improve the diagnostic workflow by shortening the time needed for the evaluation of lymph nodes in CRC specimens without impairing diagnostic accuracy.
Collapse
Affiliation(s)
- Csaba Kindler
- Department of Pathology, Laboratory Medicine, Västmanlands Hospital, Västerås, Sweden; Centre for Clinical Research, Uppsala University, Västerås, Sweden.
| | | | - John Öhrvik
- Centre for Clinical Research, Uppsala University, Västerås, Sweden
| | - Maziar Nikberg
- Centre for Clinical Research, Uppsala University, Västerås, Sweden; Department of Surgery, Västmanlands Hospital, Västerås, Sweden
| |
Collapse
|
17
|
Pan J, Hong G, Zeng H, Liao C, Li H, Yao Y, Gan Q, Wang Y, Wu S, Lin T. An artificial intelligence model for the pathological diagnosis of invasion depth and histologic grade in bladder cancer. J Transl Med 2023; 21:42. [PMID: 36691055 PMCID: PMC9869632 DOI: 10.1186/s12967-023-03888-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2022] [Accepted: 01/12/2023] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND Accurate pathological diagnosis of invasion depth and histologic grade is key for clinical management in patients with bladder cancer (BCa), but it is labour-intensive, experience-dependent and subject to interobserver variability. Here, we aimed to develop a pathological artificial intelligence diagnostic model (PAIDM) for BCa diagnosis. METHODS A total of 854 whole slide images (WSIs) from 692 patients were included and divided into training and validation sets. The PAIDM was developed using the training set based on the deep learning algorithm ScanNet, and the performance was verified at the patch level in validation set 1 and at the WSI level in validation set 2. An independent validation cohort (validation set 3) was employed to compare the PAIDM and pathologists. Model performance was evaluated using the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value and negative predictive value. RESULTS The AUCs of the PAIDM were 0.878 (95% CI 0.875-0.881) at the patch level in validation set 1 and 0.870 (95% CI 0.805-0.923) at the WSI level in validation set 2. In comparing the PAIDM and pathologists, the PAIDM achieved an AUC of 0.847 (95% CI 0.779-0.905), which was non-inferior to the average diagnostic level of pathologists. There was high consistency between the model-predicted and manually annotated areas, improving the PAIDM's interpretability. CONCLUSIONS We reported an artificial intelligence-based diagnostic model for BCa that performed well in identifying invasion depth and histologic grade. Importantly, the PAIDM performed admirably in patch-level recognition, with a promising application for transurethral resection specimens.
Collapse
Affiliation(s)
- Jiexin Pan
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Guibin Hong
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Hong Zeng
- Department of Pathology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chengxiao Liao
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
| | - Huarun Li
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
| | - Yuhui Yao
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
| | - Qinghua Gan
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
| | - Yun Wang
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China
| | - Shaoxu Wu
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China.
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China.
- Guangdong Provincial Clinical Research Center for Urological Diseases, Guangzhou, Guangdong, China.
| | - Tianxin Lin
- Department of Urology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, 107th Yanjiangxi Road, Guangzhou, China.
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, China.
- Guangdong Provincial Clinical Research Center for Urological Diseases, Guangzhou, Guangdong, China.
| |
Collapse
|
18
|
Kwon MR, Lee JH, Park J, Park SS, Ju EJ, Ko EJ, Shin SH, Son GW, Lee HW, Kim YJ, Song SY, Jeong SY, Choi EK. NCK-associated protein 1 regulates metastasis and is a novel prognostic marker for colorectal cancer. Cell Death Dis 2023; 9:7. [PMID: 36639705 DOI: 10.1038/s41420-023-01303-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 12/21/2022] [Accepted: 01/04/2023] [Indexed: 01/14/2023]
Abstract
Metastatic colorectal cancer (CRC) remains a substantial problem for mortality and requires screening and early detection efforts to increase survival. Epithelial-mesenchymal transition (EMT) and circulation of tumor cells in the blood play important roles in metastasis. To identify a novel target for metastasis of CRC, we conducted a gene microarray analysis using extracted RNA from the blood of preclinical models. We found that NCK-associated protein 1 (NCKAP1) was significantly increased in the blood RNA of patient-derived xenograft (PDX) models of colon cancer. In the NCKAP1 gene knockdown-induced human colon cancer cell lines HCT116 and HT29, there was a reduced wound healing area and significant inhibition of migration and invasion. As the result of marker screening for cytoskeleton and cellular interactions, CRC treated with siRNA of NCKAP1 exhibited significant induction of CDH1 and phalloidin expression, which indicates enhanced adherent cell junctions and cytoskeleton. In HCT116 cells with a mesenchymal state induced by TGFβ1, metastasis was inhibited by NCKAP1 gene knockdown through the inhibition of migration, and there was increased CTNNB1 expression and decreased FN expression. We established metastasis models for colon cancer to liver transition by intrasplenic injection shRNA of NCKAP1-transfected HCT116 cells or by implanting tumor tissue generated with the cells on cecal pouch. In metastasis xenograft models, tumor growth and liver metastasis were markedly reduced. Taken together, these data demonstrate that NCKAP1 is a novel gene regulating EMT that can contribute to developing a diagnostic marker for the progression of metastasis and new therapeutics for metastatic CRC treatment.
Collapse
|
19
|
Li N, Fan L, Xu H, Zhang X, Bai Z, Li M, Xiong S, Jiang L, Yang J, Chen S, Qiao Y, Chen B. An AI-Aided Diagnostic Framework for Hematologic Neoplasms Based on Morphologic Features and Medical Expertise. J Transl Med 2023; 103:100055. [PMID: 36870286 DOI: 10.1016/j.labinv.2022.100055] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 12/21/2022] [Accepted: 12/27/2022] [Indexed: 01/11/2023] Open
Abstract
A morphologic examination is essential for the diagnosis of hematological diseases. However, its conventional manual operation is time-consuming and laborious. Herein, we attempt to establish an artificial intelligence (AI)-aided diagnostic framework integrating medical expertise. This framework acts as a virtual hematological morphologist (VHM) for diagnosing hematological neoplasms. Two datasets were established as follows: An image dataset was used to train the Faster Region-based Convolutional Neural Network to develop an image-based morphologic feature extraction model. A case dataset containing retrospective morphologic diagnostic data was used to train a support vector machine algorithm to develop a feature-based case identification model based on diagnostic criteria. Integrating these 2 models established a whole-process AI-aided diagnostic framework, namely, VHM, and a 2-stage strategy was applied to practice case diagnosis. The recall and precision of VHM in bone marrow cell classification were 94.65% and 93.95%, respectively. The balanced accuracy, sensitivity, and specificity of VHM were 97.16%, 99.09%, and 92%, respectively, in the differential diagnosis of normal and abnormal cases, and 99.23%, 97.96%, and 100%, respectively, in the precise diagnosis of chronic myelogenous leukemia in chronic phase. This work represents the first attempt, to our knowledge, to extract multimodal morphologic features and to integrate a feature-based case diagnosis model for designing a comprehensive AI-aided morphologic diagnostic framework. The performance of our knowledge-based framework was superior to that of the widely used end-to-end AI-based diagnostic framework in terms of testing accuracy (96.88% vs 68.75%) or generalization ability (97.11% vs 68.75%) in differentiating normal and abnormal cases. The remarkable advantage of VHM is that it follows the logic of clinical diagnostic procedures, making it a reliable and interpretable hematological diagnostic tool.
Collapse
|
20
|
Liao J, Li X, Gan Y, Han S, Rong P, Wang W, Li W, Zhou L. Artificial intelligence assists precision medicine in cancer treatment. Front Oncol 2023; 12:998222. [PMID: 36686757 PMCID: PMC9846804 DOI: 10.3389/fonc.2022.998222] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 11/22/2022] [Indexed: 01/06/2023] Open
Abstract
Cancer is a major medical problem worldwide. Due to its high heterogeneity, the use of the same drugs or surgical methods in patients with the same tumor may have different curative effects, leading to the need for more accurate treatment methods for tumors and personalized treatments for patients. The precise treatment of tumors is essential, which renders obtaining an in-depth understanding of the changes that tumors undergo urgent, including changes in their genes, proteins and cancer cell phenotypes, in order to develop targeted treatment strategies for patients. Artificial intelligence (AI) based on big data can extract the hidden patterns, important information, and corresponding knowledge behind the enormous amount of data. For example, the ML and deep learning of subsets of AI can be used to mine the deep-level information in genomics, transcriptomics, proteomics, radiomics, digital pathological images, and other data, which can make clinicians synthetically and comprehensively understand tumors. In addition, AI can find new biomarkers from data to assist tumor screening, detection, diagnosis, treatment and prognosis prediction, so as to providing the best treatment for individual patients and improving their clinical outcomes.
Collapse
Affiliation(s)
- Jinzhuang Liao
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Xiaoying Li
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Yu Gan
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Shuangze Han
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China
| | - Pengfei Rong
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China,Cell Transplantation and Gene Therapy Institute, The Third Xiangya Hospital, Central South University, Changsha, Hunan, China,*Correspondence: Pengfei Rong, ; Wei Wang, ; Wei Li, ; Li Zhou,
| | - Wei Wang
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China,Cell Transplantation and Gene Therapy Institute, The Third Xiangya Hospital, Central South University, Changsha, Hunan, China,*Correspondence: Pengfei Rong, ; Wei Wang, ; Wei Li, ; Li Zhou,
| | - Wei Li
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China,Cell Transplantation and Gene Therapy Institute, The Third Xiangya Hospital, Central South University, Changsha, Hunan, China,*Correspondence: Pengfei Rong, ; Wei Wang, ; Wei Li, ; Li Zhou,
| | - Li Zhou
- Department of Radiology, The Third Xiangya Hospital of Central South University, Changsha, Hunan, China,Cell Transplantation and Gene Therapy Institute, The Third Xiangya Hospital, Central South University, Changsha, Hunan, China,Department of Pathology, The Xiangya Hospital of Central South University, Changsha, Hunan, China,*Correspondence: Pengfei Rong, ; Wei Wang, ; Wei Li, ; Li Zhou,
| |
Collapse
|
21
|
Ferreira I, Montenegro CS, Coelho D, Pereira M, da Mata S, Carvalho S, Araújo AC, Abrantes C, Ruivo JM, Garcia H, Oliveira RC. Digital pathology implementation in a private laboratory: The CEDAP experience. J Pathol Inform 2023; 14:100180. [PMID: 36687527 PMCID: PMC9853351 DOI: 10.1016/j.jpi.2022.100180] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 12/23/2022] [Accepted: 12/28/2022] [Indexed: 01/05/2023] Open
Abstract
Introduction The transition to digital pathology has been carried out by several laboratories across the globe, with some cases described in Portugal. In this article, we describe the transition to digital pathology in a high-volume private laboratory, considering the main challenges and opportunities. Material and methods Our process started in 2020, with laboratory workflow adaptation and we are currently using a high-capacity scanner (Aperio GT450DX) to digitize slides at 20×. The visualization system, Aperio eSlide Manager WebViewer, is integrated into the Laboratory System. The validation process followed the Royal College of Pathologists Guidelines. Results Regarding validation, the first phase detected an error rate of 6.8%, mostly due to digitization errors. Phase optimization and collaboration with technical services led to improvements in this process. In the second validation phase, most of the slides had the desired quality for evaluation, with only an error rate of 0.6%, corrected with a new scan. The interpathologist correlation had a total agreement rate of 96.87% and 3.13% partial agreement. Conclusion The implementation and validation of digital pathology was a success, being ready for prime time. The total integration of all laboratory systems and the acquisition of new equipment will maximize their use, especially with the application of artificial intelligence algorithms.
Collapse
Affiliation(s)
- Inês Ferreira
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | | | - Daniel Coelho
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Maria Pereira
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Sara da Mata
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Sofia Carvalho
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal,Hospital de Santa Luzia de Viana do Castelo, ULSAM, EPE, Viana do castelo, Portugal
| | - Ana Catarina Araújo
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Carlos Abrantes
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - José Mário Ruivo
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Helena Garcia
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal
| | - Rui Caetano Oliveira
- Germano de Sousa - Centro de Diagnóstico Histopatológico CEDAP, Coimbra, Portugal,Institute for Clinical and Biomedical Research (iCBR), Faculty of Medicine, University of Coimbra, Coimbra, Portugal,Centre of Investigation on Genetics and Oncobiology (CIMAGO), Coimbra, Portugal,Clinical and Academic Centre of Coimbra (CACC), Coimbra, Portugal,Corresponding author.
| |
Collapse
|
22
|
Zhu L, Shi H, Wei H, Wang C, Shi S, Zhang F, Yan R, Liu Y, He T, Wang L, Cheng J, Duan H, Du H, Meng F, Zhao W, Gu X, Guo L, Ni Y, He Y, Guan T, Han A. An accurate prediction of the origin for bone metastatic cancer using deep learning on digital pathological images. EBioMedicine 2022; 87:104426. [PMID: 36577348 PMCID: PMC9803701 DOI: 10.1016/j.ebiom.2022.104426] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Revised: 12/07/2022] [Accepted: 12/07/2022] [Indexed: 12/28/2022] Open
Abstract
BACKGROUND Determining the origin of bone metastatic cancer (OBMC) is of great significance to clinical therapeutics. It is challenging for pathologists to determine the OBMC with limited clinical information and bone biopsy. METHODS We designed a regional multiple-instance learning algorithm to predict the OBMC based on hematoxylin-eosin (H&E) staining slides alone. We collected 1041 cases from eight different hospitals and labeled 26,431 regions of interest to train the model. The performance of the model was assessed by ten-fold cross validation and external validation. Under the guidance of top3 predictions, we conducted an IHC test on 175 cases of unknown origins to compare the consistency of the results predicted by the model and indicated by the IHC markers. We also applied the model to identify whether there was tumor or not in a region, as well as distinguishing squamous cell carcinoma, adenocarcinoma, and neuroendocrine tumor. FINDINGS In the within-cohort, our model achieved a top1-accuracy of 91.35% and a top3-accuracy of 97.75%. In the external cohort, our model displayed a good generalizability with a top3-accuracy of 97.44%. The top1 consistency between the results of the model and the immunohistochemistry markers was 83.90% and the top3 consistency was 94.33%. The model obtained an accuracy of 98.98% to identify whether there was tumor or not and an accuracy of 93.85% to differentiate three types of cancers. INTERPRETATION Our model demonstrated good performance to predict the OBMC from routine histology and had great potential for assisting pathologists with determining the OBMC accurately. FUNDING National Science Foundation of China (61875102 and 61975089), Natural Science Foundation of Guangdong province (2021A15-15012379 and 2022A1515 012550), Science and Technology Research Program of Shenzhen City (JCYJ20200109110606054 and WDZC20200821141349001), and Tsinghua University Spring Breeze Fund (2020Z99CFZ023).
Collapse
Affiliation(s)
- Lianghui Zhu
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Huijuan Shi
- Department of Pathology, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Huiting Wei
- Department of Pathology, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chengjiang Wang
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Shanshan Shi
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Fenfen Zhang
- Department of Pathology, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Renao Yan
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Yiqing Liu
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Tingting He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Liyuan Wang
- Department of Pathology, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Junru Cheng
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Hufei Duan
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China
| | - Hong Du
- Department of Pathology, Guangzhou First People's Hospital, School of Medicine, South China University of Technology, Guangzhou, China
| | - Fengjiao Meng
- Department of Pathology, Zhongshan People's Hospital, Zhongshan, China
| | - Wenli Zhao
- Department of Pathology, The First People's Hospital of Huizhou, Huizhou, China
| | - Xia Gu
- Department of Pathology, the First Affiliated Hospital of Guangzhou Medical University, Guangzhou, China
| | - Linlang Guo
- Department of Pathology, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Yingpeng Ni
- Department of Pathology, Jieyang People's Hospital (Jieyang Affiliated Hospital, Sun Yat-Sen University), Jieyang, China
| | - Yonghong He
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China,Corresponding author.
| | - Tian Guan
- Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Shenzhen, Guangdong, China,Corresponding author.
| | - Anjia Han
- Department of Pathology, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China,Corresponding author.
| |
Collapse
|
23
|
Chuang WY, Yu WH, Lee YC, Zhang QY, Chang H, Shih LY, Yeh CJ, Lin SMT, Chang SH, Ueng SH, Wang TH, Hsueh C, Kuo CF, Chuang SS, Yeh CY. Deep Learning-Based Nuclear Morphometry Reveals an Independent Prognostic Factor in Mantle Cell Lymphoma. Am J Pathol 2022; 192:1763-1778. [PMID: 36150505 DOI: 10.1016/j.ajpath.2022.08.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 07/18/2022] [Accepted: 08/15/2022] [Indexed: 06/16/2023]
Abstract
Blastoid/pleomorphic morphology is associated with short survival in mantle cell lymphoma (MCL), but its prognostic value is overridden by Ki-67 in multivariate analysis. Herein, a nuclear segmentation model was developed using deep learning, and nuclei of tumor cells in 103 MCL cases were automatically delineated. Eight nuclear morphometric attributes were extracted from each nucleus. The mean, variance, skewness, and kurtosis of each attribute were calculated for each case, resulting in 32 morphometric parameters. Compared with those in classic MCL, 17 morphometric parameters were significantly different in blastoid/pleomorphic MCL. Using univariate analysis, 16 morphometric parameters (including 14 significantly different between classic and blastoid/pleomorphic MCL) emerged as significant prognostic factors. Using multivariate analysis, Biologic MCL International Prognostic Index (bMIPI) risk group (P = 0.025), low skewness of nuclear irregularity (P = 0.020), and high mean of nuclear irregularity (P = 0.047) emerged as independent adverse prognostic factors. Additionally, a morphometric score calculated from the skewness and mean of nuclear irregularity (P = 0.0038) was an independent prognostic factor in addition to bMIPI risk group (P = 0.025), and a summed morphometric bMIPI score was useful for risk stratification of patients with MCL (P = 0.000001). These results demonstrate, for the first time, that a nuclear morphometric score is an independent prognostic factor in MCL. It is more robust than blastoid/pleomorphic morphology and can be objectively measured.
Collapse
Affiliation(s)
- Wen-Yu Chuang
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, Taoyuan, Taiwan; School of Medicine, Chang Gung University, Taoyuan, Taiwan; Chang Gung Molecular Medicine Research Center, Chang Gung University, Taoyuan, Taiwan; Center for Vascularized Composite Allotransplantation, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | | | - Yen-Chen Lee
- School of Medicine, Chang Gung University, Taoyuan, Taiwan
| | | | - Hung Chang
- School of Medicine, Chang Gung University, Taoyuan, Taiwan; Division of Hematology and Oncology, Department of Internal Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Lee-Yung Shih
- School of Medicine, Chang Gung University, Taoyuan, Taiwan; Division of Hematology and Oncology, Department of Internal Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Chi-Ju Yeh
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, Taoyuan, Taiwan; School of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Samuel Mu-Tse Lin
- aetherAI, Co, Ltd, Taipei, Taiwan; Taipei American School, Taipei, Taiwan
| | - Shang-Hung Chang
- School of Medicine, Chang Gung University, Taoyuan, Taiwan; Center for Big Data Analytics and Statistics, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Shir-Hwa Ueng
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, Taoyuan, Taiwan; School of Medicine, Chang Gung University, Taoyuan, Taiwan; Chang Gung Molecular Medicine Research Center, Chang Gung University, Taoyuan, Taiwan
| | - Tong-Hong Wang
- Tissue Bank, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Chuen Hsueh
- Department of Pathology, Chang Gung Memorial Hospital and Chang Gung University, Taoyuan, Taiwan; School of Medicine, Chang Gung University, Taoyuan, Taiwan; Chang Gung Molecular Medicine Research Center, Chang Gung University, Taoyuan, Taiwan
| | - Chang-Fu Kuo
- School of Medicine, Chang Gung University, Taoyuan, Taiwan; Center for Artificial Intelligence in Medicine, Chang Gung Memorial Hospital, Taoyuan, Taiwan
| | - Shih-Sung Chuang
- Department of Pathology, Chi-Mei Medical Center, Tainan, Taiwan.
| | | |
Collapse
|
24
|
Huang SC, Chen CC, Lan J, Hsieh TY, Chuang HC, Chien MY, Ou TS, Chen KH, Wu RC, Liu YJ, Cheng CT, Huang YJ, Tao LW, Hwu AF, Lin IC, Hung SH, Yeh CY, Chen TC. Deep neural network trained on gigapixel images improves lymph node metastasis detection in clinical settings. Nat Commun 2022; 13:3347. [PMID: 35688834 DOI: 10.1038/s41467-022-30746-1] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Accepted: 05/17/2022] [Indexed: 12/13/2022] Open
Abstract
The pathological identification of lymph node (LN) metastasis is demanding and tedious. Although convolutional neural networks (CNNs) possess considerable potential in improving the process, the ultrahigh-resolution of whole slide images hinders the development of a clinically applicable solution. We design an artificial-intelligence-assisted LN assessment workflow to facilitate the routine counting of metastatic LNs. Unlike previous patch-based approaches, our proposed method trains CNNs by using 5-gigapixel images, obviating the need for lesion-level annotations. Trained on 5907 LN images, our algorithm identifies metastatic LNs in gastric cancer with a slide-level area under the receiver operating characteristic curve (AUC) of 0.9936. Clinical experiments reveal that the workflow significantly improves the sensitivity of micrometastasis identification (81.94% to 95.83%, P < .001) and isolated tumor cells (67.95% to 96.15%, P < .001) in a significantly shorter review time (−31.5%, P < .001). Cross-site evaluation indicates that the algorithm is highly robust (AUC = 0.9829). The pathological identification of lymph node metastasis in whole-slide images is demanding and tedious. Here, the authors design an artificial-intelligence-assisted assessment workflow to facilitate the routine counting of metastatic LNs.
Collapse
|
25
|
Yin M, Lin J, Liu L, Gao J, Xu W, Yu C, Qu S, Liu X, Qian L, Xu C, Zhu J. Development of a Deep Learning Model for Malignant Small Bowel Tumors Survival: A SEER-Based Study. Diagnostics (Basel) 2022; 12. [PMID: 35626403 DOI: 10.3390/diagnostics12051247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 05/11/2022] [Accepted: 05/12/2022] [Indexed: 11/29/2022] Open
Abstract
Background This study aims to explore a deep learning (DL) algorithm for developing a prognostic model and perform survival analyses in SBT patients. Methods The demographic and clinical features of patients with SBTs were extracted from the Surveillance, Epidemiology and End Results (SEER) database. We randomly split the samples into the training set and the validation set at 7:3. Cox proportional hazards (Cox-PH) analysis and the DeepSurv algorithm were used to develop models. The performance of the Cox-PH and DeepSurv models was evaluated using receiver operating characteristic curves, calibration curves, C-statistics and decision-curve analysis (DCA). A Kaplan−Meier (K−M) survival analysis was performed for further explanation on prognostic effect of the Cox-PH model. Results The multivariate analysis demonstrated that seven variables were associated with cancer-specific survival (CSS) (all p < 0.05). The DeepSurv model showed better performance than the Cox-PH model (C-index: 0.871 vs. 0.866). The calibration curves and DCA revealed that the two models had good discrimination and calibration. Moreover, patients with ileac malignancy and N2 stage disease were not responding to surgery according to the K−M analysis. Conclusions This study reported a DeepSurv model that performed well in CSS in SBT patients. It might offer insights into future research to explore more DL algorithms in cohort studies.
Collapse
|
26
|
Davri A, Birbas E, Kanavos T, Ntritsos G, Giannakeas N, Tzallas AT, Batistatou A. Deep Learning on Histopathological Images for Colorectal Cancer Diagnosis: A Systematic Review. Diagnostics (Basel) 2022; 12:837. [PMID: 35453885 DOI: 10.3390/diagnostics12040837] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 03/22/2022] [Accepted: 03/25/2022] [Indexed: 02/04/2023] Open
Abstract
Colorectal cancer (CRC) is the second most common cancer in women and the third most common in men, with an increasing incidence. Pathology diagnosis complemented with prognostic and predictive biomarker information is the first step for personalized treatment. The increased diagnostic load in the pathology laboratory, combined with the reported intra- and inter-variability in the assessment of biomarkers, has prompted the quest for reliable machine-based methods to be incorporated into the routine practice. Recently, Artificial Intelligence (AI) has made significant progress in the medical field, showing potential for clinical applications. Herein, we aim to systematically review the current research on AI in CRC image analysis. In histopathology, algorithms based on Deep Learning (DL) have the potential to assist in diagnosis, predict clinically relevant molecular phenotypes and microsatellite instability, identify histological features related to prognosis and correlated to metastasis, and assess the specific components of the tumor microenvironment.
Collapse
|
27
|
Tang H, Li G, Liu C, Huang D, Zhang X, Qiu Y, Liu Y. Diagnosis of lymph node metastasis in head and neck squamous cell carcinoma using deep learning. Laryngoscope Investig Otolaryngol 2022; 7:161-169. [PMID: 35155794 PMCID: PMC8823170 DOI: 10.1002/lio2.742] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Accepted: 01/04/2022] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND To build an automatic pathological diagnosis model to assess the lymph node metastasis status of head and neck squamous cell carcinoma (HNSCC) based on deep learning algorithms. STUDY DESIGN A retrospective study. METHODS A diagnostic model integrating two-step deep learning networks was trained to analyze the metastasis status in 85 images of HNSCC lymph nodes. The diagnostic model was tested in a test set of 21 images with metastasis and 29 images without metastasis. All images were scanned from HNSCC lymph node sections stained with hematoxylin-eosin (HE). RESULTS In the test set, the overall accuracy, sensitivity, and specificity of the diagnostic model reached 86%, 100%, and 75.9%, respectively. CONCLUSIONS Our two-step diagnostic model can be used to automatically assess the status of HNSCC lymph node metastasis with high sensitivity. LEVEL OF EVIDENCE NA.
Collapse
Affiliation(s)
- Haosheng Tang
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
| | - Guo Li
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
- National Clinical Research Center for Geriatric Disorders (Xiangya Hospital)ChangshaHunanChina
| | - Chao Liu
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
| | - Donghai Huang
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
| | - Xin Zhang
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
| | - Yuanzheng Qiu
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
- National Clinical Research Center for Geriatric Disorders (Xiangya Hospital)ChangshaHunanChina
| | - Yong Liu
- Department of Otolaryngology‐Head and Neck Surgery, Xiangya HospitalCentral South UniversityChangshaHunanChina
- Otolaryngology Major Disease Research Key Laboratory of Hunan ProvinceChangshaHunanChina
- Clinical Research Center for Laryngopharyngeal and Voice Disorders in Hunan ProvinceChangshaHunanChina
- National Clinical Research Center for Geriatric Disorders (Xiangya Hospital)ChangshaHunanChina
| |
Collapse
|