1
|
Sajithkumar A, Thomas J, Saji AM, Ali F, E K HH, Adampulan HAG, Sarathchand S. Artificial Intelligence in pathology: current applications, limitations, and future directions. Ir J Med Sci 2024; 193:1117-1121. [PMID: 37542634 DOI: 10.1007/s11845-023-03479-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Accepted: 07/26/2023] [Indexed: 08/07/2023]
Abstract
PURPOSE Given AI's recent success in computer vision applications, majority of pathologists anticipate that it will be able to assist them with a variety of digital pathology activities. Massive improvements in deep learning have enabled a synergy between Artificial Intelligence (AI) and deep learning, enabling image-based diagnosis against the backdrop of digital pathology. AI-based solutions are being developed to eliminate errors and save pathologists time. AIMS In this paper, we will discuss the components that went into the use of Artificial Intelligence in Pathology, its use in the medical profession, the obstacles and constraints that it encounters, and the future possibilities of AI in the medical field. CONCLUSIONS Based on these factors, we elaborate upon the use of AI in medical pathology and provide future recommendations for its successful implementation in this field.
Collapse
Affiliation(s)
- Akhil Sajithkumar
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India.
| | - Jubin Thomas
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Ajish Meprathumalil Saji
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Fousiya Ali
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Haneena Hasin E K
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Hannan Abdul Gafoor Adampulan
- Department of Oral Pathology and Microbiology, Malabar Dental College and Research Centre, Manoor Chekanoor Road, Mudur PO, Edappal, Malappuram Dist, 679578, India
| | - Swathy Sarathchand
- Sree Narayana Institute of Medical Sciences, Chalakka - Kuthiathode Rd, North Kuthiathode, Kunnukara, Kerala, 683594, India
| |
Collapse
|
2
|
Kefeli J, Tatonetti N. TCGA-Reports: A machine-readable pathology report resource for benchmarking text-based AI models. PATTERNS (NEW YORK, N.Y.) 2024; 5:100933. [PMID: 38487800 PMCID: PMC10935496 DOI: 10.1016/j.patter.2024.100933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Revised: 10/16/2023] [Accepted: 01/25/2024] [Indexed: 03/17/2024]
Abstract
In cancer research, pathology report text is a largely untapped data source. Pathology reports are routinely generated, more nuanced than structured data, and contain added insight from pathologists. However, there are no publicly available datasets for benchmarking report-based models. Two recent advances suggest the urgent need for a benchmark dataset. First, improved optical character recognition (OCR) techniques will make it possible to access older pathology reports in an automated way, increasing the data available for analysis. Second, recent improvements in natural language processing (NLP) techniques using artificial intelligence (AI) allow more accurate prediction of clinical targets from text. We apply state-of-the-art OCR and customized post-processing to report PDFs from The Cancer Genome Atlas, generating a machine-readable corpus of 9,523 reports. Finally, we perform a proof-of-principle cancer-type classification across 32 tissues, achieving 0.992 average AU-ROC. This dataset will be useful to researchers across specialties, including research clinicians, clinical trial investigators, and clinical NLP researchers.
Collapse
Affiliation(s)
- Jenna Kefeli
- Department of Systems Biology, Columbia University, New York, NY 10032, USA
| | - Nicholas Tatonetti
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA 90048, USA
| |
Collapse
|
3
|
Lopez-Beltran A, Cookson MS, Guercio BJ, Cheng L. Advances in diagnosis and treatment of bladder cancer. BMJ 2024; 384:e076743. [PMID: 38346808 DOI: 10.1136/bmj-2023-076743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/15/2024]
Abstract
Bladder cancer remains a leading cause of cancer death worldwide and is associated with substantial impacts on patient quality of life, morbidity, mortality, and cost to the healthcare system. Gross hematuria frequently precedes the diagnosis of bladder cancer. Non-muscle-invasive bladder cancer (NMIBC) is managed initially with transurethral resection of a bladder tumor (TURBT), followed by a risk stratified approach to adjuvant intravesical therapy (IVe), and is associated with an overall survival of 90%. However, cure rates remain lower for muscle invasive bladder cancer (MIBC) owing to a variety of factors. NMIBC and MIBC groupings are heterogeneous and have unique pathological and molecular characteristics. Indeed, The Cancer Genome Atlas project identified genetic drivers and luminal and basal molecular subtypes of MIBC with distinct treatment responses. For NMIBC, IVe immunotherapy (primarily BCG) is the gold standard treatment for high grade and high risk NMIBC to reduce or prevent both recurrence and progression after initial TURBT; novel trials incorporate immune checkpoint inhibitors. IVe gene therapy and combination IVe chemotherapy have recently been completed, with promising results. For localized MIBC, essential goals are improving care and reducing morbidity following cystectomy or bladder preserving strategies. In metastatic disease, advances in understanding of the genomic landscape and tumor microenvironment have led to the implementation of immune checkpoint inhibitors, targeted treatments, and antibody-drug conjugates. Defining better selection criteria to identify the patients most likely to benefit from a specific treatment is an urgent need.
Collapse
Affiliation(s)
- Antonio Lopez-Beltran
- Department of Morphological Sciences, Unit of Anatomic Pathology, University of Cordoba Medical School, Cordoba, Spain
| | - Michael S Cookson
- Department of Urology, University of Oklahoma Health Sciences Center and the Stephenson Cancer Center, Oklahoma City, OK, US
| | - Brendan J Guercio
- Department of Medicine, James P. Wilmot Cancer Institute, University of Rochester Medical Center, Rochester, NY, US
| | - Liang Cheng
- Department of Pathology and Laboratory Medicine, Department of Surgery (Urology), Brown University, Providence, RI, US
- Legorreta Cancer Center, Brown University
- Lifespan Health Care System, Brown University
| |
Collapse
|
4
|
Laurie MA, Zhou SR, Islam MT, Shkolyar E, Xing L, Liao JC. Bladder Cancer and Artificial Intelligence: Emerging Applications. Urol Clin North Am 2024; 51:63-75. [PMID: 37945103 PMCID: PMC10697017 DOI: 10.1016/j.ucl.2023.07.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
Bladder cancer is a common and heterogeneous disease that poses a significant burden to the patient and health care system. Major unmet needs include effective early detection strategy, imprecision of risk stratification, and treatment-associated morbidities. The existing clinical paradigm is imprecise, which results in missed tumors, suboptimal therapy, and disease progression. Artificial intelligence holds immense potential to address many unmet needs in bladder cancer, including early detection, risk stratification, treatment planning, quality assessment, and outcome prediction. Despite recent advances, extensive work remains to affirm the efficacy of artificial intelligence as a decision-making tool for bladder cancer management.
Collapse
Affiliation(s)
- Mark A Laurie
- Department of Urology, Stanford University School of Medicine, 453 Quarry Road, Mail Code 5656, Palo Alto, CA 94304, USA; Department of Radiation Oncology, Stanford University School of Medicine, 875 Blake Wilbur Drive Room G204, Stanford, CA 94305-5847, USA; Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, USA; Institute for Computational and Mathematical Engineering, Stanford University School of Engineering, Stanford, CA 94305, USA
| | - Steve R Zhou
- Department of Urology, Stanford University School of Medicine, 453 Quarry Road, Mail Code 5656, Palo Alto, CA 94304, USA
| | - Md Tauhidul Islam
- Department of Radiation Oncology, Stanford University School of Medicine, 875 Blake Wilbur Drive Room G204, Stanford, CA 94305-5847, USA
| | - Eugene Shkolyar
- Department of Urology, Stanford University School of Medicine, 453 Quarry Road, Mail Code 5656, Palo Alto, CA 94304, USA; Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, USA
| | - Lei Xing
- Department of Radiation Oncology, Stanford University School of Medicine, 875 Blake Wilbur Drive Room G204, Stanford, CA 94305-5847, USA
| | - Joseph C Liao
- Department of Urology, Stanford University School of Medicine, 453 Quarry Road, Mail Code 5656, Palo Alto, CA 94304, USA; Veterans Affairs Palo Alto Health Care System, Palo Alto, CA 94304, USA.
| |
Collapse
|
5
|
Grobet-Jeandin E, Lenfant L, Pinar U, Parra J, Mozer P, Renard-Penna R, Thibault C, Rouprêt M, Seisen T. Management of patients with muscle-invasive bladder cancer with clinical evidence of pelvic lymph node metastases. Nat Rev Urol 2024:10.1038/s41585-023-00842-y. [PMID: 38297079 DOI: 10.1038/s41585-023-00842-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/01/2023] [Indexed: 02/02/2024]
Abstract
Identification of clinically positive pelvic lymph node metastases (cN+) in patients with muscle-invasive bladder cancer is currently challenging, as the diagnostic accuracy of available imaging modalities is limited. Conventional CT is still considered the gold-standard approach to diagnose lymph node metastases in these patients. The development of innovative diagnostic methods including radiomics, artificial intelligence-based models and molecular biomarkers might offer new perspectives for the diagnosis of cN+ disease. With regard to the treatment of these patients, multimodal strategies are likely to provide the best oncological outcomes, especially using induction chemotherapy followed by radical cystectomy and pelvic lymph node dissection in responders to chemotherapy. Additionally, the use of adjuvant nivolumab has been shown to decrease the risk of recurrence in patients who still harbour ypT2-T4a and/or ypN+ disease after surgery. Alternatively, the use of avelumab maintenance therapy can be offered to patients with unresectable cN+ tumours who have at least stable disease after induction chemotherapy alone. Lastly, patients with cN+ tumours who are not responding to induction chemotherapy are potential candidates for receiving second-line treatment with pembrolizumab.
Collapse
Affiliation(s)
- Elisabeth Grobet-Jeandin
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
- Division of Urology, Geneva University Hospitals, Geneva, Switzerland
| | - Louis Lenfant
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
| | - Ugo Pinar
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
| | - Jérôme Parra
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
| | - Pierre Mozer
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
| | - Raphaele Renard-Penna
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Radiology, 75013, Paris, France
| | - Constance Thibault
- Department of medical oncology, Hôpital Européen Georges Pompidou, Institut du Cancer Paris CARPEM, AP-HP centre, Paris, France
| | - Morgan Rouprêt
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France
| | - Thomas Seisen
- Sorbonne University, GRC 5, Predictive Onco-Urology, APHP, Pitié-Salpêtrière Hospital, Urology, 75013, Paris, France.
| |
Collapse
|
6
|
Wagner SJ, Matek C, Shetab Boushehri S, Boxberg M, Lamm L, Sadafi A, Winter DJE, Marr C, Peng T. Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology. Mod Pathol 2024; 37:100350. [PMID: 37827448 DOI: 10.1016/j.modpat.2023.100350] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 10/02/2023] [Accepted: 10/03/2023] [Indexed: 10/14/2023]
Abstract
Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.
Collapse
Affiliation(s)
- Sophia J Wagner
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Computation, Information and Technology, Technical University of Munich, Garching, Germany
| | - Christian Matek
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
| | - Sayedali Shetab Boushehri
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Data & Analytics (D&A), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Germany
| | - Melanie Boxberg
- Institute of Pathology, Technical University Munich, Munich, Germany; Institute of Pathology Munich-North, Munich, Germany
| | - Lorenz Lamm
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Helmholtz Pioneer Campus, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Ario Sadafi
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Dominik J E Winter
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Life Sciences, Technical University of Munich, Weihenstephan, Germany
| | - Carsten Marr
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| | - Tingying Peng
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| |
Collapse
|
7
|
Kefeli J, Tatonetti N. Benchmark Pathology Report Text Corpus with Cancer Type Classification. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.08.03.23293618. [PMID: 37609238 PMCID: PMC10441484 DOI: 10.1101/2023.08.03.23293618] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
In cancer research, pathology report text is a largely un-tapped data source. Pathology reports are routinely generated, more nuanced than structured data, and contain added insight from pathologists. However, there are no publicly-available datasets for benchmarking report-based models. Two recent advances suggest the urgent need for a benchmark dataset. First, improved optical character recognition (OCR) techniques will make it possible to access older pathology reports in an automated way, increasing data available for analysis. Second, recent improvements in natural language processing (NLP) techniques using AI allow more accurate prediction of clinical targets from text. We apply state-of-the-art OCR and customized post-processing to publicly available report PDFs from The Cancer Genome Atlas, generating a machine-readable corpus of 9,523 reports. We perform a proof-of-principle cancer-type classification across 32 tissues, achieving 0.992 average AU-ROC. This dataset will be useful to researchers across specialties, including research clinicians, clinical trial investigators, and clinical NLP researchers.
Collapse
Affiliation(s)
- Jenna Kefeli
- Department of Systems Biology, Columbia University, New York, New York, 10032, United States
| | - Nicholas Tatonetti
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, California, 90048, United States
| |
Collapse
|
8
|
Senthil Kumar K, Miskovic V, Blasiak A, Sundar R, Pedrocchi ALG, Pearson AT, Prelaj A, Ho D. Artificial Intelligence in Clinical Oncology: From Data to Digital Pathology and Treatment. Am Soc Clin Oncol Educ Book 2023; 43:e390084. [PMID: 37235822 DOI: 10.1200/edbk_390084] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
Recently, a wide spectrum of artificial intelligence (AI)-based applications in the broader categories of digital pathology, biomarker development, and treatment have been explored. In the domain of digital pathology, these have included novel analytical strategies for realizing new information derived from standard histology to guide treatment selection and biomarker development to predict treatment selection and response. In therapeutics, these have included AI-driven drug target discovery, drug design and repurposing, combination regimen optimization, modulated dosing, and beyond. Given the continued advances that are emerging, it is important to develop workflows that seamlessly combine the various segments of AI innovation to comprehensively augment the diagnostic and interventional arsenal of the clinical oncology community. To overcome challenges that remain with regard to the ideation, validation, and deployment of AI in clinical oncology, recommendations toward bringing this workflow to fruition are also provided from clinical, engineering, implementation, and health care economics considerations. Ultimately, this work proposes frameworks that can potentially integrate these domains toward the sustainable adoption of practice-changing AI by the clinical oncology community to drive improved patient outcomes.
Collapse
Affiliation(s)
- Kirthika Senthil Kumar
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
| | - Vanja Miskovic
- Department of Electronics, Informatics, and Bioengineering, Politecnico di Milano, Milan, Italy
- Department of Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Agata Blasiak
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Raghav Sundar
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Haematology-Oncology, National University Cancer Institute, National University Hospital
- Department of Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- Singapore Gastric Cancer Consortium, Singapore
- NUS Centre for Cancer Research (N2CR), National University of Singapore, Singapore
| | | | - Alexander T Pearson
- Section of Hematology/Oncology, Department of Medicine, University of Chicago, Chicago, IL
- University of Chicago Comprehensive Cancer Center, Chicago, IL
| | - Arsela Prelaj
- Department of Electronics, Informatics, and Bioengineering, Politecnico di Milano, Milan, Italy
- Department of Medical Oncology, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Dean Ho
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore
- The N.1 Institute for Health (N.1), National University of Singapore, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| |
Collapse
|
9
|
Ameen YA, Badary DM, Abonnoor AEI, Hussain KF, Sewisy AA. Which data subset should be augmented for deep learning? a simulation study using urothelial cell carcinoma histopathology images. BMC Bioinformatics 2023; 24:75. [PMID: 36869300 PMCID: PMC9983182 DOI: 10.1186/s12859-023-05199-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Accepted: 02/21/2023] [Indexed: 03/05/2023] Open
Abstract
BACKGROUND Applying deep learning to digital histopathology is hindered by the scarcity of manually annotated datasets. While data augmentation can ameliorate this obstacle, its methods are far from standardized. Our aim was to systematically explore the effects of skipping data augmentation; applying data augmentation to different subsets of the whole dataset (training set, validation set, test set, two of them, or all of them); and applying data augmentation at different time points (before, during, or after dividing the dataset into three subsets). Different combinations of the above possibilities resulted in 11 ways to apply augmentation. The literature contains no such comprehensive systematic comparison of these augmentation ways. RESULTS Non-overlapping photographs of all tissues on 90 hematoxylin-and-eosin-stained urinary bladder slides were obtained. Then, they were manually classified as either inflammation (5948 images), urothelial cell carcinoma (5811 images), or invalid (3132 images; excluded). If done, augmentation was eight-fold by flipping and rotation. Four convolutional neural networks (Inception-v3, ResNet-101, GoogLeNet, and SqueezeNet), pre-trained on the ImageNet dataset, were fine-tuned to binary classify images of our dataset. This task was the benchmark for our experiments. Model testing performance was evaluated using accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve. Model validation accuracy was also estimated. The best testing performance was achieved when augmentation was done to the remaining data after test-set separation, but before division into training and validation sets. This leaked information between the training and the validation sets, as evidenced by the optimistic validation accuracy. However, this leakage did not cause the validation set to malfunction. Augmentation before test-set separation led to optimistic results. Test-set augmentation yielded more accurate evaluation metrics with less uncertainty. Inception-v3 had the best overall testing performance. CONCLUSIONS In digital histopathology, augmentation should include both the test set (after its allocation), and the remaining combined training/validation set (before being split into separate training and validation sets). Future research should try to generalize our results.
Collapse
Affiliation(s)
- Yusra A Ameen
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt.
| | - Dalia M Badary
- Department of Pathology, Faculty of Medicine, Assiut University, Asyut, Egypt
| | | | - Khaled F Hussain
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt
| | - Adel A Sewisy
- Department of Computer Science, Faculty of Computers and Information, Assiut University, Asyut, Egypt
| |
Collapse
|
10
|
Xue P, Si M, Qin D, Wei B, Seery S, Ye Z, Chen M, Wang S, Song C, Zhang B, Ding M, Zhang W, Bai A, Yan H, Dang L, Zhao Y, Rezhake R, Zhang S, Qiao Y, Qu Y, Jiang Y. Unassisted Clinicians Versus Deep Learning-Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis. J Med Internet Res 2023; 25:e43832. [PMID: 36862499 PMCID: PMC10020907 DOI: 10.2196/43832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 01/19/2023] [Accepted: 02/13/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND A number of publications have demonstrated that deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, but these algorithms are frequently considered as opponents rather than partners. Despite the clinicians-in-the-loop DL approach having great potential, no study has systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. OBJECTIVE We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. METHODS PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between January 1, 2012, and December 7, 2021. Any type of study design was permitted that focused on comparing unassisted clinicians and DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform-data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality. RESULTS In total, 9796 studies were identified, of which 48 were deemed eligible for systematic review. Twenty-five of these studies made comparisons between unassisted clinicians and DL-assisted clinicians and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95% CI 80%-86%) for unassisted clinicians and 88% (95% CI 86%-90%) for DL-assisted clinicians. Pooled specificity was 86% (95% CI 83%-88%) for unassisted clinicians and 88% (95% CI 85%-90%) for DL-assisted clinicians. The pooled sensitivity and specificity values for DL-assisted clinicians were higher than for unassisted clinicians, at ratios of 1.07 (95% CI 1.05-1.09) and 1.03 (95% CI 1.02-1.05), respectively. Similar diagnostic performance by DL-assisted clinicians was also observed across the predefined subgroups. CONCLUSIONS The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised, because the evidence provided in the reviewed studies does not cover all the minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data-science approaches may improve DL-assisted practice, although further research is required. TRIAL REGISTRATION PROSPERO CRD42021281372; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=281372.
Collapse
Affiliation(s)
- Peng Xue
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyu Si
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Dongxu Qin
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bingrui Wei
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Samuel Seery
- Faculty of Health and Medicine, Division of Health Research, Lancaster University, Lancaster, United Kingdom
| | - Zichen Ye
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyang Chen
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Sumeng Wang
- Department of Cancer Epidemiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Cheng Song
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bo Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ming Ding
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wenling Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Anying Bai
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Huijiao Yan
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Le Dang
- Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuqian Zhao
- Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, School of Medicine, University of Electronic Science & Technology of China, Sichuan, China
| | - Remila Rezhake
- Affiliated Cancer Hospital, The 3rd Affiliated Teaching Hospital of Xinjiang Medical University, Xinjiang, China
| | - Shaokai Zhang
- Henan Cancer Hospital, Affiliated Cancer Hospital of Zhengzhou University, Henan, China
| | - Youlin Qiao
- Center for Global Health, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yimin Qu
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yu Jiang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
11
|
Patkar S, Beck J, Harmon S, Mazcko C, Turkbey B, Choyke P, Brown GT, LeBlanc A. Deep Domain Adversarial Learning for Species-Agnostic Classification of Histologic Subtypes of Osteosarcoma. THE AMERICAN JOURNAL OF PATHOLOGY 2023; 193:60-72. [PMID: 36309101 PMCID: PMC9798510 DOI: 10.1016/j.ajpath.2022.09.009] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 08/31/2022] [Accepted: 09/28/2022] [Indexed: 11/07/2022]
Abstract
Osteosarcomas (OSs) are aggressive bone tumors with many divergent histologic patterns. During pathology review, OSs are subtyped based on the predominant histologic pattern; however, tumors often demonstrate multiple patterns. This high tumor heterogeneity coupled with scarcity of samples compared with other tumor types render histology-based prognosis of OSs challenging. To combat lower case numbers in humans, dogs with spontaneous OSs have been suggested as a model species. Herein, a convolutional neural network was adversarially trained to classify distinct histologic patterns of OS in humans using mostly canine OS data during training. Adversarial training improved domain adaption of a histologic subtype classifier from canines to humans, achieving an average multiclass F1 score of 0.77 (95% CI, 0.74-0.79) and 0.80 (95% CI, 0.78-0.81) when compared with the ground truth in canines and humans, respectively. Finally, this trained model, when used to characterize the histologic landscape of 306 canine OSs, uncovered distinct clusters with markedly different clinical responses to standard-of-care therapy.
Collapse
Affiliation(s)
- Sushant Patkar
- Artificial Intelligence Resource, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland.
| | - Jessica Beck
- Comparative Oncology Program, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - Stephanie Harmon
- Artificial Intelligence Resource, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - Christina Mazcko
- Comparative Oncology Program, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - Baris Turkbey
- Artificial Intelligence Resource, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - Peter Choyke
- Artificial Intelligence Resource, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - G. Thomas Brown
- Artificial Intelligence Resource, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland
| | - Amy LeBlanc
- Comparative Oncology Program, Molecular Imaging Branch, National Cancer Institute, NIH, Bethesda, Maryland.
| |
Collapse
|
12
|
Parwani AV, Patel A, Zhou M, Cheville JC, Tizhoosh H, Humphrey P, Reuter VE, True LD. An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS). J Pathol Inform 2023; 14:100177. [PMID: 36654741 PMCID: PMC9841212 DOI: 10.1016/j.jpi.2022.100177] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 12/20/2022] [Accepted: 12/20/2022] [Indexed: 12/31/2022] Open
Abstract
Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.
Collapse
Affiliation(s)
- Anil V. Parwani
- The Ohio State University, Columbus, Ohio, USA
- Corresponding author.
| | - Ankush Patel
- The Ohio State University, 2441 60th Ave SE, Mercer Island, Washington 98040, USA
| | - Ming Zhou
- Tufts University, Medford, Massachusetts, USA
| | | | | | | | | | | |
Collapse
|
13
|
Rivero Belenchón I, Checcucci E, Gómez Rivas J, Puliatti S, Taratkin M, Kowalewski KF, Rodler S, Veccia A, Medina Lopez RA, Cacciamani G. Comment on "Artificial intelligence to predict oncological outcome directly from hematoxylin and eosin-stained slides in urology: a systematic review". Minerva Urol Nephrol 2022; 74:810-812. [PMID: 36629813 DOI: 10.23736/s2724-6051.22.05180-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Affiliation(s)
- Inés Rivero Belenchón
- Department of Urology and Nephrology, Biomedical Institute of Seville (IBiS), Virgen del Rocío University Hospital, University of Seville, Seville, Spain - .,Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands -
| | - Enrico Checcucci
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Division of Urology, Department of Surgery, IRCC Candiolo Cancer Institute, Candiolo, Turin, Italy
| | - Juan Gómez Rivas
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, San Carlos Hospital, Madrid, Spain
| | - Stefano Puliatti
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Mark Taratkin
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Institute for Urology and Reproductive Health, Sechenov University, Moscow, Russia
| | - Karl-Friedrich Kowalewski
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, University Medical Center Mannheim, University of Heidelberg, Mannheim, Germany
| | - Severin Rodler
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Department of Urology, Klinikum der Univertität München, Munich, Germany
| | - Alessandro Veccia
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Unit of Urology, AOUI Verona, Verona, Italy
| | - Rafael A Medina Lopez
- Department of Urology and Nephrology, Biomedical Institute of Seville (IBiS), Virgen del Rocío University Hospital, University of Seville, Seville, Spain
| | - Giovanni Cacciamani
- Young Academic Urologist-Urotechnology Working Party (ESUT-YAU), European Association of Urology, Arnhem, the Neatherlands.,Catherine and Joseph Aresty Department of Urology, US Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | | |
Collapse
|
14
|
Wessels F, Kuntz S, Krieghoff-Henning E, Schmitt M, Braun V, Worst TS, Neuberger M, Steeg M, Gaiser T, Fröhling S, Michel MS, Nuhn P, Brinker TJ. Artificial intelligence to predict oncological outcome directly from hematoxylin and eosin-stained slides in urology. Minerva Urol Nephrol 2022; 74:538-550. [PMID: 35274903 DOI: 10.23736/s2724-6051.22.04758-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
INTRODUCTION Artificial intelligence (AI) has been successfully applied for automatic tumor detection and grading in histopathological image analysis in urologic oncology. The aim of this review was to assess the applicability of these approaches in image-based oncological outcome prediction. EVIDENCE ACQUISITION A systematic literature search was conducted using the databases MEDLINE through PubMed and Web of Science up to April 20, 2021. Studies investigating AI approaches to determine the risk of recurrence, metastasis, or survival directly from H&E-stained tissue sections in prostate, renal cell or urothelial carcinoma were included. Characteristics of the AI approach and performance metrics were extracted and summarized. Risk of bias (RoB) was assessed using the PROBAST tool. EVIDENCE SYNTHESIS 16 studies yielding a total of 6658 patients and reporting on 17 outcome predictions were included. Six studies focused on renal cell, six on prostate and three on urothelial carcinoma while one study investigated renal cell and urothelial carcinoma. Handcrafted feature extraction was used in five, a convolutional neural network (CNN) in six and a deep feature extraction in four studies. One study compared a CNN with handcrafted feature extraction. In seven outcome predictions, a multivariable comparison with clinicopathological parameters was reported. Five of them showed statistically significant hazard ratios for the AI's model's-prediction. However, RoB was high in 15 outcome predictions and unclear in two. CONCLUSIONS The included studies are promising but predominantly early pilot studies, therefore primarily highlighting the potential of AI approaches. Additional well-designed studies are needed to assess the actual clinical applicability.
Collapse
Affiliation(s)
- Frederik Wessels
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.,Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Sara Kuntz
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Eva Krieghoff-Henning
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Max Schmitt
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Volker Braun
- Library for the Medical Faculty Mannheim of Heidelberg University, Mannheim, Germany
| | - Thomas S Worst
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Manuel Neuberger
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Matthias Steeg
- Institute of Pathology, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Timo Gaiser
- Institute of Pathology, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Stefan Fröhling
- National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Maurice-Stephan Michel
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Philipp Nuhn
- Department of Urology and Urological Surgery, Medical Faculty Mannheim of Heidelberg University, University Medical Center Mannheim, Mannheim, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany -
| |
Collapse
|
15
|
Liu XP, Yang X, Xiong M, Mao X, Jin X, Li Z, Zhou S, Chang H. Development and validation of chest CT-based imaging biomarkers for early stage COVID-19 screening. Front Public Health 2022; 10:1004117. [PMID: 36211676 PMCID: PMC9533142 DOI: 10.3389/fpubh.2022.1004117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Accepted: 08/15/2022] [Indexed: 01/27/2023] Open
Abstract
Coronavirus Disease 2019 (COVID-19) is currently a global pandemic, and early screening is one of the key factors for COVID-19 control and treatment. Here, we developed and validated chest CT-based imaging biomarkers for COVID-19 patient screening from two independent hospitals with 419 patients. We identified the vasculature-like signals from CT images and found that, compared to healthy and community acquired pneumonia (CAP) patients, COVID-19 patients display a significantly higher abundance of these signals. Furthermore, unsupervised feature learning led to the discovery of clinical-relevant imaging biomarkers from the vasculature-like signals for accurate and sensitive COVID-19 screening that have been double-blindly validated in an independent hospital (sensitivity: 0.941, specificity: 0.920, AUC: 0.971, accuracy 0.931, F1 score: 0.929). Our findings could open a new avenue to assist screening of COVID-19 patients.
Collapse
Affiliation(s)
- Xiao-Ping Liu
- Department of Pathology, Zhongnan Hospital of Wuhan University, Wuhan, China
- Department of Urology, Zhongnan Hospital of Wuhan University, Wuhan, China
| | - Xu Yang
- Key Laboratory of Modern Toxicology of Ministry of Education, School of Public Health, Nanjing Medical University, Nanjing, China
| | - Miao Xiong
- Department of Radiology, Wuhan Third Hospital, Tongren Hospital of Wuhan University, Wuhan, China
| | - Xuanyu Mao
- Department of Emergency, Zhongnan Hospital of Wuhan University, Wuhan, China
| | - Xiaoqing Jin
- Department of Emergency, Zhongnan Hospital of Wuhan University, Wuhan, China
| | - Zhiqiang Li
- Department of Neurosurgery, Zhongnan Hospital of Wuhan University, Wuhan, China
| | - Shuang Zhou
- Hubei Province Hospital of Traditional Chinese Medicine, Affiliated Hospital of Hubei University of Traditional Chinese Medicine, Hubei Institute of Traditional Chinese Medicine, Wuhan, China
| | - Hang Chang
- Department of Emergency, Zhongnan Hospital of Wuhan University, Wuhan, China
| |
Collapse
|
16
|
Rasmussen R, Sanford T, Parwani AV, Pedrosa I. Artificial Intelligence in Kidney Cancer. Am Soc Clin Oncol Educ Book 2022; 42:1-11. [PMID: 35580292 DOI: 10.1200/edbk_350862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Artificial intelligence is rapidly expanding into nearly all facets of life, particularly within the field of medicine. The diagnosis, characterization, management, and treatment of kidney cancer is ripe with areas for improvement that may be met with the promises of artificial intelligence. Here, we explore the impact of current research work in artificial intelligence for clinicians caring for patients with renal cancer, with a focus on the perspectives of radiologists, pathologists, and urologists. Promising preliminary results indicate that artificial intelligence may assist in the diagnosis and risk stratification of newly discovered renal masses and help guide the clinical treatment of patients with kidney cancer. However, much of the work in this field is still in its early stages, limited in its broader applicability, and hampered by small datasets, the varied appearance and presentation of kidney cancers, and the intrinsic limitations of the rigidly structured tasks artificial intelligence algorithms are trained to complete. Nonetheless, the continued exploration of artificial intelligence holds promise toward improving the clinical care of patients with kidney cancer.
Collapse
Affiliation(s)
- Robert Rasmussen
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX
| | - Thomas Sanford
- Department of Urology, Upstate Medical University, Syracuse, NY
| | - Anil V Parwani
- Department of Pathology, The Ohio State University, Columbus, OH
| | - Ivan Pedrosa
- Department of Radiology, The University of Texas Southwestern Medical Center, Dallas, TX.,Department of Urology, The University of Texas Southwestern Medical Center, Dallas, TX.,Advanced Imaging Research Center, The University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
17
|
Großerueschkamp F, Jütte H, Gerwert K, Tannapfel A. Advances in Digital Pathology: From Artificial Intelligence to Label-Free Imaging. Visc Med 2021; 37:482-490. [PMID: 35087898 DOI: 10.1159/000518494] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Accepted: 07/14/2021] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Digital pathology, in its primary meaning, describes the utilization of computer screens to view scanned histology slides. Digitized tissue sections can be easily shared for a second opinion. In addition, it allows tissue image analysis using specialized software to identify and measure events previously observed by a human observer. These tissue-based readouts were highly reproducible and precise. Digital pathology has developed over the years through new technologies. Currently, the most discussed development is the application of artificial intelligence to automatically analyze tissue images. However, even new label-free imaging technologies are being developed to allow imaging of tissues by means of their molecular composition. SUMMARY This review provides a summary of the current state-of-the-art and future digital pathologies. Developments in the last few years have been presented and discussed. In particular, the review provides an outlook on interesting new technologies (e.g., infrared imaging), which would allow for deeper understanding and analysis of tissue thin sections beyond conventional histopathology. KEY MESSAGES In digital pathology, mathematical methods are used to analyze images and draw conclusions about diseases and their progression. New innovative methods and techniques (e.g., label-free infrared imaging) will bring significant changes in the field in the coming years.
Collapse
Affiliation(s)
- Frederik Großerueschkamp
- Center for Protein Diagnostics (PRODI), Biospectroscopy, Ruhr University Bochum, Bochum, Germany.,Department of Biophysics, Faculty of Biology and Biotechnology, Ruhr University Bochum, Bochum, Germany
| | - Hendrik Jütte
- Center for Protein Diagnostics (PRODI), Biospectroscopy, Ruhr University Bochum, Bochum, Germany.,Institute of Pathology, Ruhr University Bochum, Bochum, Germany
| | - Klaus Gerwert
- Center for Protein Diagnostics (PRODI), Biospectroscopy, Ruhr University Bochum, Bochum, Germany.,Department of Biophysics, Faculty of Biology and Biotechnology, Ruhr University Bochum, Bochum, Germany
| | - Andrea Tannapfel
- Center for Protein Diagnostics (PRODI), Biospectroscopy, Ruhr University Bochum, Bochum, Germany.,Institute of Pathology, Ruhr University Bochum, Bochum, Germany
| |
Collapse
|
18
|
Xu F, Zhu C, Tang W, Wang Y, Zhang Y, Li J, Jiang H, Shi Z, Liu J, Jin M. Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides. Front Oncol 2021; 11:759007. [PMID: 34722313 PMCID: PMC8551965 DOI: 10.3389/fonc.2021.759007] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2021] [Accepted: 09/21/2021] [Indexed: 12/22/2022] Open
Abstract
Objectives To develop and validate a deep learning (DL)-based primary tumor biopsy signature for predicting axillary lymph node (ALN) metastasis preoperatively in early breast cancer (EBC) patients with clinically negative ALN. Methods A total of 1,058 EBC patients with pathologically confirmed ALN status were enrolled from May 2010 to August 2020. A DL core-needle biopsy (DL-CNB) model was built on the attention-based multiple instance-learning (AMIL) framework to predict ALN status utilizing the DL features, which were extracted from the cancer areas of digitized whole-slide images (WSIs) of breast CNB specimens annotated by two pathologists. Accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curves, and areas under the ROC curve (AUCs) were analyzed to evaluate our model. Results The best-performing DL-CNB model with VGG16_BN as the feature extractor achieved an AUC of 0.816 (95% confidence interval (CI): 0.758, 0.865) in predicting positive ALN metastasis in the independent test cohort. Furthermore, our model incorporating the clinical data, which was called DL-CNB+C, yielded the best accuracy of 0.831 (95%CI: 0.775, 0.878), especially for patients younger than 50 years (AUC: 0.918, 95%CI: 0.825, 0.971). The interpretation of DL-CNB model showed that the top signatures most predictive of ALN metastasis were characterized by the nucleus features including density (p = 0.015), circumference (p = 0.009), circularity (p = 0.010), and orientation (p = 0.012). Conclusion Our study provides a novel DL-based biomarker on primary tumor CNB slides to predict the metastatic status of ALN preoperatively for patients with EBC.
Collapse
Affiliation(s)
- Feng Xu
- Department of Breast Surgery, Beijing Chao-Yang Hospital, Beijing, China
| | - Chuang Zhu
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
| | - Wenqi Tang
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
| | - Ying Wang
- Department of Pathology, Beijing Chao-Yang Hospital, Beijing, China
| | - Yu Zhang
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
| | - Jie Li
- Department of Breast Surgery, Beijing Chao-Yang Hospital, Beijing, China
| | - Hongchuan Jiang
- Department of Breast Surgery, Beijing Chao-Yang Hospital, Beijing, China
| | - Zhongyue Shi
- Department of Pathology, Beijing Chao-Yang Hospital, Beijing, China
| | - Jun Liu
- School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China
| | - Mulan Jin
- Department of Pathology, Beijing Chao-Yang Hospital, Beijing, China
| |
Collapse
|
19
|
Giovagnoli MR, Ciucciarelli S, Castrichella L, Giansanti D. Artificial Intelligence in Digital Pathology: What Is the Future? Part 2: An Investigation on the Insiders. Healthcare (Basel) 2021; 9:healthcare9101347. [PMID: 34683027 PMCID: PMC8544344 DOI: 10.3390/healthcare9101347] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 10/08/2021] [Accepted: 10/09/2021] [Indexed: 01/01/2023] Open
Abstract
Motivation: This study deals with the introduction of artificial intelligence (AI) in digital pathology (DP). The study starts from the highlights of a companion paper. Objective: The aim was to investigate the consensus and acceptance of the insiders on this issue. Procedure: An electronic survey based on the standardized package Microsoft Forms (Microsoft, Redmond, WA, USA) was proposed to a sample of biomedical laboratory technicians (149 admitted in the study, 76 males, 73 females, mean age 44.2 years). Results: The survey showed no criticality. It highlighted (a) the good perception of the basic training on both groups, and (b) a uniformly low perceived knowledge of AI (as arisen from the graded questions). Expectations, perceived general impact, perceived changes in the work-flow, and worries clearly emerged in the study. Conclusions: The of AI in DP is an unstoppable process, as well as the increase of the digitalization in the health domain. Stakeholders must not look with suspicion towards AI, which can represent an important resource, but should invest in monitoring and consensus training initiatives based also on electronic surveys.
Collapse
Affiliation(s)
- Maria Rosaria Giovagnoli
- Facoltà di Medicina e Psicologia, Università Sapienza Roma, Piazzale Aldo Moro, 00185 Rome, Italy; (M.R.G.); (S.C.); (L.C.)
| | - Sara Ciucciarelli
- Facoltà di Medicina e Psicologia, Università Sapienza Roma, Piazzale Aldo Moro, 00185 Rome, Italy; (M.R.G.); (S.C.); (L.C.)
| | - Livia Castrichella
- Facoltà di Medicina e Psicologia, Università Sapienza Roma, Piazzale Aldo Moro, 00185 Rome, Italy; (M.R.G.); (S.C.); (L.C.)
| | - Daniele Giansanti
- Centre Tisp, Istituto Superiore di Sanità, 00161 Rome, Italy
- Correspondence: ; Tel.: +39-06-49902701
| |
Collapse
|
20
|
Giovagnoli MR, Giansanti D. Artificial Intelligence in Digital Pathology: What Is the Future? Part 1: From the Digital Slide Onwards. Healthcare (Basel) 2021; 9:healthcare9070858. [PMID: 34356236 PMCID: PMC8304979 DOI: 10.3390/healthcare9070858] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Revised: 07/01/2021] [Accepted: 07/04/2021] [Indexed: 12/13/2022] Open
Abstract
This commentary aims to address the field of Artificial intelligence (AI) in Digital Pathology (DP) both in terms of the global situation and research perspectives. It has four polarities. First, it revisits the evolutions of digital pathology with particular care to the two fields of the digital cytology and the digital histology. Second, it illustrates the main fields in the employment of AI in DP. Third, it looks at the future directions of the research challenges from both a clinical and technological point of view. Fourth, it discusses the transversal problems among these challenges and implications and introduces the immediate work to implement.
Collapse
Affiliation(s)
| | - Daniele Giansanti
- Centre Tisp, Istituto Superiore di Sanità, 00161 Roma, Italy
- Correspondence: ; Tel.: +39-06-49902701
| |
Collapse
|
21
|
The potential of AI in cancer care and research. Biochim Biophys Acta Rev Cancer 2021; 1876:188573. [PMID: 34052390 DOI: 10.1016/j.bbcan.2021.188573] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Revised: 05/25/2021] [Accepted: 05/25/2021] [Indexed: 02/08/2023]
Abstract
Current applications of artificial intelligence (AI), machine learning, and deep learning in cancer research and clinical care are highly diverse-from aiding radiologists in reading medical images to predicting oncoprotein folding and dynamics. The list of available AI-based tools is growing rapidly and will only continue to expand. With the immense potential for AI to advance cancer research and clinical care, the National Cancer Institute (NCI) has a responsibility to consider and support the development and evaluation of such technologies. NCI's current involvement in AI research spans the spectrum of development, implementation, and assessment. That includes generating large, publicly available, curated datasets; shifting the culture of data sharing; training the next generation of scientists in both AI and cancer sciences; fostering interdisciplinary collaborations; investing in research to improve AI methods and models that are designed specifically for cancer; widening access to computing power; procuring computer architecture for future developments; and assuring AI research and technologies follow ethical principles. In addition to a broad overview of AI applications in cancer research and care, and NCI's ongoing AI-based activities, this Perspective outlines NCI's four priority areas for future investment of cancer-focused AI development.
Collapse
|
22
|
Abstract
PURPOSE OF REVIEW Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology. RECENT FINDINGS There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application. SUMMARY In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.
Collapse
|
23
|
Abstract
PURPOSE OF REVIEW This review aims to shed light on recent applications of artificial intelligence in urologic oncology. RECENT FINDINGS Artificial intelligence algorithms harness the wealth of patient data to assist in diagnosing, staging, treating, and monitoring genitourinary malignancies. Successful applications of artificial intelligence in urologic oncology include interpreting diagnostic imaging, pathology, and genomic annotations. Many of these algorithms, however, lack external validity and can only provide predictions based on one type of dataset. SUMMARY Future applications of artificial intelligence will need to incorporate several forms of data in order to truly make headway in urologic oncology. Researchers must actively ensure future artificial intelligence developments encompass the entire prospective patient population.
Collapse
|
24
|
Kang Y, Zhu X, Wang X, Liao S, Jin M, Zhang L, Wu X, Zhao T, Zhang J, Lv J, Zhu D. Identification and Validation of the Prognostic Stemness Biomarkers in Bladder Cancer Bone Metastasis. Front Oncol 2021; 11:641184. [PMID: 33816287 PMCID: PMC8017322 DOI: 10.3389/fonc.2021.641184] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2020] [Accepted: 01/29/2021] [Indexed: 12/22/2022] Open
Abstract
Background Bladder urothelial carcinoma (BLCA) is one of the most common urinary system malignancies with a high metastasis rate. Cancer stem cells (CSCs) play an important role in the occurrence and progression of BLCA, however, its roles in bone metastasis and the prognostic stemness biomarkers have not been identified in BLCA. Method In order to identify the roles of CSC in the tumorigenesis, bone metastasis and prognosis of BLCA, the RNA sequencing data of patients with BLCA were retrieved from The Cancer Genome Atlas (TCGA) databases. The mRNA expression-based stemness index (mRNAsi) and the differential expressed genes (DEGs) were evaluated and identified. The associations between mRNAsi and the tumorigenesis, bone metastasis, clinical stage and overall survival (OS) were also established. The key prognostic stemness-related genes (PSRGs) were screened by Lasso regression, and based on them, the predict model was constructed. Its accuracy was tested by the area under the curve (AUC) of the receiver operator characteristic (ROC) curve and the risk score. Additionally, in order to explore the key regulatory network, the relationship among differentially expressing TFs, PSRGs, and absolute quantification of 50 hallmarks of cancer were also identified by Pearson correlation analysis. To verify the identified key TFs and PSRGs, their expression levels were identified by our clinical samples via immunohistochemistry (IHC). Results A total of 8,647 DEGs were identified between 411 primary BLCAs and 19 normal solid tissue samples. According to the clinical stage, mRNAsi and bone metastasis of BLCA, 2,383 stage-related DEGs, 3,680 stemness-related DEGs and 716 bone metastasis-associated DEGs were uncovered, respectively. Additionally, compared with normal tissue, mRNAsi was significantly upregulated in the primary BLCA and also associated with the prognosis (P = 0.016), bone metastasis (P < 0.001) and AJCC clinical stage (P < 0.001) of BLCA patients. A total of 20 PSRGs were further screened by Lasso regression, and based on them, we constructed the predict model with a relatively high accuracy (AUC: 0.699). Moreover, we found two key TFs (EPO, ARID3A), four key PRSGs (CACNA1E, LINC01356, CGA and SSX3) and five key hallmarks of cancer gene sets (DNA repair, myc targets, E2F targets, mTORC1 signaling and unfolded protein response) in the regulatory network. The tissue microarray of BLCA and BLCA bone metastasis also revealed high expression of the key TFs (EPO, ARID3A) and PRSGs (SSX3) in BLCA. Conclusion Our study identifies mRNAsi as a reliable index in predicting the tumorigenesis, bone metastasis and prognosis of patients with BLCA and provides a well-applied model for predicting the OS for patients with BLCA based on 20 PSRGs. Besides, we also identified the regulatory network between key PSRGs and cancer gene sets in mediating the BLCA bone metastasis.
Collapse
Affiliation(s)
- Yao Kang
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Xiaojun Zhu
- Department of Musculoskeletal Oncology, Sun Yat-Sen University Cancer Center, Guangzhou, China.,State Key Laboratory of Oncology in South China, Guangzhou, China.,Collaborative Innovation Center for Cancer Medicine, Guangzhou, China
| | - Xijun Wang
- State Key Laboratory of Oncology in South China, Guangzhou, China.,Collaborative Innovation Center for Cancer Medicine, Guangzhou, China.,Department of Head and Neck Surgery, Sun Yat-Sen University Cancer Center, Guangzhou, China
| | - Shiyao Liao
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Mengran Jin
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Li Zhang
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Xiangyang Wu
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Tingxiao Zhao
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Jun Zhang
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Jun Lv
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| | - Danjie Zhu
- Department of Orthopedics, Zhejiang Provincial People's Hospital, Hangzhou, China.,Department of Orthopedics, Hangzhou Medical College People's Hospital, Hangzhou, China
| |
Collapse
|
25
|
Yerram NK, Ball MW. EDITORIAL COMMENT. Urology 2020; 144:156-157. [DOI: 10.1016/j.urology.2020.05.095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2020] [Accepted: 05/17/2020] [Indexed: 11/25/2022]
|
26
|
Rakha EA, Toss M, Shiino S, Gamble P, Jaroensri R, Mermel CH, Chen PHC. Current and future applications of artificial intelligence in pathology: a clinical perspective. J Clin Pathol 2020; 74:409-414. [PMID: 32763920 DOI: 10.1136/jclinpath-2020-206908] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2020] [Accepted: 07/07/2020] [Indexed: 12/17/2022]
Abstract
During the last decade, a dramatic rise in the development and application of artificial intelligence (AI) tools for use in pathology services has occurred. This trend is often expected to continue and reshape the field of pathology in the coming years. The deployment of computational pathology and applications of AI tools can be considered as a paradigm shift that will change pathology services, making them more efficient and capable of meeting the needs of this era of precision medicine. Despite the success of AI models, the translational process from discovery to clinical applications has been slow. The gap between self-contained research and clinical environment may be too wide and has been largely neglected. In this review, we cover the current and prospective applications of AI in pathology. We examine its applications in diagnosis and prognosis, and we offer insights for considerations that could improve clinical applicability of these tools. Then, we discuss its potential to improve workflow efficiency, and its benefits in pathologist education. Finally, we review the factors that could influence adoption in clinical practices and the associated regulatory processes.
Collapse
Affiliation(s)
- Emad A Rakha
- Histopathology, University of Nottingham School of Medicine, Nottingham, UK
| | - Michael Toss
- Histopathology, University of Nottingham School of Medicine, Nottingham, UK
| | - Sho Shiino
- Histopathology, University of Nottingham School of Medicine, Nottingham, UK
| | - Paul Gamble
- Google Health, Google, Palo Alto, California, USA
| | | | | | | |
Collapse
|