1
|
Kludt C, Wang Y, Ahmad W, Bychkov A, Fukuoka J, Gaisa N, Kühnel M, Jonigk D, Pryalukhin A, Mairinger F, Klein F, Schultheis AM, Seper A, Hulla W, Brägelmann J, Michels S, Klein S, Quaas A, Büttner R, Tolkach Y. Next-generation lung cancer pathology: Development and validation of diagnostic and prognostic algorithms. Cell Rep Med 2024; 5:101697. [PMID: 39178857 DOI: 10.1016/j.xcrm.2024.101697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2024] [Revised: 06/25/2024] [Accepted: 07/31/2024] [Indexed: 08/26/2024]
Abstract
Non-small cell lung cancer (NSCLC) is one of the most common malignant tumors. In this study, we develop a clinically useful computational pathology platform for NSCLC that can be a foundation for multiple downstream applications and provide immediate value for patient care optimization and individualization. We train the primary multi-class tissue segmentation algorithm on a substantial, high-quality, manually annotated dataset of whole-slide images with lung adenocarcinoma and squamous cell carcinomas. We investigate two downstream applications. NSCLC subtyping algorithm is trained and validated using a large, multi-institutional (n = 6), multi-scanner (n = 5), international cohort of NSCLC cases (slides/patients 4,097/1,527). Moreover, we develop four AI-derived, fully explainable, quantitative, prognostic parameters (based on tertiary lymphoid structure and necrosis assessment) and validate them for different clinical endpoints. The computational platform enables the high-precision, quantitative analysis of H&E-stained slides. The developed prognostic parameters facilitate robust and independent risk stratification of patients with NSCLC.
Collapse
Affiliation(s)
- Carina Kludt
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Yuan Wang
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Waleed Ahmad
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa 296-0041, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki 852-8131, Japan
| | - Junya Fukuoka
- Department of Pathology, Kameda Medical Center, Kamogawa 296-0041, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki 852-8131, Japan
| | - Nadine Gaisa
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany; Institute of Pathology, University Hospital Ulm, 89081 Ulm, Germany
| | - Mark Kühnel
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany
| | - Danny Jonigk
- Institute of Pathology, University Hospital Aachen, 52074 Aachen, Germany; German Center for Lung Research, DZL, BREATH, 30625 Hanover, Germany
| | - Alexey Pryalukhin
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria
| | - Fabian Mairinger
- Institute of Pathology, University Hospital Essen, 45147 Essen, Germany
| | - Franziska Klein
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany
| | - Anne Maria Schultheis
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Alexander Seper
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria; Danube Private University, 3500 Krems an der Donau, Austria
| | - Wolfgang Hulla
- Institute of Clinical Pathology and Molecular Pathology, Wiener Neustadt State Hospital, 2700 Wiener Neustadt, Austria
| | - Johannes Brägelmann
- University of Cologne, Faculty of Medicine and University Hospital Cologne, Department of Translational Genomics, 50937 Cologne, Germany; Mildred Scheel School of Oncology, Faculty of Medicine and University Hospital Cologne, University of Cologne, 50937 Cologne, Germany; University of Cologne, Faculty of Medicine and University Hospital Cologne, Center for Molecular Medicine Cologne, 50937 Cologne, Germany
| | - Sebastian Michels
- University of Cologne, Faculty of Medicine and University Hospital of Colone, Lung Cancer Group Cologne, Department I for Internal Medicine and Center for Integrated Oncology Aachen Bonn Cologne Dusseldorf, 50937 Cologne, Germany
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany.
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, 50937 Cologne, Germany; Medical Faculty University of Cologne, 50937 Cologne, Germany.
| |
Collapse
|
2
|
Deboever N, Jones CM, Yamashita K, Ajani JA, Hofstetter WL. Advances in diagnosis and management of cancer of the esophagus. BMJ 2024; 385:e074962. [PMID: 38830686 DOI: 10.1136/bmj-2023-074962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/05/2024]
Abstract
Esophageal cancer is the seventh most common malignancy worldwide, with over 470 000 new cases diagnosed each year. Two distinct histological subtypes predominate, and should be considered biologically separate disease entities.1 These subtypes are esophageal adenocarcinoma (EAC) and esophageal squamous cell carcinoma (ESCC). Outcomes remain poor regardless of subtype, with most patients presenting with late stage disease.2 Novel strategies to improve early detection of the respective precursor lesions, squamous dysplasia, and Barrett's esophagus offer the potential to improve outcomes. The introduction of a limited number of biologic agents, as well as immune checkpoint inhibitors, is resulting in improvements in the systemic treatment of locally advanced and metastatic esophageal cancer. These developments, coupled with improvements in minimally invasive surgical and endoscopic treatment approaches, as well as adaptive and precision radiotherapy technologies, offer the potential to improve outcomes still further. This review summarizes the latest advances in the diagnosis and management of esophageal cancer, and the developments in understanding of the biology of this disease.
Collapse
Affiliation(s)
- Nathaniel Deboever
- Department of Thoracic and Cardiovascular Surgery, MD Anderson Cancer Center, Houston, TX, USA
| | - Christopher M Jones
- Early Cancer Institute, Department of Oncology, University of Cambridge, Cambridge, UK
- Addenbrooke's Hospital, Cambridge University Hospitals NHS Foundation Trust, Cambridge, UK
| | - Kohei Yamashita
- Department of Gastrointestinal Medical Oncology, MD Anderson Cancer Center, Houston, TX, USA
| | - Jaffer A Ajani
- Department of Gastrointestinal Medical Oncology, MD Anderson Cancer Center, Houston, TX, USA
| | - Wayne L Hofstetter
- Department of Thoracic and Cardiovascular Surgery, MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
3
|
Seoni S, Shahini A, Meiburger KM, Marzola F, Rotunno G, Acharya UR, Molinari F, Salvi M. All you need is data preparation: A systematic review of image harmonization techniques in Multi-center/device studies for medical support systems. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108200. [PMID: 38677080 DOI: 10.1016/j.cmpb.2024.108200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2024] [Revised: 04/20/2024] [Accepted: 04/22/2024] [Indexed: 04/29/2024]
Abstract
BACKGROUND AND OBJECTIVES Artificial intelligence (AI) models trained on multi-centric and multi-device studies can provide more robust insights and research findings compared to single-center studies. However, variability in acquisition protocols and equipment can introduce inconsistencies that hamper the effective pooling of multi-source datasets. This systematic review evaluates strategies for image harmonization, which standardizes appearances to enable reliable AI analysis of multi-source medical imaging. METHODS A literature search using PRISMA guidelines was conducted to identify relevant papers published between 2013 and 2023 analyzing multi-centric and multi-device medical imaging studies that utilized image harmonization approaches. RESULTS Common image harmonization techniques included grayscale normalization (improving classification accuracy by up to 24.42 %), resampling (increasing the percentage of robust radiomics features from 59.5 % to 89.25 %), and color normalization (enhancing AUC by up to 0.25 in external test sets). Initially, mathematical and statistical methods dominated, but machine and deep learning adoption has risen recently. Color imaging modalities like digital pathology and dermatology have remained prominent application areas, though harmonization efforts have expanded to diverse fields including radiology, nuclear medicine, and ultrasound imaging. In all the modalities covered by this review, image harmonization improved AI performance, with increasing of up to 24.42 % in classification accuracy and 47 % in segmentation Dice scores. CONCLUSIONS Continued progress in image harmonization represents a promising strategy for advancing healthcare by enabling large-scale, reliable analysis of integrated multi-source datasets using AI. Standardizing imaging data across clinical settings can help realize personalized, evidence-based care supported by data-driven technologies while mitigating biases associated with specific populations or acquisition protocols.
Collapse
Affiliation(s)
- Silvia Seoni
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Alen Shahini
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Kristen M Meiburger
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Francesco Marzola
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Giulia Rotunno
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia; Centre for Health Research, University of Southern Queensland, Australia
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - Massimo Salvi
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy.
| |
Collapse
|
4
|
Giammanco A, Bychkov A, Schallenberg S, Tsvetkov T, Fukuoka J, Pryalukhin A, Mairinger F, Seper A, Hulla W, Klein S, Quaas A, Büttner R, Tolkach Y. Fast-Track Development and Multi-Institutional Clinical Validation of an Artificial Intelligence Algorithm for Detection of Lymph Node Metastasis in Colorectal Cancer. Mod Pathol 2024; 37:100496. [PMID: 38636778 DOI: 10.1016/j.modpat.2024.100496] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Revised: 03/24/2024] [Accepted: 04/10/2024] [Indexed: 04/20/2024]
Abstract
Lymph node metastasis (LNM) detection can be automated using artificial intelligence (AI)-based diagnostic tools. Only limited studies have addressed this task for colorectal cancer (CRC). This study aimed to develop of a clinical-grade digital pathology tool for LNM detection in CRC using the original fast-track framework. The training cohort included 432 slides from one department. A segmentation algorithm detecting 8 relevant tissue classes was trained. The test cohorts consisted of materials from 5 pathology departments digitized by 4 different scanning systems. A high-quality, large training data set was generated within 7 days and a minimal amount of annotation work using fast-track principles. The AI tool showed very high accuracy for LNM detection in all cohorts, with sensitivity, negative predictive value, and specificity ranges of 0.980 to 1.000, 0.997 to 1.000, and 0.913 to 0.990, correspondingly. Only 5 of 14,460 analyzed test slides with tumor cells over all cohorts were classified as false negative (3/5 representing clusters of tumor cells in lymphatic vessels). A clinical-grade tool was trained in a short time using fast-track development principles and validated using the largest international, multi-institutional, multiscanner cohort of cases to date, showing very high precision for LNM detection in CRC. We are releasing a part of the test data sets to facilitate academic research.
Collapse
Affiliation(s)
- Avri Giammanco
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Andrey Bychkov
- Department of Pathology, Kameda Medical Center, Kamogawa, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki, Japan
| | | | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Junya Fukuoka
- Department of Pathology, Kameda Medical Center, Kamogawa, Japan; Department of Pathology Informatics, Nagasaki University, Nagasaki, Japan
| | - Alexey Pryalukhin
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria
| | - Fabian Mairinger
- Institute of Pathology, University Hospital Essen, Essen, Germany
| | - Alexander Seper
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria; Danube Private University, Wien, Austria
| | - Wolfgang Hulla
- Institute of Pathology, Wiener Neustadt State Hospital, Wiener Neustadt, Austria
| | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Büttner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|
5
|
Schmidt T, Fuchs HF, Thomas MN, Müller DT, Lukomski L, Scholz M, Bruns CJ. [Tailored surgery in the treatment of gastroesophageal cancer]. CHIRURGIE (HEIDELBERG, GERMANY) 2024; 95:261-267. [PMID: 38411664 DOI: 10.1007/s00104-024-02056-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 02/07/2024] [Indexed: 02/28/2024]
Abstract
The surgical options and particularly perioperative treatment, have significantly advanced in the case of gastroesophageal cancer. This progress enables a 5-year survival rate of nearly 50% to be achieved through curative multimodal treatment concepts for locally advanced cancer. Therefore, in tumor boards and surgical case discussions the question increasingly arises regarding the type of treatment that provides optimal oncological and functional outcomes for individual patients with pre-existing diseases. It is therefore essential to carefully assess whether organ-preserving treatment might also be considered in the future or in what way minimally invasive or robotic surgery can offer advantages. Simultaneously, the boundaries of surgical and oncological treatment are currently being shifted in order to enable curative forms of treatment for patients with pre-existing conditions or those with oligometastatic diseases. With the integration of artificial intelligence into decision-making processes, new possibilities for information processing are increasingly becoming available to incorporate even more data into making decisions in the future.
Collapse
Affiliation(s)
- Thomas Schmidt
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland.
| | - Hans F Fuchs
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| | - Michael N Thomas
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| | - Dolores T Müller
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| | - Leandra Lukomski
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| | - Matthias Scholz
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| | - Christiane J Bruns
- Klinik und Poliklinik für Allgemein‑, Viszeral‑, Tumor- und Transplantationschirurgie, Uniklinik Köln, Kerpenerstr. 62, 50937, Köln, Deutschland
| |
Collapse
|
6
|
Naseri S, Shukla S, Hiwale KM, Jagtap MM, Gadkari P, Gupta K, Deshmukh M, Sagar S. From Pixels to Prognosis: A Narrative Review on Artificial Intelligence's Pioneering Role in Colorectal Carcinoma Histopathology. Cureus 2024; 16:e59171. [PMID: 38807833 PMCID: PMC11129955 DOI: 10.7759/cureus.59171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2024] [Accepted: 04/27/2024] [Indexed: 05/30/2024] Open
Abstract
Colorectal carcinoma, a prevalent and deadly malignancy, necessitates precise histopathological assessment for effective diagnosis and prognosis. Artificial intelligence (AI) emerges as a transformative force in this realm, offering innovative solutions to enhance traditional histopathological methods. This narrative review explores AI's pioneering role in colorectal carcinoma histopathology, encompassing its evolution, techniques, and advancements. AI algorithms, notably machine learning and deep learning, have revolutionized image analysis, facilitating accurate diagnosis and prognosis prediction. Furthermore, AI-driven histopathological analysis unveils potential biomarkers and therapeutic targets, heralding personalized treatment approaches. Despite its promise, challenges persist, including data quality, interpretability, and integration. Collaborative efforts among researchers, clinicians, and AI developers are imperative to surmount these hurdles and realize AI's full potential in colorectal carcinoma care. This review underscores AI's transformative impact and implications for future oncology research, clinical practice, and interdisciplinary collaboration.
Collapse
Affiliation(s)
- Suhit Naseri
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| | - Samarth Shukla
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| | - K M Hiwale
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| | - Miheer M Jagtap
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| | - Pravin Gadkari
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| | - Kartik Gupta
- Radiation Oncology, Delhi State Cancer Institute, Delhi, IND
| | - Mamta Deshmukh
- Pathology, Indian Institute of Medical Sciences and Research, Jalna, IND
| | - Shakti Sagar
- Pathology, Jawaharlal Nehru Medical College, Datta Meghe Institute of Higher Education & Research, Wardha, IND
| |
Collapse
|
7
|
Chen RJ, Ding T, Lu MY, Williamson DFK, Jaume G, Song AH, Chen B, Zhang A, Shao D, Shaban M, Williams M, Oldenburg L, Weishaupt LL, Wang JJ, Vaidya A, Le LP, Gerber G, Sahai S, Williams W, Mahmood F. Towards a general-purpose foundation model for computational pathology. Nat Med 2024; 30:850-862. [PMID: 38504018 PMCID: PMC11403354 DOI: 10.1038/s41591-024-02857-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Accepted: 02/05/2024] [Indexed: 03/21/2024]
Abstract
Quantitative evaluation of tissue images is crucial for computational pathology (CPath) tasks, requiring the objective characterization of histopathological entities from whole-slide images (WSIs). The high resolution of WSIs and the variability of morphological features present significant challenges, complicating the large-scale annotation of data for high-performance applications. To address this challenge, current efforts have proposed the use of pretrained image encoders through transfer learning from natural image datasets or self-supervised learning on publicly available histopathology datasets, but have not been extensively developed and evaluated across diverse tissue types at scale. We introduce UNI, a general-purpose self-supervised model for pathology, pretrained using more than 100 million images from over 100,000 diagnostic H&E-stained WSIs (>77 TB of data) across 20 major tissue types. The model was evaluated on 34 representative CPath tasks of varying diagnostic difficulty. In addition to outperforming previous state-of-the-art models, we demonstrate new modeling capabilities in CPath such as resolution-agnostic tissue classification, slide classification using few-shot class prototypes, and disease subtyping generalization in classifying up to 108 cancer types in the OncoTree classification system. UNI advances unsupervised representation learning at scale in CPath in terms of both pretraining data and downstream evaluation, enabling data-efficient artificial intelligence models that can generalize and transfer to a wide range of diagnostically challenging tasks and clinical workflows in anatomic pathology.
Collapse
Affiliation(s)
- Richard J Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Tong Ding
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA
| | - Ming Y Lu
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
| | - Guillaume Jaume
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Andrew H Song
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Andrew Zhang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Daniel Shao
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Muhammad Shaban
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Mane Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Lukas Oldenburg
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Luca L Weishaupt
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Judy J Wang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Anurag Vaidya
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Long Phi Le
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Health Sciences and Technology, Harvard-MIT, Cambridge, MA, USA
| | - Georg Gerber
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Sharifa Sahai
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Systems Biology, Harvard University, Cambridge, MA, USA
| | - Walt Williams
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
- Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA.
- Cancer Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
- Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
8
|
Yin R, Chen H, Tao T, Zhang K, Yang G, Shi F, Jiang Y, Gui J. Expanding from unilateral to bilateral: A robust deep learning-based approach for predicting radiographic osteoarthritis progression. Osteoarthritis Cartilage 2024; 32:338-347. [PMID: 38113994 DOI: 10.1016/j.joca.2023.11.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Revised: 10/31/2023] [Accepted: 11/29/2023] [Indexed: 12/21/2023]
Abstract
OBJECTIVE To develop and validate a deep learning (DL) model for predicting osteoarthritis (OA) progression based on bilateral knee joint views. METHODS In this retrospective study, knee joints from bilateral posteroanterior knee radiographs of participants in the Osteoarthritis Initiative were analyzed. At baseline, participants were divided into testing set 1 and development set according to the different enrolled sites. The development set was further divided into a training set and a validation set in an 8:2 ratio for model development. At 48-month follow-up, eligible patients were formed testing set 2. The Bilateral Knee Neural Network (BikNet) was developed using bilateral views, with the knee to be predicted as the main view and the contralateral knee as the auxiliary view. DenseNet and ResNext were also trained and compared as the unilateral model. Two reader tests were conducted to evaluate the model's value in predicting incident OA. RESULTS Totally 3583 participants were evaluated. The BikNet we proposed outperformed ResNext and DenseNet (all area under the curve [AUC] < 0.71, P < 0.001) with AUC values of 0.761 and 0.745 in testing sets 1 and 2, respectively. With assistance of the BikNet increased clinicians' sensitivity (from 28.1-63.2% to 42.1-68.4%) and specificity (from 57.4-83.4% to 64.1-87.5%) of incident OA prediction and improved inter-observer reliability. CONCLUSION The DL model, constructed based on bilateral knee views, holds promise for enhancing the assessment of OA and demonstrates greater robustness during subsequent follow-up evaluations as compared with unilateral models. BikNet represents a potential tool or imaging biomarker for predicting OA progression.
Collapse
Affiliation(s)
- Rui Yin
- Nanjing Medical University, Nanjing, China; Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing, China.
| | - Hao Chen
- School of Computer Science, University of Birmingham, Birmingham, UK.
| | - Tianqi Tao
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing, China.
| | - Kaibin Zhang
- Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing, China.
| | - Guangxu Yang
- Department of Orthopedic Surgery, Nanjing Pukou Hospital, Nanjing, China.
| | - Fajian Shi
- Department of Orthopedic Surgery, Nanjing Pukou Hospital, Nanjing, China.
| | - Yiqiu Jiang
- Nanjing Medical University, Nanjing, China; Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing, China.
| | - Jianchao Gui
- Nanjing Medical University, Nanjing, China; Department of Sports Medicine and Joint Surgery, Nanjing First Hospital, Nanjing, China.
| |
Collapse
|
9
|
Yun D, Yang HL, Kwon S, Lee SR, Kim K, Kim K, Lee HC, Jung CW, Kim YS, Han SS. Automatic segmentation of atrial fibrillation and flutter in single-lead electrocardiograms by self-supervised learning and Transformer architecture. J Am Med Inform Assoc 2023; 31:79-88. [PMID: 37949101 PMCID: PMC10746317 DOI: 10.1093/jamia/ocad219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 10/20/2023] [Indexed: 11/12/2023] Open
Abstract
OBJECTIVES Automatic detection of atrial fibrillation and flutter (AF/AFL) is a significant concern in preventing stroke and mitigating hemodynamic instability. Herein, we developed a Transformer-based deep learning model for AF/AFL segmentation in single-lead electrocardiograms (ECGs) by self-supervised learning with masked signal modeling (MSM). MATERIALS AND METHODS We retrieved data from 11 open-source databases on PhysioNet; 7 of these databases included labeled ECGs, while the other 4 were without labels. Each database contained ECG recordings with durations of ≥30 s. A total of 24 intradialytic ECGs with paroxysmal AF/AFL during 4 h of hemodialysis sessions at Seoul National University Hospital were used for external validation. The model was pretrained by predicting masked areas of ECG signals and fine-tuned by predicting AF/AFL areas. Cross-database validation was used for evaluation, and the intersection over union (IOU) was used as a main performance metric in external database validation. RESULTS In the 7 labeled databases, the areas marked as AF/AFL constituted 41.1% of the total ECG signals, ranging from 0.19% to 51.31%. In the evaluation per ECG segment, the model achieved IOU values of 0.9254 and 0.9477 for AF/AFL segmentation and other segmentation tasks, respectively. When applied to intradialytic ECGs with paroxysmal AF/AFL, the IOUs for the segmentation of AF/AFL and non-AF/AFL were 0.9896 and 0.9650, respectively. Model performance by different training procedure indicated that pretraining with MSM and the application of an appropriate masking ratio both contributed to the model performance. It also showed higher IOUs of AF/AFL labels than in previous studies when training and test databases were matched. CONCLUSION The present model with self-supervised learning by MSM performs robustly in segmenting AF/AFL.
Collapse
Affiliation(s)
- Donghwan Yun
- Division of Nephrology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Department of Biomedical Sciences, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Hyun-Lim Yang
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Biomedical Research Institute, Seoul National University Hospital, Seoul, Republic of Korea
| | - Soonil Kwon
- Division of Cardiology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - So-Ryoung Lee
- Division of Cardiology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kyungju Kim
- Division of Nephrology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Kwangsoo Kim
- Transdisciplinary Department of Medicine & Advanced Technology, Seoul National University Hospital, Seoul, Republic of Korea
| | - Hyung-Chul Lee
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Chul-Woo Jung
- Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Yon Su Kim
- Division of Nephrology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
- Department of Biomedical Sciences, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Seung Seok Han
- Division of Nephrology, Department of Internal Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
10
|
Griem J, Eich ML, Schallenberg S, Pryalukhin A, Bychkov A, Fukuoka J, Zayats V, Hulla W, Munkhdelger J, Seper A, Tsvetkov T, Mukhopadhyay A, Sanner A, Stieber J, Fuchs M, Babendererde N, Schömig-Markiefka B, Klein S, Buettner R, Quaas A, Tolkach Y. Artificial Intelligence-Based Tool for Tumor Detection and Quantitative Tissue Analysis in Colorectal Specimens. Mod Pathol 2023; 36:100327. [PMID: 37683932 DOI: 10.1016/j.modpat.2023.100327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 08/11/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
Digital pathology adoption allows for applying computational algorithms to routine pathology tasks. Our study aimed to develop a clinical-grade artificial intelligence (AI) tool for precise multiclass tissue segmentation in colorectal specimens (resections and biopsies) and clinically validate the tool for tumor detection in biopsy specimens. The training data set included 241 precisely manually annotated whole-slide images (WSIs) from multiple institutions. The algorithm was trained for semantic segmentation of 11 tissue classes with an additional module for biopsy WSI classification. Six case cohorts from 5 pathology departments (4 countries) were used for formal and clinical validation, digitized by 4 different scanning systems. The developed algorithm showed high precision of segmentation of different tissue classes in colorectal specimens with composite multiclass Dice score of up to 0.895 and pixel-wise tumor detection specificity and sensitivity of up to 0.958 and 0.987, respectively. In the clinical validation study on multiple external cohorts, the AI tool reached sensitivity of 1.0 and specificity of up to 0.969 for tumor detection in biopsy WSI. The AI tool analyzes most biopsy cases in less than 1 minute, allowing effective integration into clinical routine. We developed and extensively validated a highly accurate, clinical-grade tool for assistive diagnostic processing of colorectal specimens. This tool allows for quantitative deciphering of colorectal cancer tissue for development of prognostic and predictive biomarkers and personalization of oncologic care. This study is a foundation for a SemiCOL computational challenge. We open-source multiple manually annotated and weakly labeled test data sets, representing a significant contribution to the colorectal cancer computational pathology field.
Collapse
Affiliation(s)
- Johanna Griem
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Marie-Lisa Eich
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | - Alexey Pryalukhin
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | - Andrey Bychkov
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Junya Fukuoka
- Department of Pathology Informatics, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan; Department of Pathology, Kameda Medical Center, Kamogawa, Japan
| | - Vitaliy Zayats
- Laboratory for Medical Artificial Intelligence, The Resource Center for Universal Design and Rehabilitation Technologies (RCUD and RT), Moscow, Russia
| | - Wolfgang Hulla
- Institute of Pathology, State Hospital Wiener Neustadt, Wiener Neustadt, Austria
| | | | - Alexander Seper
- Danube Private University, Medical Faculty, Krems-Stein, Austria
| | - Tsvetan Tsvetkov
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | | | | | | | - Moritz Fuchs
- Technical University Darmstadt, Darmstadt, Germany
| | | | | | - Sebastian Klein
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Reinhard Buettner
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Alexander Quaas
- Institute of Pathology, University Hospital Cologne, Cologne, Germany
| | - Yuri Tolkach
- Institute of Pathology, University Hospital Cologne, Cologne, Germany.
| |
Collapse
|