1
|
van Eekelen L, Spronck J, Looijen-Salamon M, Vos S, Munari E, Girolami I, Eccher A, Acs B, Boyaci C, de Souza GS, Demirel-Andishmand M, Meesters LD, Zegers D, van der Woude L, Theelen W, van den Heuvel M, Grünberg K, van Ginneken B, van der Laak J, Ciompi F. Comparing deep learning and pathologist quantification of cell-level PD-L1 expression in non-small cell lung cancer whole-slide images. Sci Rep 2024; 14:7136. [PMID: 38531958 DOI: 10.1038/s41598-024-57067-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2023] [Accepted: 03/14/2024] [Indexed: 03/28/2024] Open
Abstract
Programmed death-ligand 1 (PD-L1) expression is currently used in the clinic to assess eligibility for immune-checkpoint inhibitors via the tumor proportion score (TPS), but its efficacy is limited by high interobserver variability. Multiple papers have presented systems for the automatic quantification of TPS, but none report on the task of determining cell-level PD-L1 expression and often reserve their evaluation to a single PD-L1 monoclonal antibody or clinical center. In this paper, we report on a deep learning algorithm for detecting PD-L1 negative and positive tumor cells at a cellular level and evaluate it on a cell-level reference standard established by six readers on a multi-centric, multi PD-L1 assay dataset. This reference standard also provides for the first time a benchmark for computer vision algorithms. In addition, in line with other papers, we also evaluate our algorithm at slide-level by measuring the agreement between the algorithm and six pathologists on TPS quantification. We find a moderately low interobserver agreement at cell-level level (mean reader-reader F1 score = 0.68) which our algorithm sits slightly under (mean reader-AI F1 score = 0.55), especially for cases from the clinical center not included in the training set. Despite this, we find good AI-pathologist agreement on quantifying TPS compared to the interobserver agreement (mean reader-reader Cohen's kappa = 0.54, 95% CI 0.26-0.81, mean reader-AI kappa = 0.49, 95% CI 0.27-0.72). In conclusion, our deep learning algorithm demonstrates promise in detecting PD-L1 expression at a cellular level and exhibits favorable agreement with pathologists in quantifying the tumor proportion score (TPS). We publicly release our models for use via the Grand-Challenge platform.
Collapse
Affiliation(s)
- Leander van Eekelen
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands.
| | - Joey Spronck
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Monika Looijen-Salamon
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Shoko Vos
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Ilaria Girolami
- Department of Pathology, Provincial Hospital of Bolzano (SABES-ASDAA), Bolzano-Bozen, Italy
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, Verona, Italy
| | - Balazs Acs
- Department of Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
| | - Ceren Boyaci
- Department of Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
| | - Gabriel Silva de Souza
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Muradije Demirel-Andishmand
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Luca Dulce Meesters
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Daan Zegers
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Lieke van der Woude
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Willemijn Theelen
- Department of Thoracic Oncology, Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Michel van den Heuvel
- Respiratory Diseases Department, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Katrien Grünberg
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Bram van Ginneken
- Department of Radiology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, P.O.Box 9101, 6500 HB, Nijmegen, The Netherlands
| |
Collapse
|
2
|
Chelebian E, Avenel C, Ciompi F, Wählby C. DEPICTER: Deep representation clustering for histology annotation. Comput Biol Med 2024; 170:108026. [PMID: 38308865 DOI: 10.1016/j.compbiomed.2024.108026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 01/24/2024] [Accepted: 01/24/2024] [Indexed: 02/05/2024]
Abstract
Automatic segmentation of histopathology whole-slide images (WSI) usually involves supervised training of deep learning models with pixel-level labels to classify each pixel of the WSI into tissue regions such as benign or cancerous. However, fully supervised segmentation requires large-scale data manually annotated by experts, which can be expensive and time-consuming to obtain. Non-fully supervised methods, ranging from semi-supervised to unsupervised, have been proposed to address this issue and have been successful in WSI segmentation tasks. But these methods have mainly been focused on technical advancements in algorithmic performance rather than on the development of practical tools that could be used by pathologists or researchers in real-world scenarios. In contrast, we present DEPICTER (Deep rEPresentatIon ClusTERing), an interactive segmentation tool for histopathology annotation that produces a patch-wise dense segmentation map at WSI level. The interactive nature of DEPICTER leverages self- and semi-supervised learning approaches to allow the user to participate in the segmentation producing reliable results while reducing the workload. DEPICTER consists of three steps: first, a pretrained model is used to compute embeddings from image patches. Next, the user selects a number of benign and cancerous patches from the multi-resolution image. Finally, guided by the deep representations, label propagation is achieved using our novel seeded iterative clustering method or by directly interacting with the embedding space via feature space gating. We report both real-time interaction results with three pathologists and evaluate the performance on three public cancer classification dataset benchmarks through simulations. The code and demos of DEPICTER are publicly available at https://github.com/eduardchelebian/depicter.
Collapse
Affiliation(s)
- Eduard Chelebian
- Department of Information Technology and SciLifeLab, Uppsala University, Uppsala, Sweden.
| | - Chirstophe Avenel
- Department of Information Technology and SciLifeLab, Uppsala University, Uppsala, Sweden
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Carolina Wählby
- Department of Information Technology and SciLifeLab, Uppsala University, Uppsala, Sweden
| |
Collapse
|
3
|
Vos AM, Pijnenborg L, van Vliet S, Kodach LL, Ciompi F, van der Post RS, Simmer F, Nagtegaal ID. Biological background of colorectal polyps and carcinomas with heterotopic ossification: A national study and literature review. Hum Pathol 2024; 145:34-41. [PMID: 38367815 DOI: 10.1016/j.humpath.2024.02.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 02/12/2024] [Accepted: 02/14/2024] [Indexed: 02/19/2024]
Abstract
The biological mechanisms and potential clinical impact of heterotopic ossification (HO) in colorectal neoplasms are not fully understood. This study investigates the clinicopathological characteristics of colorectal neoplasms associated with HO and examines the potential role of the bone morphogenetic protein (BMP) pathway in development of HO. An artificial intelligence (AI) based classification of colorectal cancers (CRC) exhibiting HO and their association with consensus molecular subtypes (CMS) is performed. The study included 77 cases via the Dutch nationwide Pathology databank. Immunohistochemistry for BMP2, SMAD4, and Osterix was performed. An AI algorithm assessed the tumour-stroma ratio to approximate the CMS. A literature search yielded 96 case reports, which were analysed and compared with our cases for clinicopathological parameters. HO was more frequently observed in our cohort in traditional serrated adenomas (25%), tubulovillous adenomas (25%) and juvenile polyps (25%), while in the literature it was most often seen in juvenile polyps (38.2%) and inflammatory polyps (29.4%). In both cohorts, carcinomas were mostly conventional (>60%) followed by mucinous and serrated adenocarcinomas. Higher expression of BMP2, SMAD4, and Osterix was observed in tumour and/or stromal cells directly surrounding bone, indicating activation of the BMP pathway. The tumour-stroma analysis appointed >50% of the cases to the mesenchymal subtype (CMS4) (59%). HO has a predilection for serrated and juvenile/inflammatory polyps, mucinous and serrated adenocarcinomas. BMP signalling is activated and seems to play a role in formation of HO in colorectal neoplasms. In line with TGFβ/BMP pathway activation associated with CMS4 CRC, HO seems associated with CMS4.
Collapse
Affiliation(s)
- Anne-Marie Vos
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands.
| | - Lieke Pijnenborg
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Shannon van Vliet
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Liudmila L Kodach
- Department of Molecular Pathology, Dutch Cancer Institute, Amsterdam, the Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Rachel S van der Post
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Femke Simmer
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Iris D Nagtegaal
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| |
Collapse
|
4
|
Jiao Y, van der Laak J, Albarqouni S, Li Z, Tan T, Bhalerao A, Cheng S, Ma J, Pocock J, Pluim JPW, Koohbanani NA, Bashir RMS, Raza SEA, Liu S, Graham S, Wetstein S, Khurram SA, Liu X, Rajpoot N, Veta M, Ciompi F. LYSTO: The Lymphocyte Assessment Hackathon and Benchmark Dataset. IEEE J Biomed Health Inform 2024; 28:1161-1172. [PMID: 37878422 DOI: 10.1109/jbhi.2023.3327489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2023]
Abstract
We introduce LYSTO, the Lymphocyte Assessment Hackathon, which was held in conjunction with the MICCAI 2019 Conference in Shenzhen (China). The competition required participants to automatically assess the number of lymphocytes, in particular T-cells, in images of colon, breast, and prostate cancer stained with CD3 and CD8 immunohistochemistry. Differently from other challenges setup in medical image analysis, LYSTO participants were solely given a few hours to address this problem. In this paper, we describe the goal and the multi-phase organization of the hackathon; we describe the proposed methods and the on-site results. Additionally, we present post-competition results where we show how the presented methods perform on an independent set of lung cancer slides, which was not part of the initial competition, as well as a comparison on lymphocyte assessment between presented methods and a panel of pathologists. We show that some of the participants were capable to achieve pathologist-level performance at lymphocyte assessment. After the hackathon, LYSTO was left as a lightweight plug-and-play benchmark dataset on grand-challenge website, together with an automatic evaluation platform.
Collapse
|
5
|
Jahangir CA, Page DB, Broeckx G, Gonzalez CA, Burke C, Murphy C, Reis-Filho JS, Ly A, Harms PW, Gupta RR, Vieth M, Hida AI, Kahila M, Kos Z, van Diest PJ, Verbandt S, Thagaard J, Khiroya R, Abduljabbar K, Acosta Haab G, Acs B, Adams S, Almeida JS, Alvarado-Cabrero I, Azmoudeh-Ardalan F, Badve S, Baharun NB, Bellolio ER, Bheemaraju V, Blenman KR, Botinelly Mendonça Fujimoto L, Burgues O, Chardas A, Cheang MCU, Ciompi F, Cooper LA, Coosemans A, Corredor G, Dantas Portela FL, Deman F, Demaria S, Dudgeon SN, Elghazawy M, Fernandez-Martín C, Fineberg S, Fox SB, Giltnane JM, Gnjatic S, Gonzalez-Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hart SN, Hartman J, Hewitt S, Horlings HM, Husain Z, Irshad S, Janssen EA, Kataoka TR, Kawaguchi K, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Akturk G, Scott E, Kovács A, Laenkholm AV, Lang-Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Madabhushi A, Maley SK, Manur Narasimhamurthy V, Marks DK, McDonald ES, Mehrotra R, Michiels S, Kharidehal D, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault-Llorca F, Perera RD, Pinard CJ, Pinto-Cardenas JC, Pruneri G, Pusztai L, Rajpoot NM, Rapoport BL, Rau TT, Ribeiro JM, Rimm D, Vincent-Salomon A, Saltz J, Sayed S, Hytopoulos E, Mahon S, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, Verghese GE, Viale G, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Bartlett J, Loibl S, Denkert C, Savas P, Loi S, Specht Stovgaard E, Salgado R, Gallagher WM, Rahman A. Image-based multiplex immune profiling of cancer tissues: translational implications. A report of the International Immuno-oncology Biomarker Working Group on Breast Cancer. J Pathol 2024; 262:271-288. [PMID: 38230434 DOI: 10.1002/path.6238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 11/17/2023] [Indexed: 01/18/2024]
Abstract
Recent advances in the field of immuno-oncology have brought transformative changes in the management of cancer patients. The immune profile of tumours has been found to have key value in predicting disease prognosis and treatment response in various cancers. Multiplex immunohistochemistry and immunofluorescence have emerged as potent tools for the simultaneous detection of multiple protein biomarkers in a single tissue section, thereby expanding opportunities for molecular and immune profiling while preserving tissue samples. By establishing the phenotype of individual tumour cells when distributed within a mixed cell population, the identification of clinically relevant biomarkers with high-throughput multiplex immunophenotyping of tumour samples has great potential to guide appropriate treatment choices. Moreover, the emergence of novel multi-marker imaging approaches can now provide unprecedented insights into the tumour microenvironment, including the potential interplay between various cell types. However, there are significant challenges to widespread integration of these technologies in daily research and clinical practice. This review addresses the challenges and potential solutions within a structured framework of action from a regulatory and clinical trial perspective. New developments within the field of immunophenotyping using multiplexed tissue imaging platforms and associated digital pathology are also described, with a specific focus on translational implications across different subtypes of cancer. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - David B Page
- Earle A Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA
| | - Glenn Broeckx
- Department of Pathology PA2, GZA-ZNA Hospitals, Antwerp, Belgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of Medicine, Antwerp University, Antwerp, Belgium
| | - Claudia A Gonzalez
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Caoimbhe Burke
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Clodagh Murphy
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Jorge S Reis-Filho
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Amy Ly
- Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - Paul W Harms
- Departments of Pathology and Dermatology, University of Michigan, Ann Arbor, MI, USA
| | - Rajarsi R Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Akira I Hida
- Department of Pathology, Matsuyama Shimin Hospital, Matsuyama, Japan
| | - Mohamed Kahila
- Department of Pathology, Yale School of Medicine, New Haven, CT, USA
| | - Zuzana Kos
- Department of Pathology and Laboratory Medicine, University of British Columbia, BC Cancer, Vancouver, British Columbia, Canada
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
- Johns Hopkins Oncology Center, Baltimore, MD, USA
| | - Sara Verbandt
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Jeppe Thagaard
- Technical University of Denmark, Kgs. Lyngby, Denmark
- Visiopharm A/S, Hørsholm, Denmark
| | - Reena Khiroya
- Department of Cellular Pathology, University College Hospital, London, UK
| | - Khalid Abduljabbar
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK
| | | | - Balazs Acs
- Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
- Department of Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
| | - Sylvia Adams
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
- Department of Medicine, NYU Grossman School of Medicine, Manhattan, NY, USA
| | - Jonas S Almeida
- Division of Cancer Epidemiology and Genetics (DCEG), National Cancer Institute (NCI), Rockville, MD, USA
| | | | | | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Emory University School of Medicine, Emory University Winship Cancer Institute, Atlanta, GA, USA
| | | | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de Medicina, Universidad de La Frontera, Temuco, Chile
| | | | - Kim Rm Blenman
- Department of Internal Medicine Section of Medical Oncology and Yale Cancer Center, Yale School of Medicine, New Haven, CT, USA
- Department of Computer Science, Yale School of Engineering and Applied Science, New Haven, CT, USA
| | | | - Octavio Burgues
- Pathology Department, Hospital Cliníco Universitario de Valencia/Incliva, Valencia, Spain
| | - Alexandros Chardas
- Department of Pathobiology & Population Sciences, The Royal Veterinary College, London, UK
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR-CTSU, Division of Clinical Studies, The Institute of Cancer Research, London, UK
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Lee Ad Cooper
- Department of Pathology, Northwestern Feinberg School of Medicine, Chicago, IL, USA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and Immunotherapy, KU Leuven, Leuven, Belgium
| | - Germán Corredor
- Biomedical Engineering Department, Emory University, Atlanta, GA, USA
| | | | - Frederik Deman
- Department of Pathology PA2, GZA-ZNA Hospitals, Antwerp, Belgium
| | - Sandra Demaria
- Department of Radiation Oncology, Weill Cornell Medical College, New York, NY, USA
- Department of Pathology, Weill Cornell Medicine, New York, NY, USA
| | - Sarah N Dudgeon
- Computational Biology and Bioinformatics, Yale University, New Haven, CT, USA
| | - Mahmoud Elghazawy
- University of Surrey, Guildford, UK
- Ain Shams University, Cairo, Egypt
| | - Claudio Fernandez-Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain
| | - Susan Fineberg
- Montefiore Medical Center and the Albert Einstein College of Medicine, New York, NY, USA
| | - Stephen B Fox
- Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria, Australia
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/Onc, and Pathology, Tisch Cancer Institute - Precision Immunology Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, Faculty of Life Sciences and Medicine, School of Cancer & Pharmaceutical Sciences, King's College London, London, UK
- The Breast Cancer Now Research Unit, Faculty of Life Sciences and Medicine, School of Cancer and Pharmaceutical Sciences, King's College London, London, UK
| | - Niels Halama
- Department of Translational Immunotherapy, German Cancer Research Center, Heidelberg, Germany
| | | | | | - Steven N Hart
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Johan Hartman
- Department of Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
- Department of Oncology-Pathology, Karolinska Institutet, Stockholm, Sweden
| | - Stephen Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Hugo M Horlings
- Division of Pathology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | - Sheeba Irshad
- King's College London & Guys & St Thomas NHS Trust, London, UK
| | - Emiel Am Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | | | - Kosuke Kawaguchi
- Department of Breast Surgery, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Andrey I Khramtsov
- Department of Pathology and Laboratory Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | - Pawan Kirtani
- Histopathology, Aakash Healthcare Super Speciality Hospital, New Delhi, India
| | - Liudmila L Kodach
- Department of Pathology, Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product Development, F. Hoffmann-La Roche AG, Basel, Switzerland
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co., Inc., Kenilworth, NJ, USA
| | - Ely Scott
- Translational Medicine, Bristol Myers Squibb, Princeton, NJ, USA
| | - Anikó Kovács
- Department of Clinical Pathology, Sahlgrenska University Hospital, Gothenburg, Sweden
- Institute of Biomedicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Anne-Vibeke Laenkholm
- Department of Surgical Pathology, Zealand University Hospital, Roskilde, Denmark
- Department of Surgical Pathology, University of Copenhagen, Copenhagen, Denmark
| | - Corinna Lang-Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Denis Larsimont
- Institut Jules Bordet, Université Libre de Bruxelles, Brussels, Belgium
| | - Jochen K Lennerz
- Center for Integrated Diagnostics, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM U900, Paris, France
| | - Xiaoxian Li
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, Pathology, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Sai K Maley
- NRG Oncology/NSABP Foundation, Pittsburgh, PA, USA
| | | | - Douglas K Marks
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
| | - Elizabeth S McDonald
- Breast Cancer Translational Research Group, University of Pennsylvania, Philadelphia, PA, USA
| | - Ravi Mehrotra
- Indian Cancer Genomic Atlas, Pune, India
- Centre for Health, Innovation and Policy Foundation, Noida, India
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, Inserm, University Paris-Saclay, Ligue Contre le Cancer labeled Team, Villejuif, France
| | - Durga Kharidehal
- Department of Pathology, Narayana Medical College and Hospital, Nellore, India
| | - Fayyaz Ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCL and Cellular Pathology Department, UCLH, London, UK
| | - Shamim Mushtaq
- Department of Biochemistry, Ziauddin University, Karachi, Pakistan
| | - Hussain Nighat
- Pathology and Laboratory Medicine, All India Institute of Medical Sciences, Raipur, India
| | - Thomas Papathomas
- Institute of Metabolism and Systems Research, University of Birmingham, Birmingham, UK
- Department of Clinical Pathology, Drammen Sykehus, Vestre Viken HF, Drammen, Norway
| | - Frederique Penault-Llorca
- Service de Pathologie et Biopathologie, Centre Jean PERRIN, INSERM U1240 Imagerie Moléculaire et Stratégies Théranostiques (IMoST), Université Clermont Auvergne, Clermont-Ferrand, France
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure Engineering, University of Melbourne, Melbourne, Victoria, Australia
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
| | - Christopher J Pinard
- Radiogenomics Laboratory, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
- Department of Clinical Studies, Ontario Veterinary College, University of Guelph, Guelph, Ontario, Canada
- Department of Oncology, Lakeshore Animal Health Partners, Mississauga, Ontario, Canada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE-AI), University of Guelph, Guelph, Ontario, Canada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
- Faculty of Medicine and Surgery, University of Milan, Milan, Italy
| | - Lajos Pusztai
- Yale Cancer Center, Yale University, New Haven, CT, USA
- Department of Medical Oncology, Yale School of Medicine, Yale University, New Haven, CT, USA
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of Rosebank, Johannesburg, South Africa
- Department of Immunology, Faculty of Health Sciences, University of Pretoria, Pretoria, South Africa
| | - Tilman T Rau
- Institute of Pathology, University Hospital Düsseldorf and Heinrich-Heine-University, Düsseldorf, Germany
| | | | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
- Department of Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Anne Vincent-Salomon
- Department of Diagnostic and Theranostic Medicine, Institut Curie, University Paris-Sciences et Lettres, Paris, France
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook Medicine, New York, NY, USA
| | - Shahin Sayed
- Department of Pathology, Aga Khan University, Nairobi, Kenya
| | - Evangelos Hytopoulos
- Department of Pathology, Aga Khan University, Nairobi, Kenya
- iRhythm Technologies Inc., San Francisco, CA, USA
| | - Sarah Mahon
- Mater Misericordiae University Hospital, Dublin, Ireland
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.-C. Heuson, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
- Medical Oncology Department, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Albrecht Stenzinger
- Institute of Pathology, University Hospital Heidelberg, Centers for Personalized Medicine (ZPM), Heidelberg, Germany
| | | | - Daniel Sur
- Department of Medical Oncology, University of Medicine and Pharmacy "Iuliu Hatieganu", Cluj-Napoca, Romania
| | - Fraser Symmans
- University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Jonas Teuwen
- AI for Oncology Lab, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Trine Tramm
- Department of Pathology, Institute of Clinical Medicine, Aarhus University Hospital, Aarhus, Denmark
| | - William T Tran
- Department of Radiation Oncology, University of Toronto and Sunnybrook Health Sciences Centre, Toronto, Canada
| | - Jeroen van der Laak
- Head of Integrative Genomics Analysis in Clinical Trials, ICR-CTSU, Division of Clinical Studies, The Institute of Cancer Research, London, UK
| | - Gregory E Verghese
- Cancer Bioinformatics, Faculty of Life Sciences and Medicine, School of Cancer & Pharmaceutical Sciences, King's College London, London, UK
- The Breast Cancer Now Research Unit, Faculty of Life Sciences and Medicine, School of Cancer and Pharmaceutical Sciences, King's College London, London, UK
| | - Giuseppe Viale
- Department of Pathology, European Institute of Oncology & University of Milan, Milan, Italy
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Thomas Walter
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM U900, Paris, France
| | | | - Hannah Y Wen
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer Center, Shanghai, PR China
| | - Yinyin Yuan
- Department of Translational Molecular Pathology, Division of Pathology and Laboratory Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Sibylle Loibl
- Department of Medicine and Research, German Breast Group, Neu-Isenburg, Germany
| | - Carsten Denkert
- Institut für Pathologie, Philipps-Universität Marburg und Universitätsklinikum Marburg, Marburg, Germany
| | - Peter Savas
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- The Sir Peter MacCallum Department of Medical Oncology, University of Melbourne, Melbourne, Victoria, Australia
| | - Sherene Loi
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
- Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, Victoria, Australia
| | | | - Roberto Salgado
- Department of Pathology PA2, GZA-ZNA Hospitals, Antwerp, Belgium
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, Victoria, Australia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway Institute, University College Dublin, Dublin, Ireland
| | - Arman Rahman
- UCD School of Medicine, UCD Conway Institute, University College Dublin, Dublin, Ireland
| |
Collapse
|
6
|
Vermorgen S, Gelton T, Bult P, Kusters-Vandevelde HVN, Hausnerová J, Van de Vijver K, Davidson B, Stefansson IM, Kooreman LFS, Qerimi A, Huvila J, Gilks B, Shahi M, Zomer S, Bartosch C, Pijnenborg JMA, Bulten J, Ciompi F, Simons M. Endometrial Pipelle Biopsy Computer-Aided Diagnosis: A Feasibility Study. Mod Pathol 2024; 37:100417. [PMID: 38154654 DOI: 10.1016/j.modpat.2023.100417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Revised: 12/02/2023] [Accepted: 12/19/2023] [Indexed: 12/30/2023]
Abstract
Endometrial biopsies are important in the diagnostic workup of women who present with abnormal uterine bleeding or hereditary risk of endometrial cancer. In general, approximately 10% of all endometrial biopsies demonstrate endometrial (pre)malignancy that requires specific treatment. As the diagnostic evaluation of mostly benign cases results in a substantial workload for pathologists, artificial intelligence (AI)-assisted preselection of biopsies could optimize the workflow. This study aimed to assess the feasibility of AI-assisted diagnosis for endometrial biopsies (endometrial Pipelle biopsy computer-aided diagnosis), trained on daily-practice whole-slide images instead of highly selected images. Endometrial biopsies were classified into 6 clinically relevant categories defined as follows: nonrepresentative, normal, nonneoplastic, hyperplasia without atypia, hyperplasia with atypia, and malignant. The agreement among 15 pathologists, within these classifications, was evaluated in 91 endometrial biopsies. Next, an algorithm (trained on a total of 2819 endometrial biopsies) rated the same 91 cases, and we compared its performance using the pathologist's classification as the reference standard. The interrater reliability among pathologists was moderate with a mean Cohen's kappa of 0.51, whereas for a binary classification into benign vs (pre)malignant, the agreement was substantial with a mean Cohen's kappa of 0.66. The AI algorithm performed slightly worse for the 6 categories with a moderate Cohen's kappa of 0.43 but was comparable for the binary classification with a substantial Cohen's kappa of 0.65. AI-assisted diagnosis of endometrial biopsies was demonstrated to be feasible in discriminating between benign and (pre)malignant endometrial tissues, even when trained on unselected cases. Endometrial premalignancies remain challenging for both pathologists and AI algorithms. Future steps to improve reliability of the diagnosis are needed to achieve a more refined AI-assisted diagnostic solution for endometrial biopsies that covers both premalignant and malignant diagnoses.
Collapse
Affiliation(s)
- Sanne Vermorgen
- Department of Pathology, Radboudumc, Nijmegen, the Netherlands
| | - Thijs Gelton
- Department of Pathology, Radboudumc, Nijmegen, the Netherlands
| | - Peter Bult
- Department of Pathology, Radboudumc, Nijmegen, the Netherlands
| | | | - Jitka Hausnerová
- Department of Pathology, University Hospital Brno, Brno, Czech Republic
| | | | - Ben Davidson
- Department of Pathology, Oslo University Hospital, Norwegian Radium Hospital, Oslo, Norway; University of Oslo, Faculty of Medicine, Institute of Clinical Medicine, Oslo, Norway
| | - Ingunn Marie Stefansson
- Centre for Cancer Biomarkers CCBIO, Department of Clinical Medicine, Section for Pathology, University of Bergen, Bergen, Norway; Department of Pathology, Haukeland University Hospital Bergen, Bergen, Norway
| | - Loes F S Kooreman
- Department of Pathology, GROW School for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, the Netherlands
| | - Adelina Qerimi
- Department of Pathology, ViraTherapeutics GmbH, Innsbruck, Austria
| | - Jutta Huvila
- Department of Pathology, University of Turku, Turku University Hospital, Turku, Finland
| | - Blake Gilks
- Department of Pathology, University of British Columbia, Vancouver, Canada
| | - Maryam Shahi
- Department of Pathology, Mayo Clinic, Rochester, Minnesota
| | - Saskia Zomer
- Department of Pathology, Canisius-Wilhelmina Hospital, Nijmegen, the Netherlands
| | - Carla Bartosch
- Department of Pathology, Portuguese Oncology Institute Lisbon, Lisbon, Portugal
| | | | - Johan Bulten
- Department of Pathology, Radboudumc, Nijmegen, the Netherlands
| | | | - Michiel Simons
- Department of Pathology, Radboudumc, Nijmegen, the Netherlands.
| |
Collapse
|
7
|
van Rijthoven M, Obahor S, Pagliarulo F, van den Broek M, Schraml P, Moch H, van der Laak J, Ciompi F, Silina K. Multi-resolution deep learning characterizes tertiary lymphoid structures and their prognostic relevance in solid tumors. Commun Med (Lond) 2024; 4:5. [PMID: 38182879 PMCID: PMC10770129 DOI: 10.1038/s43856-023-00421-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Accepted: 11/30/2023] [Indexed: 01/07/2024] Open
Abstract
BACKGROUND Tertiary lymphoid structures (TLSs) are dense accumulations of lymphocytes in inflamed peripheral tissues, including cancer, and are associated with improved survival and response to immunotherapy in various solid tumors. Histological TLS quantification has been proposed as a novel predictive and prognostic biomarker, but lack of standardized methods of TLS characterization hampers assessment of TLS densities across different patients, diseases, and clinical centers. METHODS We introduce an approach based on HookNet-TLS, a multi-resolution deep learning model, for automated and unbiased TLS quantification and identification of germinal centers in routine hematoxylin and eosin stained digital pathology slides. We developed HookNet-TLS using n = 1019 manually annotated TCGA slides from clear cell renal cell carcinoma, muscle-invasive bladder cancer, and lung squamous cell carcinoma. RESULTS Here we show that HookNet-TLS automates TLS quantification across multiple cancer types achieving human-level performance and demonstrates prognostic associations similar to visual assessment. CONCLUSIONS HookNet-TLS has the potential to be used as a tool for objective quantification of TLS in routine H&E digital pathology slides. We make HookNet-TLS publicly available to promote its use in research.
Collapse
Affiliation(s)
- Mart van Rijthoven
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands.
| | - Simon Obahor
- Institute of Experimental Immunology, University of Zurich, Zurich, Switzerland
| | - Fabio Pagliarulo
- Institute of Experimental Immunology, University of Zurich, Zurich, Switzerland
| | | | - Peter Schraml
- Department of Pathology and Molecular Pathology, University Hospital Zurich, Zurich, Switzerland
| | - Holger Moch
- Department of Pathology and Molecular Pathology, University Hospital Zurich, Zurich, Switzerland
- Faculty of Medicine, University of Zurich, Zurich, Switzerland
| | - Jeroen van der Laak
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands
| | - Francesco Ciompi
- Pathology Department, Radboud University Medical Center, Nijmegen, Netherlands
| | - Karina Silina
- Institute of Pharmaceutical Sciences, Swiss Federal Institute of Technology, Zurich, Switzerland
| |
Collapse
|
8
|
Brouwer NPM, Khan A, Bokhorst JM, Ayatollahi F, Hay J, Ciompi F, Simmer F, Hugen N, de Wilt JHW, Berger MD, Lugli A, Zlobec I, Edwards J, Nagtegaal ID. The Complexity of Shapes: How the Circularity of Tumor Nodules Affects Prognosis in Colorectal Cancer. Mod Pathol 2024; 37:100376. [PMID: 37926423 DOI: 10.1016/j.modpat.2023.100376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 10/04/2023] [Accepted: 10/30/2023] [Indexed: 11/07/2023]
Abstract
The current stratification of tumor nodules in colorectal cancer (CRC) staging is subjective and leads to high interobserver variability. In this study, the objective assessment of the shape of lymph node metastases (LNMs), extranodal extension (ENE), and tumor deposits (TDs) was correlated with outcomes. A test cohort and a validation cohort were included from 2 different institutions. The test cohort consisted of 190 cases of stage III CRC. Slides with LNMs and TDs were annotated and processed using a segmentation algorithm to determine their shape. The complexity ratio was calculated for every shape and correlated with outcomes. A cohort of 160 stage III CRC cases was used to validate findings. TDs showed significantly more complex shapes than LNMs with ENE, which were more complex than LNMs without ENE (P < .001). In the test cohort, patients with the highest sum of complexity ratios had significantly lower disease-free survival (P < .01). When only the nodule with the highest complexity was considered, this effect was even stronger (P < .001). This maximum complexity ratio per patient was identified as an independent prognostic factor in the multivariate analysis (hazard ratio, 2.47; P < .05). The trends in the validation cohort confirmed the results. More complex nodules in stage III CRC were correlated with significantly worse disease-free survival, even if only based on the most complex nodule. These results suggest that more complex nodules reflect more invasive tumor biology. As most of the more complex nodules were diagnosed as TDs, we suggest providing a more prominent role for TDs in the nodal stage and include an objective complexity measure in their definition.
Collapse
Affiliation(s)
- Nelleke P M Brouwer
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Amjad Khan
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - John-Melle Bokhorst
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Fazael Ayatollahi
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jennifer Hay
- Glasgow Tissue Research Facility, University of Glasgow, Queen Elizabeth University Hospital, Glasgow, United Kingdom
| | - Francesco Ciompi
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Femke Simmer
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Niek Hugen
- Department of Surgery, Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Johannes H W de Wilt
- Department of Surgery, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Martin D Berger
- Department of Medical Oncology, Inselspital, University Hospital of Bern, University of Bern, Bern, Switzerland
| | - Alessandro Lugli
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Inti Zlobec
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Joanne Edwards
- School of Cancer Sciences, University of Glasgow, Glasgow, United Kingdom
| | - Iris D Nagtegaal
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
9
|
Aswolinskiy W, Munari E, Horlings HM, Mulder L, Bogina G, Sanders J, Liu YH, van den Belt-Dusebout AW, Tessier L, Balkenhol M, Stegeman M, Hoven J, Wesseling J, van der Laak J, Lips EH, Ciompi F. PROACTING: predicting pathological complete response to neoadjuvant chemotherapy in breast cancer from routine diagnostic histopathology biopsies with deep learning. Breast Cancer Res 2023; 25:142. [PMID: 37957667 PMCID: PMC10644597 DOI: 10.1186/s13058-023-01726-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 10/02/2023] [Indexed: 11/15/2023] Open
Abstract
BACKGROUND Invasive breast cancer patients are increasingly being treated with neoadjuvant chemotherapy; however, only a fraction of the patients respond to it completely. To prevent overtreatment, there is an urgent need for biomarkers to predict treatment response before administering the therapy. METHODS In this retrospective study, we developed hypothesis-driven interpretable biomarkers based on deep learning, to predict the pathological complete response (pCR, i.e., the absence of tumor cells in the surgical resection specimens) to neoadjuvant chemotherapy solely using digital pathology H&E images of pre-treatment breast biopsies. Our approach consists of two steps: First, we use deep learning to characterize aspects of the tumor micro-environment by detecting mitoses and segmenting tissue into several morphology compartments including tumor, lymphocytes and stroma. Second, we derive computational biomarkers from the segmentation and detection output to encode slide-level relationships of components of the tumor microenvironment, such as tumor and mitoses, stroma, and tumor infiltrating lymphocytes (TILs). RESULTS We developed and evaluated our method on slides from n = 721 patients from three European medical centers with triple-negative and Luminal B breast cancers and performed external independent validation on n = 126 patients from a public dataset. We report the predictive value of the investigated biomarkers for predicting pCR with areas under the receiver operating characteristic curve between 0.66 and 0.88 across the tested cohorts. CONCLUSION The proposed computational biomarkers predict pCR, but will require more evaluation and finetuning for clinical application. Our results further corroborate the potential role of deep learning to automate TILs quantification, and their predictive value in breast cancer neoadjuvant treatment planning, along with automated mitoses quantification. We made our method publicly available to extract segmentation-based biomarkers for research purposes.
Collapse
Affiliation(s)
- Witali Aswolinskiy
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Enrico Munari
- Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Hugo M Horlings
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Lennart Mulder
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| | - Joyce Sanders
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Yat-Hee Liu
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | - Leslie Tessier
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
- Center for Integrated Oncology (Institut du cancer de l'Ouest), Angers, France
| | - Maschenka Balkenhol
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Michelle Stegeman
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeffrey Hoven
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jelle Wesseling
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
- Leiden University Medical Center, Leiden, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Esther H Lips
- The Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| |
Collapse
|
10
|
Bokhorst JM, Ciompi F, Öztürk SK, Oguz Erdogan AS, Vieth M, Dawson H, Kirsch R, Simmer F, Sheahan K, Lugli A, Zlobec I, van der Laak J, Nagtegaal ID. Fully Automated Tumor Bud Assessment in Hematoxylin and Eosin-Stained Whole Slide Images of Colorectal Cancer. Mod Pathol 2023; 36:100233. [PMID: 37257824 DOI: 10.1016/j.modpat.2023.100233] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Revised: 04/25/2023] [Accepted: 05/20/2023] [Indexed: 06/02/2023]
Abstract
Tumor budding (TB), the presence of single cells or small clusters of up to 4 tumor cells at the invasive front of colorectal cancer (CRC), is a proven risk factor for adverse outcomes. International definitions are necessary to reduce interobserver variability. According to the current international guidelines, hotspots at the invasive front should be counted in hematoxylin and eosin (H&E)-stained slides. This is time-consuming and prone to interobserver variability; therefore, there is a need for computer-aided diagnosis solutions. In this study, we report an artificial intelligence-based method for detecting TB in H&E-stained whole slide images. We propose a fully automated pipeline to identify the tumor border, detect tumor buds, characterize them based on the number of tumor cells, and produce a TB density map to identify the TB hotspot. The method outputs the TB count in the hotspot as a computational biomarker. We show that the proposed automated TB detection workflow performs on par with a panel of 5 pathologists at detecting tumor buds and that the hotspot-based TB count is an independent prognosticator in both the univariate and the multivariate analysis, validated on a cohort of n = 981 patients with CRC. Computer-aided detection of tumor buds based on deep learning can perform on par with expert pathologists for the detection and quantification of tumor buds in H&E-stained CRC histopathology slides, strongly facilitating the introduction of budding as an independent prognosticator in clinical routine and clinical trials.
Collapse
Affiliation(s)
- John-Melle Bokhorst
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Sonay Kus Öztürk
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Michael Vieth
- Klinikum of Pathology, Bayreuth University, Bayreuth, Germany
| | - Heather Dawson
- Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| | - Richard Kirsch
- University of Toronto, Mount Sinai Hospital, Toronto, Canada
| | - Femke Simmer
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Kieran Sheahan
- Department of Pathology, St Vincent's Hospital, Dublin, Ireland
| | | | - Inti Zlobec
- Klinikum of Pathology, Bayreuth University, Bayreuth, Germany
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | - Iris D Nagtegaal
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
11
|
Menotti L, Silvello G, Atzori M, Boytcheva S, Ciompi F, Di Nunzio GM, Fraggetta F, Giachelle F, Irrera O, Marchesin S, Marini N, Müller H, Primov T. Modelling digital health data: The ExaMode ontology for computational pathology. J Pathol Inform 2023; 14:100332. [PMID: 37705689 PMCID: PMC10495665 DOI: 10.1016/j.jpi.2023.100332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 07/14/2023] [Accepted: 08/16/2023] [Indexed: 09/15/2023] Open
Abstract
Computational pathology can significantly benefit from ontologies to standardize the employed nomenclature and help with knowledge extraction processes for high-quality annotated image datasets. The end goal is to reach a shared model for digital pathology to overcome data variability and integration problems. Indeed, data annotation in such a specific domain is still an unsolved challenge and datasets cannot be steadily reused in diverse contexts due to heterogeneity issues of the adopted labels, multilingualism, and different clinical practices. Material and methods This paper presents the ExaMode ontology, modeling the histopathology process by considering 3 key cancer diseases (colon, cervical, and lung tumors) and celiac disease. The ExaMode ontology has been designed bottom-up in an iterative fashion with continuous feedback and validation from pathologists and clinicians. The ontology is organized into 5 semantic areas that defines an ontological template to model any disease of interest in histopathology. Results The ExaMode ontology is currently being used as a common semantic layer in: (i) an entity linking tool for the automatic annotation of medical records; (ii) a web-based collaborative annotation tool for histopathology text reports; and (iii) a software platform for building holistic solutions integrating multimodal histopathology data. Discussion The ontology ExaMode is a key means to store data in a graph database according to the RDF data model. The creation of an RDF dataset can help develop more accurate algorithms for image analysis, especially in the field of digital pathology. This approach allows for seamless data integration and a unified query access point, from which we can extract relevant clinical insights about the considered diseases using SPARQL queries.
Collapse
Affiliation(s)
- Laura Menotti
- Department of Information Engineering, University of Padua, Padova, Italy
| | - Gianmaria Silvello
- Department of Information Engineering, University of Padua, Padova, Italy
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland, Delémont, Switzerland
- Department of Neuroscience, University of Padua, Padova, Italy
| | | | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | | | - Fabio Giachelle
- Department of Information Engineering, University of Padua, Padova, Italy
| | - Ornella Irrera
- Department of Information Engineering, University of Padua, Padova, Italy
| | - Stefano Marchesin
- Department of Information Engineering, University of Padua, Padova, Italy
| | - Niccolò Marini
- Information Systems Institute, University of Applied Sciences Western Switzerland, Delémont, Switzerland
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland, Delémont, Switzerland
| | | |
Collapse
|
12
|
Thagaard J, Broeckx G, Page DB, Jahangir CA, Verbandt S, Kos Z, Gupta R, Khiroya R, Abduljabbar K, Acosta Haab G, Acs B, Akturk G, Almeida JS, Alvarado‐Cabrero I, Amgad M, Azmoudeh‐Ardalan F, Badve S, Baharun NB, Balslev E, Bellolio ER, Bheemaraju V, Blenman KRM, Botinelly Mendonça Fujimoto L, Bouchmaa N, Burgues O, Chardas A, Chon U Cheang M, Ciompi F, Cooper LAD, Coosemans A, Corredor G, Dahl AB, Dantas Portela FL, Deman F, Demaria S, Doré Hansen J, Dudgeon SN, Ebstrup T, Elghazawy M, Fernandez‐Martín C, Fox SB, Gallagher WM, Giltnane JM, Gnjatic S, Gonzalez‐Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hart SN, Hartman J, Hauberg S, Hewitt S, Hida AI, Horlings HM, Husain Z, Hytopoulos E, Irshad S, Janssen EAM, Kahila M, Kataoka TR, Kawaguchi K, Kharidehal D, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Kovács A, Laenkholm A, Lang‐Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Ly A, Madabhushi A, Maley SK, Manur Narasimhamurthy V, Marks DK, McDonald ES, Mehrotra R, Michiels S, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault‐Llorca F, Perera RD, Pinard CJ, Pinto‐Cardenas JC, Pruneri G, Pusztai L, Rahman A, Rajpoot NM, Rapoport BL, Rau TT, Reis‐Filho JS, Ribeiro JM, Rimm D, Roslind A, Vincent‐Salomon A, Salto‐Tellez M, Saltz J, Sayed S, Scott E, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Fineberg S, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, van Diest PJ, Verghese GE, Viale G, Vieth M, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Zin RM, Adams S, Bartlett J, Loibl S, Denkert C, Savas P, Loi S, Salgado R, Specht Stovgaard E. Pitfalls in machine learning-based assessment of tumor-infiltrating lymphocytes in breast cancer: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J Pathol 2023; 260:498-513. [PMID: 37608772 PMCID: PMC10518802 DOI: 10.1002/path.6155] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 06/07/2023] [Indexed: 08/24/2023]
Abstract
The clinical significance of the tumor-immune interaction in breast cancer is now established, and tumor-infiltrating lymphocytes (TILs) have emerged as predictive and prognostic biomarkers for patients with triple-negative (estrogen receptor, progesterone receptor, and HER2-negative) breast cancer and HER2-positive breast cancer. How computational assessments of TILs might complement manual TIL assessment in trial and daily practices is currently debated. Recent efforts to use machine learning (ML) to automatically evaluate TILs have shown promising results. We review state-of-the-art approaches and identify pitfalls and challenges of automated TIL evaluation by studying the root cause of ML discordances in comparison to manual TIL quantification. We categorize our findings into four main topics: (1) technical slide issues, (2) ML and image analysis aspects, (3) data challenges, and (4) validation issues. The main reason for discordant assessments is the inclusion of false-positive areas or cells identified by performance on certain tissue patterns or design choices in the computational implementation. To aid the adoption of ML for TIL assessment, we provide an in-depth discussion of ML and image analysis, including validation issues that need to be considered before reliable computational reporting of TILs can be incorporated into the trial and routine clinical management of patients with triple-negative breast cancer. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- Jeppe Thagaard
- Technical University of DenmarkKongens LyngbyDenmark
- Visiopharm A/SHørsholmDenmark
| | - Glenn Broeckx
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of MedicineAntwerp UniversityAntwerpBelgium
| | - David B Page
- Earle A Chiles Research InstituteProvidence Cancer InstitutePortlandORUSA
| | - Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | - Sara Verbandt
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Zuzana Kos
- Department of Pathology and Laboratory MedicineBC Cancer Vancouver Centre, University of British ColumbiaVancouverBritish ColumbiaCanada
| | - Rajarsi Gupta
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Reena Khiroya
- Department of Cellular PathologyUniversity College Hospital LondonLondonUK
| | | | | | - Balazs Acs
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co IncRahwayNJUSA
| | - Jonas S Almeida
- Division of Cancer Epidemiology and Genetics (DCEG)National Cancer Institute (NCI)Rockville, MDUSA
| | | | - Mohamed Amgad
- Department of PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | | | - Sunil Badve
- Department of Pathology and Laboratory Medicine, Emory University School of MedicineEmory University Winship Cancer InstituteAtlantaGAUSA
| | | | - Eva Balslev
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de MedicinaUniversidad de La FronteraTemucoChile
| | | | - Kim RM Blenman
- Department of Internal Medicine Section of Medical Oncology and Yale Cancer CenterYale School of MedicineNew HavenCTUSA
- Department of Computer ScienceYale School of Engineering and Applied ScienceNew HavenCTUSA
| | | | - Najat Bouchmaa
- Institute of Biological Sciences, Faculty of Medical SciencesMohammed VI Polytechnic University (UM6P)Ben‐GuerirMorocco
| | - Octavio Burgues
- Pathology DepartmentHospital Cliníco Universitario de Valencia/InclivaValenciaSpain
| | - Alexandros Chardas
- Department of Pathobiology & Population SciencesThe Royal Veterinary CollegeLondonUK
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR‐CTSU, Division of Clinical StudiesThe Institute of Cancer ResearchLondonUK
| | - Francesco Ciompi
- Radboud University Medical CenterDepartment of PathologyNijmegenThe Netherlands
| | - Lee AD Cooper
- Department of PathologyNorthwestern Feinberg School of MedicineChicagoILUSA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and ImmunotherapyKU LeuvenLeuvenBelgium
| | - Germán Corredor
- Biomedical Engineering DepartmentEmory UniversityAtlantaGAUSA
| | - Anders B Dahl
- Technical University of DenmarkKongens LyngbyDenmark
| | | | | | - Sandra Demaria
- Department of Radiation OncologyWeill Cornell MedicineNew YorkNYUSA
- Department of Pathology and Laboratory MedicineWeill Cornell MedicineNew YorkNYUSA
| | | | - Sarah N Dudgeon
- Conputational Biology and BioinformaticsYale UniversityNew HavenCTUSA
| | | | | | - Claudio Fernandez‐Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN‐techUniversitat Politècnica de ValènciaValenciaSpain
| | - Stephen B Fox
- Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/Onc, and Pathology, Tisch Cancer Institute – Precision Immunology InstituteIcahn School of Medicine at Mount SinaiNew YorkNYUSA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Niels Halama
- Department of Translational ImmunotherapyGerman Cancer Research CenterHeidelbergGermany
| | - Matthew G Hanna
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | | | - Steven N Hart
- Department of Laboratory Medicine and PathologyMayo ClinicRochester, MNUSA
| | - Johan Hartman
- Department of Oncology and PathologyKarolinska InstitutetStockholmSweden
- Department of Clinical Pathology and Cancer DiagnosticsKarolinska University HospitalStockholmSweden
| | - Søren Hauberg
- Technical University of DenmarkKongens LyngbyDenmark
| | - Stephen Hewitt
- Laboratory of Pathology, Center for Cancer Research, National Cancer InstituteNational Institutes of HealthBethesdaMDUSA
| | - Akira I Hida
- Department of PathologyMatsuyama Shimin HospitalMatsuyamaJapan
| | - Hugo M Horlings
- Division of PathologyNetherlands Cancer Institute (NKI)AmsterdamThe Netherlands
| | | | | | - Sheeba Irshad
- King's College London & Guy's & St Thomas’ NHS TrustLondonUK
| | - Emiel AM Janssen
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | | | | | - Kosuke Kawaguchi
- Department of Breast SurgeryKyoto University Graduate School of MedicineKyotoJapan
| | | | - Andrey I Khramtsov
- Department of Pathology and Laboratory MedicineAnn & Robert H. Lurie Children's Hospital of ChicagoChicagoILUSA
| | - Umay Kiraz
- Department of PathologyStavanger University HospitalStavangerNorway
- Department of Chemistry, Bioscience and Environmental TechnologyUniversity of StavangerStavangerNorway
| | - Pawan Kirtani
- Department of HistopathologyAakash Healthcare Super Speciality HospitalNew DelhiIndia
| | - Liudmila L Kodach
- Department of PathologyNetherlands Cancer Institute – Antoni van Leeuwenhoek HospitalAmsterdamThe Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product DevelopmentF. Hoffmann‐La Roche AGBaselSwitzerland
| | - Anikó Kovács
- Department of Clinical PathologySahlgrenska University HospitalGothenburgSweden
- Institute of Biomedicine, Sahlgrenska AcademyUniversity of GothenburgGothenburgSweden
| | - Anne‐Vibeke Laenkholm
- Department of Surgical PathologyZealand University HospitalRoskildeDenmark
- Department of Surgical PathologyUniversity of CopenhagenCopenhagenDenmark
| | - Corinna Lang‐Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Denis Larsimont
- Institut Jules BordetUniversité Libre de BruxellesBrusselsBelgium
| | - Jochen K Lennerz
- Center for Integrated DiagnosticsMassachusetts General Hospital/Harvard Medical SchoolBostonMAUSA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | - Xiaoxian Li
- Department of Pathology and Laboratory MedicineEmory UniversityAtlantaGAUSA
| | - Amy Ly
- Department of PathologyMassachusetts General HospitalBostonMAUSA
| | - Anant Madabhushi
- Department of Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, PathologyGeorgia Institute of Technology and Emory UniversityAtlantaGAUSA
| | - Sai K Maley
- NRG Oncology/NSABP FoundationPittsburghPAUSA
| | | | | | - Elizabeth S McDonald
- Breast Cancer Translational Research GroupUniversity of PennsylvaniaPhiladelphiaPAUSA
| | - Ravi Mehrotra
- Indian Cancer Genomic AtlasPuneIndia
- Centre for Health, Innovation and Policy FoundationNoidaIndia
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, InsermUniversity Paris‐Saclay, Ligue Contre le Cancer labeled TeamVillejuifFrance
| | - Fayyaz ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and PathologyUniversity of WashingtonSeattle, WAUSA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCL and Cellular Pathology DepartmentUCLHLondonUK
| | - Shamim Mushtaq
- Department of BiochemistryZiauddin UniversityKarachiPakistan
| | - Hussain Nighat
- Pathology and Laboratory MedicineAll India Institute of Medical sciencesRaipurIndia
| | - Thomas Papathomas
- Institute of Metabolism and Systems ResearchUniversity of BirminghamBirminghamUK
- Department of Clinical PathologyDrammen Sykehus, Vestre Viken HFDrammenNorway
| | - Frederique Penault‐Llorca
- Centre Jean Perrin, Université Clermont Auvergne, INSERM, U1240 Imagerie Moléculaire et Stratégies ThéranostiquesClermont FerrandFrance
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure EngineeringUniversity of MelbourneMelbourneVictoriaAustralia
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Christopher J Pinard
- Radiogenomics LaboratorySunnybrook Health Sciences CentreTorontoOntarioCanada
- Department of Clinical Studies, Ontario Veterinary CollegeUniversity of GuelphGuelphOntarioCanada
- Department of OncologyLakeshore Animal Health PartnersMississaugaOntarioCanada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE‐AI)University of GuelphGuelphOntarioCanada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory MedicineFondazione IRCCS Istituto Nazionale dei TumoriMilanItaly
- Faculty of Medicine and SurgeryUniversity of MilanMilanItaly
| | - Lajos Pusztai
- Yale Cancer CenterYale UniversityNew HavenCTUSA
- Department of Medical Oncology, Yale School of MedicineYale UniversityNew HavenCTUSA
| | - Arman Rahman
- UCD School of Biomolecular and Biomedical Science, UCD Conway InstituteUniversity College DublinDublinIreland
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of RosebankJohannesburgSouth Africa
- Department of Immunology, Faculty of Health SciencesUniversity of PretoriaPretoriaSouth Africa
| | - Tilman T Rau
- Institute of PathologyUniversity Hospital Düsseldorf and Heinrich‐Heine‐University DüsseldorfDüsseldorfGermany
| | - Jorge S Reis‐Filho
- Department of Pathology and Laboratory MedicineMemorial Sloan Kettering Cancer CenterNew YorkNYUSA
| | - Joana M Ribeiro
- Département de Médecine OncologiqueGustave RoussyVillejuifFrance
| | - David Rimm
- Department of PathologyYale University School of MedicineNew HavenCTUSA
- Department of MedicineYale University School of MedicineNew HavenCTUSA
| | - Anne Roslind
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
| | - Anne Vincent‐Salomon
- Department of Diagnostic and Theranostic Medicine, Institut CurieUniversity Paris‐Sciences et LettresParisFrance
| | - Manuel Salto‐Tellez
- Integrated Pathology UnitThe Institute of Cancer ResearchLondonUK
- Precision Medicine CentreQueen's University BelfastBelfastUK
| | - Joel Saltz
- Department of Biomedical InformaticsStony Brook UniversityStony BrookNYUSA
| | - Shahin Sayed
- Department of PathologyAga Khan UniversityNairobiKenya
| | - Ely Scott
- Translational PathologyTranslational Sciences and Diagnostics/Translational Medicine/R&D, Bristol Myers SquibbPrincetonNJUSA
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast PathologyNorthwestern University Feinberg School of MedicineChicagoILUSA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.‐C. Heuson, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
- Medical Oncology Department, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB)Université Libre de Bruxelles (ULB)BrusselsBelgium
| | - Albrecht Stenzinger
- Institute of PathologyUniversity Hospital HeidelbergHeidelbergGermany
- Centers for Personalized Medicine (ZPM)HeidelbergGermany
| | | | - Daniel Sur
- Department of Medical OncologyUniversity of Medicine and Pharmacy “Iuliu Hatieganu”Cluj‐NapocaRomania
| | - Susan Fineberg
- Montefiore Medical CenterBronxNYUSA
- Albert Einstein College of MedicineBronxNYUSA
| | - Fraser Symmans
- University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of OncologyKU LeuvenLeuvenBelgium
| | - Jonas Teuwen
- AI for Oncology Lab, The Netherlands Cancer InstituteAmsterdamThe Netherlands
| | | | - Trine Tramm
- Department of PathologyAarhus University HospitalAarhusDenmark
- Institute of Clinical MedicineAarhus UniversityAarhusDenmark
| | - William T Tran
- Department of Radiation OncologyUniversity of Toronto and Sunnybrook Health Sciences CentreTorontoOntarioCanada
| | - Jeroen van der Laak
- Department of PathologyRadboud University Medical CenterNijmegenThe Netherlands
| | - Paul J van Diest
- Department of PathologyUniversity Medical Center UtrechtThe Netherlands
- Johns Hopkins Oncology CenterBaltimoreMDUSA
| | - Gregory E Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
- The Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and MedicineKing's College LondonLondonUK
| | - Giuseppe Viale
- Department of PathologyEuropean Institute of OncologyMilanItaly
- Department of PathologyUniversity of MilanMilanItaly
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbHFriedrich‐Alexander‐University Erlangen‐NurembergBayreuthGermany
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer ScienceUniversity of WarwickCoventryUK
| | - Thomas Walter
- Centre for Computational Biology (CBIO)Mines Paris, PSL UniversityParisFrance
- Institut CuriePSL UniversityParisFrance
- INSERMParisFrance
| | | | - Hannah Y Wen
- Department of PathologyMemorial Sloan Kettering Cancer CenterNew YorkUSA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer CenterShanghaiPR China
| | - Yinyin Yuan
- Department of Translational Molecular Pathology, Division of Pathology and Laboratory MedicineThe University of Texas MD Anderson Cancer CenterHoustonTXUSA
| | - Reena Md Zin
- Department of Pathology, Faculty of MedicineUniversiti Kebangsaan MalaysiaKuala LumpurMalaysia
| | - Sylvia Adams
- Perlmutter Cancer CenterNYU Langone HealthNew YorkNYUSA
- Department of MedicineNYU Grossman School of MedicineManhattanNYUSA
| | | | - Sibylle Loibl
- Department of Medicine and ResearchGerman Breast GroupNeu‐IsenburgGermany
| | - Carsten Denkert
- Institut für PathologiePhilipps‐Universität Marburg und Universitätsklinikum MarburgMarburgGermany
| | - Peter Savas
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Sherene Loi
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- The Sir Peter MacCallum Department of Medical OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Roberto Salgado
- Department of PathologyGZA‐ZNA HospitalsAntwerpBelgium
- Division of Cancer ResearchPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
| | - Elisabeth Specht Stovgaard
- Department of PathologyHerlev and Gentofte HospitalHerlevDenmark
- Department of Clinical MedicineUniversity of CopenhagenCopenhagenDenmark
| |
Collapse
|
13
|
Page DB, Broeckx G, Jahangir CA, Verbandt S, Gupta RR, Thagaard J, Khiroya R, Kos Z, Abduljabbar K, Acosta Haab G, Acs B, Akturk G, Almeida JS, Alvarado-Cabrero I, Azmoudeh-Ardalan F, Badve S, Baharun NB, Bellolio ER, Bheemaraju V, Blenman KR, Botinelly Mendonça Fujimoto L, Bouchmaa N, Burgues O, Cheang MCU, Ciompi F, Cooper LA, Coosemans A, Corredor G, Dantas Portela FL, Deman F, Demaria S, Dudgeon SN, Elghazawy M, Ely S, Fernandez-Martín C, Fineberg S, Fox SB, Gallagher WM, Giltnane JM, Gnjatic S, Gonzalez-Ericsson PI, Grigoriadis A, Halama N, Hanna MG, Harbhajanka A, Hardas A, Hart SN, Hartman J, Hewitt S, Hida AI, Horlings HM, Husain Z, Hytopoulos E, Irshad S, Janssen EA, Kahila M, Kataoka TR, Kawaguchi K, Kharidehal D, Khramtsov AI, Kiraz U, Kirtani P, Kodach LL, Korski K, Kovács A, Laenkholm AV, Lang-Schwarz C, Larsimont D, Lennerz JK, Lerousseau M, Li X, Ly A, Madabhushi A, Maley SK, Manur Narasimhamurthy V, Marks DK, McDonald ES, Mehrotra R, Michiels S, Minhas FUAA, Mittal S, Moore DA, Mushtaq S, Nighat H, Papathomas T, Penault-Llorca F, Perera RD, Pinard CJ, Pinto-Cardenas JC, Pruneri G, Pusztai L, Rahman A, Rajpoot NM, Rapoport BL, Rau TT, Reis-Filho JS, Ribeiro JM, Rimm D, Vincent-Salomon A, Salto-Tellez M, Saltz J, Sayed S, Siziopikou KP, Sotiriou C, Stenzinger A, Sughayer MA, Sur D, Symmans F, Tanaka S, Taxter T, Tejpar S, Teuwen J, Thompson EA, Tramm T, Tran WT, van der Laak J, van Diest PJ, Verghese GE, Viale G, Vieth M, Wahab N, Walter T, Waumans Y, Wen HY, Yang W, Yuan Y, Adams S, Bartlett JMS, Loibl S, Denkert C, Savas P, Loi S, Salgado R, Specht Stovgaard E. Spatial analyses of immune cell infiltration in cancer: current methods and future directions: A report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J Pathol 2023; 260:514-532. [PMID: 37608771 DOI: 10.1002/path.6165] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 06/19/2023] [Indexed: 08/24/2023]
Abstract
Modern histologic imaging platforms coupled with machine learning methods have provided new opportunities to map the spatial distribution of immune cells in the tumor microenvironment. However, there exists no standardized method for describing or analyzing spatial immune cell data, and most reported spatial analyses are rudimentary. In this review, we provide an overview of two approaches for reporting and analyzing spatial data (raster versus vector-based). We then provide a compendium of spatial immune cell metrics that have been reported in the literature, summarizing prognostic associations in the context of a variety of cancers. We conclude by discussing two well-described clinical biomarkers, the breast cancer stromal tumor infiltrating lymphocytes score and the colon cancer Immunoscore, and describe investigative opportunities to improve clinical utility of these spatial biomarkers. © 2023 The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
- David B Page
- Earle A Chiles Research Institute, Providence Cancer Institute, Portland, OR, USA
| | - Glenn Broeckx
- Department of Pathology, GZA-ZNA Hospitals, Antwerp, Belgium
- Centre for Oncological Research (CORE), MIPPRO, Faculty of Medicine, Antwerp University, Antwerp, Belgium
| | - Chowdhury Arif Jahangir
- UCD School of Biomolecular and Biomedical Science, University College Dublin, Dublin, Ireland
| | - Sara Verbandt
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Rajarsi R Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, USA
| | - Jeppe Thagaard
- Technical University of Denmark, Kongens Lyngby, Denmark
- Visiopharm A/S, Hørsholm, Denmark
| | - Reena Khiroya
- Department of Cellular Pathology, University College Hospital, London, UK
| | - Zuzana Kos
- Department of Pathology and Laboratory Medicine, BC Cancer Vancouver Centre, University of British Columbia, Vancouver, BC, Canada
| | - Khalid Abduljabbar
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK
| | | | - Balazs Acs
- Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden
- Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
| | - Guray Akturk
- Translational Molecular Biomarkers, Merck & Co Inc, Kenilworth, NJ, USA
| | - Jonas S Almeida
- National Cancer Institute, Division of Cancer Epidemiology and Genetics (DCEG), Rockville, MD, USA
| | | | | | - Sunil Badve
- Pathology and Laboratory Medicine, Emory University School of Medicine, Emory University Winship Cancer Institute, Atlanta, GA, USA
| | | | - Enrique R Bellolio
- Departamento de Anatomía Patológica, Facultad de Medicina, Universidad de La Frontera, Temuco, Chile
| | | | - Kim Rm Blenman
- Internal Medicine Section of Medical Oncology and Yale Cancer Center, Yale School of Medicine, New Haven, CT, USA
- Computer Science, Yale School of Engineering and Applied Science, New Haven, CT, USA
| | | | - Najat Bouchmaa
- Institute of Biological Sciences, Faculty of Medical Sciences, Mohammed VI Polytechnic University (UM6P), Ben-Guerir, Morocco
| | - Octavio Burgues
- Pathology Department, Hospital Cliníco Universitario de Valencia/Incliva, Valencia, Spain
| | - Maggie Chon U Cheang
- Head of Integrative Genomics Analysis in Clinical Trials, ICR-CTSU, Division of Clinical Studies, Institute of Cancer Research, London, UK
| | - Francesco Ciompi
- Radboud University Medical Center, Department of Pathology, Nijmegen, The Netherlands
| | - Lee Ad Cooper
- Department of Pathology, Northwestern Feinberg School of Medicine, Chicago, IL, USA
| | - An Coosemans
- Department of Oncology, Laboratory of Tumor Immunology and Immunotherapy, KU Leuven, Leuven, Belgium
| | - Germán Corredor
- Biomedical Engineering Department, Emory University, Atlanta, GA, USA
| | | | - Frederik Deman
- Department of Pathology, GZA-ZNA Hospitals, Antwerp, Belgium
| | - Sandra Demaria
- Department of Radiation Oncology, Weill Cornell Medical College, New York, NY, USA
- Department of Pathology, Weill Cornell Medicine, New York, NY, USA
| | - Sarah N Dudgeon
- Conputational Biology and Bioinformatics, Yale University, New Haven, CT, USA
| | - Mahmoud Elghazawy
- University of Surrey, Guildford, UK
- Ain Shams University, Cairo, Egypt
| | - Scott Ely
- Translational Pathology, Translational Sciences and Diagnostics/Translational Medicine/R&D, Bristol Myers Squibb, Princeton, NJ, USA
| | - Claudio Fernandez-Martín
- Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, HUMAN-tech, Universitat Politècnica de València, Valencia, Spain
| | - Susan Fineberg
- Montefiore Medical Center and the Albert Einstein College of Medicine, New York, NY, USA
| | - Stephen B Fox
- Department of Pathology, Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, VIC, Australia
| | - William M Gallagher
- UCD School of Biomolecular and Biomedical Science, University College Dublin, Dublin, Ireland
| | | | - Sacha Gnjatic
- Department of Oncological Sciences, Medicine Hem/Onc, and Pathology, Tisch Cancer Institute - Precision Immunology Institute, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | - Anita Grigoriadis
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
- Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
| | - Niels Halama
- Translational Immunotherapy, German Cancer Research Center, Heidelberg, Germany
| | | | | | - Alexandros Hardas
- Pathobiology & Population Sciences, The Royal Veterinary College, London, UK
| | - Steven N Hart
- Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Johan Hartman
- Clinical Pathology and Cancer Diagnostics, Karolinska University Hospital, Stockholm, Sweden
- Department of Oncology-Pathology, Karolinska Institutet, Stockholm, Sweden
| | - Stephen Hewitt
- Department of Pathology, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Akira I Hida
- Department of Pathology, Matsuyama Shimin Hospital, Matsuyama, Japan
| | - Hugo M Horlings
- Division of Pathology, Netherlands Cancer Institute (NKI), Amsterdam, The Netherlands
| | | | | | - Sheeba Irshad
- King's College London & Guy's & St Thomas' NHS Trust, London, UK
| | - Emiel Am Janssen
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | - Mohamed Kahila
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | | | - Kosuke Kawaguchi
- Department of Breast Surgery, Kyoto University Graduate School of Medicine, Kyoto, Japan
| | - Durga Kharidehal
- Department of Pathology, Narayana Medical College, Nellore, India
| | - Andrey I Khramtsov
- Pathology and Laboratory Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, USA
| | - Umay Kiraz
- Department of Pathology, Stavanger University Hospital, Stavanger, Norway
- Department of Chemistry, Bioscience and Environmental Technology, University of Stavanger, Stavanger, Norway
| | - Pawan Kirtani
- Department of Histopathology, Aakash Healthcare Super Speciality Hospital, New Delhi, India
| | - Liudmila L Kodach
- Department of Pathology, Netherlands Cancer Institute - Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
| | - Konstanty Korski
- Data, Analytics and Imaging, Product Development, F.Hoffmann-La Roche AG, Basel, Switzerland
| | - Anikó Kovács
- Department of Clinical Pathology, Sahlgrenska University Hospital, Gothenburg, Sweden
- Institute of Biomedicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
| | - Anne-Vibeke Laenkholm
- Surgical Pathology, Zealand University Hospital, Roskilde, Denmark
- Surgical Pathology, University of Copenhagen, Copenhagen, Denmark
| | - Corinna Lang-Schwarz
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Denis Larsimont
- Institut Jules Bordet, Université Libre de Bruxelles, Brussels, Belgium
| | - Jochen K Lennerz
- Center for Integrated Diagnostics, Massachusetts General Hospital/Harvard Medical School, Boston, MA, USA
| | - Marvin Lerousseau
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM, U900, Paris, France
| | - Xiaoxian Li
- Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
| | - Amy Ly
- Department of Pathology, Massachusetts General Hospital, Boston, MA, USA
| | - Anant Madabhushi
- Biomedical Engineering, Radiology and Imaging Sciences, Biomedical Informatics, Pathology, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | - Sai K Maley
- NRG Oncology/NSABP Foundation, Pittsburgh, PA, USA
| | | | - Douglas K Marks
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
| | - Elizabeth S McDonald
- Breast Cancer Translational Research Group, University of Pennsylvania, Philadelphia, PA, USA
| | - Ravi Mehrotra
- Indian Cancer Genome Atlas, Pune, India
- Centre for Health, Innovation and Policy Foundation, Noida, India
| | - Stefan Michiels
- Office of Biostatistics and Epidemiology, Gustave Roussy, Oncostat U1018, Inserm, University Paris-Saclay, Ligue Contre le Cancer labeled Team, Villejuif, France
| | - Fayyaz Ul Amir Afsar Minhas
- Tissue Image Analytics Centre, Warwick Cancer Research Centre, PathLAKE Consortium, Department of Computer Science, University of Warwick, Coventry, UK
| | - Shachi Mittal
- Department of Chemical Engineering, Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA
| | - David A Moore
- CRUK Lung Cancer Centre of Excellence, UCLH, London, UK
| | - Shamim Mushtaq
- Department of Biochemistry, Ziauddin University, Karachi, Pakistan
| | - Hussain Nighat
- Pathology and Laboratory Medicine, All India Institute of Medical Sciences, Raipur, India
| | - Thomas Papathomas
- Institute of Metabolism and Systems Research, University of Birmingham, Birmingham, UK
- Department of Clinical Pathology, Drammen Sykehus, Vestre Viken HF, Drammen, Norway
| | - Frederique Penault-Llorca
- Centre Jean Perrin, INSERM U1240, Imagerie Moléculaire et Stratégies Théranostiques, Université Clermont Auvergne, Clermont-Ferrand, France
| | - Rashindrie D Perera
- School of Electrical, Mechanical and Infrastructure Engineering, University of Melbourne, Melbourne, VIC, Australia
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia
| | - Christopher J Pinard
- Radiogenomics Laboratory, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
- Department of Clinical Studies, Ontario Veterinary College, University of Guelph, Guelph, ON, Canada
- Department of Oncology, Lakeshore Animal Health Partners, Mississauga, ON, Canada
- Centre for Advancing Responsible and Ethical Artificial Intelligence (CARE-AI), University of Guelph, Guelph, ON, Canada
| | | | - Giancarlo Pruneri
- Department of Pathology and Laboratory Medicine, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
- Faculty of Medicine and Surgery, University of Milan, Milan, Italy
| | - Lajos Pusztai
- Yale Cancer Center, New Haven, CT, USA
- Department of Medical Oncology, Yale School of Medicine, New Haven, CT, USA
| | - Arman Rahman
- UCD School of Biomolecular and Biomedical Science, University College Dublin, Dublin, Ireland
| | | | - Bernardo Leon Rapoport
- The Medical Oncology Centre of Rosebank, Johannesburg, South Africa
- Department of Immunology, Faculty of Health Sciences, University of Pretoria, Pretoria, South Africa
| | - Tilman T Rau
- Institute of Pathology, University Hospital Düsseldorf and Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany
| | - Jorge S Reis-Filho
- Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Joana M Ribeiro
- Département de Médecine Oncologique, Institute Gustave Roussy, Villejuif, France
| | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
- Department of Medicine, Yale University School of Medicine, New Haven, CT, USA
| | - Anne Vincent-Salomon
- Department of Diagnostic and Theranostic Medicine, Institut Curie, University Paris-Sciences et Lettres, Paris, France
| | - Manuel Salto-Tellez
- Integrated Pathology Unit, Institute of Cancer Research, London, UK
- Precision Medicine Centre, Queen's University Belfast, Belfast, UK
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook Medicine, New York, NY, USA
| | - Shahin Sayed
- Department of Pathology, Aga Khan University, Nairobi, Kenya
| | - Kalliopi P Siziopikou
- Department of Pathology, Section of Breast Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory J.-C. Heuson, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB), Université Libre de Bruxelles (ULB), Brussels, Belgium
- Medical Oncology Department, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (HUB), Université Libre de Bruxelles, Brussels, Belgium
| | - Albrecht Stenzinger
- Institute of Pathology, University Hospital Heidelberg, Centers for Personalized Medicine (ZPM), Heidelberg, Germany
| | | | - Daniel Sur
- Department of Medical Oncology, University of Medicine and Pharmacy "Iuliu Hatieganu", Cluj-Napoca, Romania
| | - Fraser Symmans
- University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | | | - Sabine Tejpar
- Digestive Oncology, Department of Oncology, KU Leuven, Leuven, Belgium
| | - Jonas Teuwen
- AI for Oncology Lab, Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Trine Tramm
- Pathology, and Institute of Clinical Medicine, Aarhus University Hospital, Aarhus, Denmark
| | - William T Tran
- Department of Radiation Oncology, University of Toronto and Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht, The Netherlands
- Johns Hopkins Oncology Center, Baltimore, MD, USA
| | - Gregory E Verghese
- Cancer Bioinformatics, School of Cancer & Pharmaceutical Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
- Breast Cancer Now Research Unit, School of Cancer and Pharmaceutical Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
| | - Giuseppe Viale
- Department of Pathology, European Institute of Oncology & University of Milan, Milan, Italy
| | - Michael Vieth
- Institute of Pathology, Klinikum Bayreuth GmbH, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Noorul Wahab
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Coventry, UK
| | - Thomas Walter
- Centre for Computational Biology (CBIO), Mines Paris, PSL University, Paris, France
- Institut Curie, PSL University, Paris, France
- INSERM, U900, Paris, France
| | | | - Hannah Y Wen
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Wentao Yang
- Fudan Medical University Shanghai Cancer Center, Shanghai, PR China
| | - Yinyin Yuan
- Translational Molecular Pathology, Division of Pathology and Laboratory Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Sylvia Adams
- Perlmutter Cancer Center, NYU Langone Health, New York, NY, USA
- Department of Medicine, NYU Grossman School of Medicine, Manhattan, NY, USA
| | | | - Sibylle Loibl
- Department of Medicine and Research, German Breast Group, Neu-Isenburg, Germany
| | - Carsten Denkert
- Institut für Pathologie, Philipps-Universität Marburg und Universitätsklinikum Marburg, Marburg, Germany
| | - Peter Savas
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia
- Sir Peter MacCallum Department of Medical Oncology, University of Melbourne, Melbourne, VIC, Australia
| | - Sherene Loi
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia
- Sir Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, VIC, Australia
| | - Roberto Salgado
- Department of Pathology, GZA-ZNA Hospitals, Antwerp, Belgium
- Division of Cancer Research, Peter MacCallum Cancer Centre, Melbourne, VIC, Australia
| | - Elisabeth Specht Stovgaard
- Department of Pathology, Herlev and Gentofte Hospital, Herlev, Denmark
- Faculty of Health and Medical Sciences, Copenhagen University, Copenhagen, Denmark
| |
Collapse
|
14
|
Bokhorst JM, Nagtegaal ID, Fraggetta F, Vatrano S, Mesker W, Vieth M, van der Laak J, Ciompi F. Deep learning for multi-class semantic segmentation enables colorectal cancer detection and classification in digital pathology images. Sci Rep 2023; 13:8398. [PMID: 37225743 DOI: 10.1038/s41598-023-35491-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 05/18/2023] [Indexed: 05/26/2023] Open
Abstract
In colorectal cancer (CRC), artificial intelligence (AI) can alleviate the laborious task of characterization and reporting on resected biopsies, including polyps, the numbers of which are increasing as a result of CRC population screening programs ongoing in many countries all around the globe. Here, we present an approach to address two major challenges in the automated assessment of CRC histopathology whole-slide images. We present an AI-based method to segment multiple ([Formula: see text]) tissue compartments in the H &E-stained whole-slide image, which provides a different, more perceptible picture of tissue morphology and composition. We test and compare a panel of state-of-the-art loss functions available for segmentation models, and provide indications about their use in histopathology image segmentation, based on the analysis of (a) a multi-centric cohort of CRC cases from five medical centers in the Netherlands and Germany, and (b) two publicly available datasets on segmentation in CRC. We used the best performing AI model as the basis for a computer-aided diagnosis system that classifies colon biopsies into four main categories that are relevant pathologically. We report the performance of this system on an independent cohort of more than 1000 patients. The results show that with a good segmentation network as a base, a tool can be developed which can support pathologists in the risk stratification of colorectal cancer patients, among other possible uses. We have made the segmentation model available for research use on https://grand-challenge.org/algorithms/colon-tissue-segmentation/ .
Collapse
Affiliation(s)
- John-Melle Bokhorst
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Iris D Nagtegaal
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Filippo Fraggetta
- Pathology Unit Gravina Hospital, Gravina Hospital, Caltagirone, Italy
| | - Simona Vatrano
- Pathology Unit Gravina Hospital, Gravina Hospital, Caltagirone, Italy
| | - Wilma Mesker
- Leids Universitair Medisch Centrum, Leiden, The Netherlands
| | - Michael Vieth
- Klinikum Bayreuth, Friedrich-Alexander-University Erlangen-Nuremberg, Bayreuth, Germany
| | - Jeroen van der Laak
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | - Francesco Ciompi
- Department of pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
15
|
Bokhorst JM, Nagtegaal ID, Zlobec I, Dawson H, Sheahan K, Simmer F, Kirsch R, Vieth M, Lugli A, van der Laak J, Ciompi F. Semi-Supervised Learning to Automate Tumor Bud Detection in Cytokeratin-Stained Whole-Slide Images of Colorectal Cancer. Cancers (Basel) 2023; 15:cancers15072079. [PMID: 37046742 PMCID: PMC10093661 DOI: 10.3390/cancers15072079] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 03/24/2023] [Accepted: 03/29/2023] [Indexed: 04/03/2023] Open
Abstract
Tumor budding is a histopathological biomarker associated with metastases and adverse survival outcomes in colorectal carcinoma (CRC) patients. It is characterized by the presence of single tumor cells or small clusters of cells within the tumor or at the tumor-invasion front. In order to obtain a tumor budding score for a patient, the region with the highest tumor bud density must first be visually identified by a pathologist, after which buds will be counted in the chosen hotspot field. The automation of this process will expectedly increase efficiency and reproducibility. Here, we present a deep learning convolutional neural network model that automates the above procedure. For model training, we used a semi-supervised learning method, to maximize the detection performance despite the limited amount of labeled training data. The model was tested on an independent dataset in which human- and machine-selected hotspots were mapped in relation to each other and manual and machine detected tumor bud numbers in the manually selected fields were compared. We report the results of the proposed method in comparison with visual assessment by pathologists. We show that the automated tumor bud count achieves a prognostic value comparable with visual estimation, while based on an objective and reproducible quantification. We also explore novel metrics to quantify buds such as density and dispersion and report their prognostic value. We have made the model available for research use on the grand-challenge platform.
Collapse
|
16
|
Leon-Ferre RA, Carter JM, Zahrieh D, Sinnwell JP, Salgado R, Suman V, Hillman D, Boughey JC, Kalari KR, Couch FJ, Ingle JN, Balkenkohl M, Ciompi F, van der Laak J, Goetz MP. Abstract P2-11-34: Mitotic spindle hotspot counting using deep learning networks is highly associated with clinical outcomes in patients with early-stage triple-negative breast cancer who did not receive systemic therapy. Cancer Res 2023. [DOI: 10.1158/1538-7445.sabcs22-p2-11-34] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2023]
Abstract
Abstract
Background: Triple-negative breast cancers (TNBC) exhibit high rates of recurrence and mortality. However, recent studies suggest that a subset of patients (pts) with early-stage TNBC enriched in tumor-infiltrating lymphocytes (TILs) have excellent clinical outcomes even in the absence of systemic therapy. Additional histological biomarkers that could identify pts for future systemic therapy escalation/de-escalation strategies are of great interest. TNBC are frequently highly proliferative with abundant mitoses. However, classic markers of proliferation (manual mitosis counting and Ki-67) appear to offer no prognostic value. Here, we evaluated the prognostic effects of automated mitotic spindle hotspot (AMSH) counting on RFS in independent cohorts of systemically untreated early-stage TNBC.
Methods: AMSH counting was conducted with a state-of-the-art deep learning algorithm trained on the detection of mitoses within 2 mm2 areas with the highest mitotic density (i.e. hotspots) in digital H&E images. Details of the development, training and validation of the algorithm were published previously [1] in a cohort of unselected TNBC. We obtained AMSH counts in a centrally confirmed TNBC cohort from Mayo Clinic [2] and focused our analysis on pts who received locoregional therapy but no systemic therapy. Using a fractional polynomial analysis with a multivariable proportional hazards regression model, we confirmed the assumption of linearity in the log hazard for the continuous variable AMSH and evaluated whether AMSH counts were prognostic of RFS. We corroborated our findings in an independent cohort of systemically untreated TNBC pts from the Radboud University Medical Center in the Netherlands (Radboud Cohort). Results are reported at a median follow-up of 8.1 and 6.7 years for the Mayo and Netherlands cohorts, respectively.
Results: Among 182 pts with who did not receive systemic therapy in the Mayo Cohort, 140 (77%) with available AMSH counts were included. The mean age was 61 (range: 31-94), 71% were postmenopausal, 67% had tumors ≤ 2cm, and 83% were node-negative. As expected, most tumors were Nottingham grade 3 (84%) and had a high Ki-67 proliferation index (54% with Ki-67 >30%). Most tumors (73%) had stromal TILs ≤ 30%. The median AMSH count was 18 (IQR: 8, 42). AMSH counts were linearly associated with grade and tumor size, with the proportion of pts with grade 3 tumors and size > 2 cm increasing as the AMSH counts increased (p=0.007 and p=0.059, respectively). In a multivariate model controlling for nodal status, tumor size, and stromal TILs, AMSH counts were independently associated with RFS (p< 0.0001). For every 10-point increase in the AMSH count, we observed a 17% increase in the risk of experiencing an RFS event (HR 1.17, 95% CI 1.08-1.26). We corroborated our findings in the Radboud Cohort (n=126). The mean age was 68 (range: 40-96), and 81% were node-negative. While the median AMSH count was 36 (IQR: 16-63), higher than in the Mayo Cohort (p=0.004), the prognostic impact was similar, with a significant association between AMSH count and RFS (p=0.028) in a multivariate model corrected for nodal status, tumor size, and stromal TILs. For every 10-point increase in the AMSH count in the Netherlands cohort, we observed a 9% increase in the risk of experiencing an RFS event (HR 1.09, 95% CI 1.01-1.17). RFS rates according to AMSH counts for both cohorts are shown in the Table.
Conclusions: AMSH counting is a new proliferation biomarker that provides prognostic value independent of nodal status, tumor size, and stromal TILs in systemically untreated early-stage TNBC. Plans are underway to evaluate AMSH counts in additional cohorts of systemically untreated TNBC, and in other disease settings such as prior to neoadjuvant systemic therapy. If validated, this biomarker should be prospectively evaluated as a potential selection biomarker in clinical trials of systemic therapy de-escalation.
References:
1. PMID: 29994086
2. PMID: 28913760
Table RFS according to AMSH counts in the Mayo and Radboud Cohorts
Citation Format: Roberto A. Leon-Ferre, Jodi M. Carter, David Zahrieh, Jason P. Sinnwell, Roberto Salgado, Vera Suman, David Hillman, Judy C. Boughey, Krishna R. Kalari, Fergus J. Couch, James N. Ingle, Maschenka Balkenkohl, Francesco Ciompi, Jeroen van der Laak, Matthew P. Goetz. Mitotic spindle hotspot counting using deep learning networks is highly associated with clinical outcomes in patients with early-stage triple-negative breast cancer who did not receive systemic therapy [abstract]. In: Proceedings of the 2022 San Antonio Breast Cancer Symposium; 2022 Dec 6-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2023;83(5 Suppl):Abstract nr P2-11-34.
Collapse
Affiliation(s)
| | | | | | | | - Roberto Salgado
- 5GZA-ZNA-Hospitals, Antwerp, Belgium; Peter Mac Callum Cancer Centre, Melbourne, Australia
| | | | | | - Judy C. Boughey
- 8Division of Breast and Melanoma Surgical Oncology, Department of Surgery,Mayo Clinic, Rochester, Minnesota, USA
| | | | | | | | | | | | | | | |
Collapse
|
17
|
Aubreville M, Stathonikos N, Bertram CA, Klopfleisch R, Ter Hoeve N, Ciompi F, Wilm F, Marzahl C, Donovan TA, Maier A, Breen J, Ravikumar N, Chung Y, Park J, Nateghi R, Pourakpour F, Fick RHJ, Ben Hadj S, Jahanifar M, Shephard A, Dexl J, Wittenberg T, Kondo S, Lafarge MW, Koelzer VH, Liang J, Wang Y, Long X, Liu J, Razavi S, Khademi A, Yang S, Wang X, Erber R, Klang A, Lipnik K, Bolfa P, Dark MJ, Wasinger G, Veta M, Breininger K. Mitosis domain generalization in histopathology images - The MIDOG challenge. Med Image Anal 2023; 84:102699. [PMID: 36463832 DOI: 10.1016/j.media.2022.102699] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2022] [Revised: 10/28/2022] [Accepted: 11/17/2022] [Indexed: 11/27/2022]
Abstract
The density of mitotic figures (MF) within tumor tissue is known to be highly correlated with tumor proliferation and thus is an important marker in tumor grading. Recognition of MF by pathologists is subject to a strong inter-rater bias, limiting its prognostic value. State-of-the-art deep learning methods can support experts but have been observed to strongly deteriorate when applied in a different clinical environment. The variability caused by using different whole slide scanners has been identified as one decisive component in the underlying domain shift. The goal of the MICCAI MIDOG 2021 challenge was the creation of scanner-agnostic MF detection algorithms. The challenge used a training set of 200 cases, split across four scanning systems. As test set, an additional 100 cases split across four scanning systems, including two previously unseen scanners, were provided. In this paper, we evaluate and compare the approaches that were submitted to the challenge and identify methodological factors contributing to better performance. The winning algorithm yielded an F1 score of 0.748 (CI95: 0.704-0.781), exceeding the performance of six experts on the same task.
Collapse
Affiliation(s)
| | | | - Christof A Bertram
- Institute of Pathology, University of Veterinary Medicine, Vienna, Austria
| | - Robert Klopfleisch
- Institute of Veterinary Pathology, Freie Universität Berlin, Berlin, Germany
| | | | - Francesco Ciompi
- Computational Pathology Group, Radboud UMC, Nijmegen, The Netherlands
| | - Frauke Wilm
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany; Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Christian Marzahl
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Taryn A Donovan
- Department of Anatomic Pathology, Schwarzman Animal Medical Center, NY, USA
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jack Breen
- CISTIB Center for Computational Imaging and Simulation Technologies in Biomedicine, School of Computing, University of Leeds, Leeds, UK
| | - Nishant Ravikumar
- CISTIB Center for Computational Imaging and Simulation Technologies in Biomedicine, School of Computing, University of Leeds, Leeds, UK
| | - Youjin Chung
- Korea Advanced Institute of Science and Technology, Daejeon, South Korea
| | - Jinah Park
- Korea Advanced Institute of Science and Technology, Daejeon, South Korea
| | - Ramin Nateghi
- Electrical and Electronics Engineering Department, Shiraz University of Technology, Shiraz, Iran
| | - Fattaneh Pourakpour
- Iranian Brain Mapping Biobank (IBMB), National Brain Mapping Laboratory (NBML), Tehran, Iran
| | | | | | - Mostafa Jahanifar
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Warwick, UK
| | - Adam Shephard
- Tissue Image Analytics Centre, Department of Computer Science, University of Warwick, Warwick, UK
| | - Jakob Dexl
- Fraunhofer-Institute for Integrated Circuits IIS, Erlangen, Germany
| | | | | | - Maxime W Lafarge
- Department of Pathology and Molecular Pathology, University Hospital and University of Zurich, Zurich, Switzerland
| | - Viktor H Koelzer
- Department of Pathology and Molecular Pathology, University Hospital and University of Zurich, Zurich, Switzerland
| | - Jingtang Liang
- School of Life Science and Technology, Xidian University, Shannxi, China
| | - Yubo Wang
- School of Life Science and Technology, Xidian University, Shannxi, China
| | - Xi Long
- Histo Pathology Diagnostic Center, Shanghai, China
| | - Jingxin Liu
- Xi'an Jiaotong-Liverpool University, Suzhou, China
| | - Salar Razavi
- Image Analysis in Medicine Lab (IAMLAB), Electrical, Computer and Biomedical Engineering, Ryerson University, Toronto, ON, Canada
| | - April Khademi
- Image Analysis in Medicine Lab (IAMLAB), Electrical, Computer and Biomedical Engineering, Ryerson University, Toronto, ON, Canada
| | - Sen Yang
- Tencent AI Lab, Shenzhen 518057, China
| | - Xiyue Wang
- College of Computer Science, Sichuan University, Chengdu 610065, China
| | - Ramona Erber
- Institute of Pathology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Andrea Klang
- Institute of Pathology, University of Veterinary Medicine, Vienna, Austria
| | - Karoline Lipnik
- Institute of Pathology, University of Veterinary Medicine, Vienna, Austria
| | - Pompei Bolfa
- Ross University School of Veterinary Medicine, Basseterre, Saint Kitts and Nevis
| | - Michael J Dark
- College of Veterinary Medicine, University of Florida, Gainesville, FL, USA
| | - Gabriel Wasinger
- Department of Pathology, General Hospital of Vienna, Medical University of Vienna, Vienna, Austria
| | - Mitko Veta
- Medical Image Analysis Group, TU Eindhoven, Eindhoven, The Netherlands
| | - Katharina Breininger
- Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
18
|
Marini N, Marchesin S, Otálora S, Wodzinski M, Caputo A, van Rijthoven M, Aswolinskiy W, Bokhorst JM, Podareanu D, Petters E, Boytcheva S, Buttafuoco G, Vatrano S, Fraggetta F, van der Laak J, Agosti M, Ciompi F, Silvello G, Muller H, Atzori M. Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations. NPJ Digit Med 2022; 5:102. [PMID: 35869179 PMCID: PMC9307641 DOI: 10.1038/s41746-022-00635-4] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2021] [Accepted: 06/24/2022] [Indexed: 01/02/2023] Open
Abstract
The digitalization of clinical workflows and the increasing performance of deep learning algorithms are paving the way towards new methods for tackling cancer diagnosis. However, the availability of medical specialists to annotate digitized images and free-text diagnostic reports does not scale with the need for large datasets required to train robust computer-aided diagnosis methods that can target the high variability of clinical cases and data produced. This work proposes and evaluates an approach to eliminate the need for manual annotations to train computer-aided diagnosis tools in digital pathology. The approach includes two components, to automatically extract semantically meaningful concepts from diagnostic reports and use them as weak labels to train convolutional neural networks (CNNs) for histopathology diagnosis. The approach is trained (through 10-fold cross-validation) on 3’769 clinical images and reports, provided by two hospitals and tested on over 11’000 images from private and publicly available datasets. The CNN, trained with automatically generated labels, is compared with the same architecture trained with manual labels. Results show that combining text analysis and end-to-end deep neural networks allows building computer-aided diagnosis tools that reach solid performance (micro-accuracy = 0.908 at image-level) based only on existing clinical data without the need for manual annotations.
Collapse
|
19
|
Mercan C, Balkenhol M, Salgado R, Sherman M, Vielh P, Vreuls W, Polónia A, Horlings HM, Weichert W, Carter JM, Bult P, Christgen M, Denkert C, van de Vijver K, Bokhorst JM, van der Laak J, Ciompi F. Deep learning for fully-automated nuclear pleomorphism scoring in breast cancer. NPJ Breast Cancer 2022; 8:120. [DOI: 10.1038/s41523-022-00488-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2021] [Accepted: 10/21/2022] [Indexed: 11/11/2022] Open
Abstract
AbstractTo guide the choice of treatment, every new breast cancer is assessed for aggressiveness (i.e., graded) by an experienced histopathologist. Typically, this tumor grade consists of three components, one of which is the nuclear pleomorphism score (the extent of abnormalities in the overall appearance of tumor nuclei). The degree of nuclear pleomorphism is subjectively classified from 1 to 3, where a score of 1 most closely resembles epithelial cells of normal breast epithelium and 3 shows the greatest abnormalities. Establishing numerical criteria for grading nuclear pleomorphism is challenging, and inter-observer agreement is poor. Therefore, we studied the use of deep learning to develop fully automated nuclear pleomorphism scoring in breast cancer. The reference standard used for training the algorithm consisted of the collective knowledge of an international panel of 10 pathologists on a curated set of regions of interest covering the entire spectrum of tumor morphology in breast cancer. To fully exploit the information provided by the pathologists, a first-of-its-kind deep regression model was trained to yield a continuous scoring rather than limiting the pleomorphism scoring to the standard three-tiered system. Our approach preserves the continuum of nuclear pleomorphism without necessitating a large data set with explicit annotations of tumor nuclei. Once translated to the traditional system, our approach achieves top pathologist-level performance in multiple experiments on regions of interest and whole-slide images, compared to a panel of 10 and 4 pathologists, respectively.
Collapse
|
20
|
Hermsen M, Ciompi F, Adefidipe A, Denic A, Dendooven A, Smith BH, van Midden D, Bräsen JH, Kers J, Stegall MD, Bándi P, Nguyen T, Swiderska-Chadaj Z, Smeets B, Hilbrands LB, van der Laak JAWM. Convolutional Neural Networks for the Evaluation of Chronic and Inflammatory Lesions in Kidney Transplant Biopsies. Am J Pathol 2022; 192:1418-1432. [PMID: 35843265 DOI: 10.1016/j.ajpath.2022.06.009] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Revised: 06/13/2022] [Accepted: 06/22/2022] [Indexed: 06/15/2023]
Abstract
In kidney transplant biopsies, both inflammation and chronic changes are important features that predict long-term graft survival. Quantitative scoring of these features is important for transplant diagnostics and kidney research. However, visual scoring is poorly reproducible and labor intensive. The goal of this study was to investigate the potential of convolutional neural networks (CNNs) to quantify inflammation and chronic features in kidney transplant biopsies. A structure segmentation CNN and a lymphocyte detection CNN were applied on 125 whole-slide image pairs of periodic acid-Schiff- and CD3-stained slides. The CNN results were used to quantify healthy and sclerotic glomeruli, interstitial fibrosis, tubular atrophy, and inflammation within both nonatrophic and atrophic tubuli, and in areas of interstitial fibrosis. The computed tissue features showed high correlation with Banff lesion scores of five pathologists (A.A., A.Dend., J.H.B., J.K., and T.N.). Analyses on a small subset showed a moderate correlation toward higher CD3+ cell density within scarred regions and higher CD3+ cell count inside atrophic tubuli correlated with long-term change of estimated glomerular filtration rate. The presented CNNs are valid tools to yield objective quantitative information on glomeruli number, fibrotic tissue, and inflammation within scarred and non-scarred kidney parenchyma in a reproducible manner. CNNs have the potential to improve kidney transplant diagnostics and will benefit the community as a novel method to generate surrogate end points for large-scale clinical studies.
Collapse
Affiliation(s)
- Meyke Hermsen
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Adeyemi Adefidipe
- Department of Pathology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands
| | - Aleksandar Denic
- Division of Nephrology and Hypertension, Mayo Clinic, Rochester, Minnesota
| | - Amélie Dendooven
- Department of Pathology, Ghent University Hospital, Ghent, Belgium; Faculty of Medicine, University of Antwerp, Wilrijk, Antwerp, Belgium
| | - Byron H Smith
- William J. von Liebig Center for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota; Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, Minnesota
| | - Dominique van Midden
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jan Hinrich Bräsen
- Nephropathology Unit, Institute of Pathology, Hannover Medical School, Hannover, Germany
| | - Jesper Kers
- Department of Pathology, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands; Department of Pathology, Leiden University Medical Center, Leiden, the Netherlands; Center for Analytical Sciences Amsterdam, Van 't Hoff Institute for Molecular Sciences, University of Amsterdam, Amsterdam, the Netherlands
| | - Mark D Stegall
- Division of Transplantation Surgery, Mayo Clinic, Rochester, Minnesota
| | - Péter Bándi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Tri Nguyen
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Zaneta Swiderska-Chadaj
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands; Faculty of Electrical Engineering, Warsaw University of Technology, Warsaw, Poland
| | - Bart Smeets
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Luuk B Hilbrands
- Department of Nephrology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jeroen A W M van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
| |
Collapse
|
21
|
Marchesin S, Giachelle F, Marini N, Atzori M, Boytcheva S, Buttafuoco G, Ciompi F, Di Nunzio GM, Fraggetta F, Irrera O, Müller H, Primov T, Vatrano S, Silvello G. Empowering Digital Pathology Applications through Explainable Knowledge Extraction Tools. J Pathol Inform 2022; 13:100139. [PMID: 36268087 PMCID: PMC9577130 DOI: 10.1016/j.jpi.2022.100139] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 09/06/2022] [Accepted: 09/07/2022] [Indexed: 11/25/2022] Open
|
22
|
Munari E, Querzoli G, Brunelli M, Marconi M, Sommaggio M, Cocchi MA, Martignoni G, Netto GJ, Caliò A, Quatrini L, Mariotti FR, Luchini C, Girolami I, Eccher A, Segala D, Ciompi F, Zamboni G, Moretta L, Bogina G. Comparison of three validated PD-L1 immunohistochemical assays in urothelial carcinoma of the bladder: interchangeability and issues related to patient selection. Front Immunol 2022; 13:954910. [PMID: 35967344 PMCID: PMC9363581 DOI: 10.3389/fimmu.2022.954910] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 07/01/2022] [Indexed: 11/13/2022] Open
Abstract
Different programmed cell death-ligand 1 (PD-L1) assays and scoring algorithms are being used in the evaluation of PD-L1 expression for the selection of patients for immunotherapy in specific settings of advanced urothelial carcinoma (UC). In this paper, we sought to investigate three approved assays (Ventana SP142 and SP263, and Dako 22C3) in UC with emphasis on implications for patient selection for atezolizumab/pembrolizumab as the first line of treatment. Tumors from 124 patients with invasive UC of the bladder were analyzed using tissue microarrays (TMA). Serial sections were stained with SP263 and SP142 on Ventana Benchmark Ultra and with 22C3 on Dako Autostainer Link 48. Stains were evaluated independently by two observers and scored using the combined positive score (CPS) and tumor infiltrating immune cells (IC) algorithms. Differences in proportions (DP), overall percent agreement (OPA), positive percent agreement (PPA), negative percent agreement (NPA), and Cohen κ were calculated for all comparable cases. Good overall concordance in analytic performance was observed for 22C3 and SP263 with both scoring algorithms; specifically, the highest OPA was observed between 22C3 and SP263 (89.6%) when using CPS. On the other hand, SP142 consistently showed lower positivity rates with high differences in proportions (DP) compared with 22C3 and SP263 with both CPS and IC, and with a low PPA, especially when using the CPS algorithm. In conclusion, 22C3 and SP263 assays show comparable analytical performance while SP142 shows divergent staining results, with important implications for the selection of patients for both pembrolizumab and atezolizumab.
Collapse
Affiliation(s)
- Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
- *Correspondence: Enrico Munari, ; Lorenzo Moretta,
| | - Giulia Querzoli
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, Verona, Italy
| | - Matteo Brunelli
- Pathology Unit, Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | - Marcella Marconi
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| | | | - Marco A. Cocchi
- Pathology Unit, Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Guido Martignoni
- Pathology Unit, Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
- Pathology Unit, Pederzoli Hospital, Peschiera del Garda, Verona, Italy
| | - George J. Netto
- Department of Pathology, University of Alabama at Birmingham, Birmingham, AL, United States
| | - Anna Caliò
- Pathology Unit, Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | - Linda Quatrini
- Tumor Immunology Unit, Bambino Gesù Children’s Hospital (IRCCS), Rome, Italy
| | | | - Claudio Luchini
- Pathology Unit, Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
| | | | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, Verona, Italy
| | - Diego Segala
- Pathology Unit, ASST Spedali Civili, Brescia, Italy
| | - Francesco Ciompi
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, Nijmegen, Netherlands
| | - Giuseppe Zamboni
- Pathology Unit, Department of Diagnostics and Public Health, University and Hospital Trust of Verona, Verona, Italy
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| | - Lorenzo Moretta
- Tumor Immunology Unit, Bambino Gesù Children’s Hospital (IRCCS), Rome, Italy
- *Correspondence: Enrico Munari, ; Lorenzo Moretta,
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria Hospital, Negrar di Valpolicella, Verona, Italy
| |
Collapse
|
23
|
Litjens G, Ciompi F, van der Laak J. A Decade of GigaScience: The Challenges of Gigapixel Pathology Images. Gigascience 2022; 11:6608502. [PMID: 35701372 PMCID: PMC9197683 DOI: 10.1093/gigascience/giac056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2022] [Accepted: 05/23/2022] [Indexed: 11/14/2022] Open
Abstract
In the last decade, the field of computational pathology has advanced at a rapid pace because of the availability of deep neural networks, which achieved their first successes in computer vision tasks in 2012. An important driver for the progress of the field were public competitions, so called 'Grand Challenges', in which increasingly large data sets were offered to the public to solve clinically relevant tasks. Going from the first Pathology challenges, which had data obtained from 23 patients, to current challenges sharing data of thousands of patients, performance of developed deep learning solutions has reached (and sometimes surpassed) the level of experienced pathologists for specific tasks. We expect future challenges to broaden the horizon, for instance by combining data from radiology, pathology and tumor genetics, and to extract prognostic and predictive information independent of currently used grading schemes.
Collapse
Affiliation(s)
- Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands.,Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| |
Collapse
|
24
|
Satturwar S, Girolami I, Munari E, Ciompi F, Eccher A, Pantanowitz L. Program death ligand-1 immunocytochemistry in lung cancer cytological samples: A systematic review. Diagn Cytopathol 2022; 50:313-323. [PMID: 35293692 PMCID: PMC9310737 DOI: 10.1002/dc.24955] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2022] [Revised: 03/01/2022] [Accepted: 03/02/2022] [Indexed: 12/19/2022]
Abstract
In this era of personalized medicine, targeted immunotherapies like immune checkpoint inhibitors (ICI) blocking the programmed death‐1 (PD‐1)/program death ligand‐1 (PD‐L1) axis have become an integral part of treating advanced stage non‐small cell lung carcinoma (NSCLC) and many other cancer types. Multiple monoclonal antibodies are available commercially to detect PD‐L1 expression in tumor cells by immunohistochemistry (IHC). As most clinical trials initially required tumor biopsy for PD‐L1 detection by IHC, many of the currently available PD‐1/PD‐L1 assays have been developed and validated on formalin fixed tissue specimens. The majority (>50%) of lung cancer cases do not have a surgical biopsy or resection specimen available for ancillary testing and instead must rely primarily on fine needle aspiration biopsy specimens for diagnosis, staging and ancillary tests. Review of the literature shows multiple studies exploring the feasibility of PD‐L1 IHC on cytological samples. In addition, there are studies addressing various aspects of IHC validation on cytology preparations including pre‐analytical (e.g., different fixatives), analytical (e.g., antibody clone, staining platforms, inter and intra‐observer agreement, cytology‐histology concordance) and post‐analytical (e.g., clinical outcome) issues. Although promising results in this field have emerged utilizing cytology samples, many important questions still need to be addressed. This review summarizes the literature of PD‐L1 IHC in lung cytology specimens and provides practical tips for optimizing analysis.
Collapse
Affiliation(s)
- Swati Satturwar
- Department of Pathology, The Ohio State University, Columbus, Ohio, USA
| | - Ilaria Girolami
- Division of Pathology, Bolzano Central Hospital, Bolzano, Italy
| | - Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Francesco Ciompi
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, Nijmegen, Netherlands
| | - Albino Eccher
- Department of Pathology and Diagnostics, University and Hospital Trust of Verona, Verona, Italy
| | - Liron Pantanowitz
- Department of Pathology, University of Michigan, Ann Arbor, Michigan, USA
| |
Collapse
|
25
|
Faita F, Oranges T, Di Lascio N, Ciompi F, Vitali S, Aringhieri G, Janowska A, Romanelli M, Dini V. Ultra-high-frequency ultrasound and machine learning approaches for the differential diagnosis of melanocytic lesions. Exp Dermatol 2022; 31:94-98. [PMID: 33738861 DOI: 10.1111/exd.14330] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Revised: 03/14/2021] [Accepted: 03/15/2021] [Indexed: 12/12/2022]
Abstract
Malignant melanoma (MM) is one of the most dangerous skin cancers. The aim of this study was to present a potential new method for the differential diagnosis of MM from melanocytic naevi (MN). We examined 20 MM and 19 MN with a new ultra-high-frequency ultrasound (UHFUS) equipped with a 70 MHz linear probe. Ultrasonographic images were processed for calculating 8 morphological parameters (area, perimeter, circularity, area ratio, standard deviation of normalized radial range, roughness index, overlap ratio and normalized residual mean square value) and 122 texture parameters. Colour Doppler images were used to evaluate the vascularization. Features reduction was implemented by means of principal component analysis (PCA), and 23 classification algorithms were tested on the reduced features using histological response as ground-truth. Best results were obtained using only the first component of the PCA and the weighted k-nearest neighbour classifier; this combination led to an accuracy of 76.9%, area under the ROC curve of 83%, sensitivity of 84% and specificity of 70%. The histological analysis still remains the gold-standard, but the UHFUS images processing using a machine learning approach could represent a new non-invasive approach.
Collapse
Affiliation(s)
- Francesco Faita
- Institute of Clinical Physiology, National Research Council, Pisa, Italy
| | - Teresa Oranges
- Department of Dermatology, University of Pisa, Pisa, Italy
- Dermatology Unit, Department of Pediatrics, Meyer Children's University Hospital, Florence, Italy
| | - Nicole Di Lascio
- Institute of Clinical Physiology, National Research Council, Pisa, Italy
| | - Francesco Ciompi
- Computational Pathology Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Saverio Vitali
- Diagnostic and Interventional Radiology, University of Pisa, Pisa, Italy
| | - Giacomo Aringhieri
- Diagnostic and Interventional Radiology, University of Pisa, Pisa, Italy
| | - Agata Janowska
- Department of Dermatology, University of Pisa, Pisa, Italy
| | | | - Valentina Dini
- Department of Dermatology, University of Pisa, Pisa, Italy
| |
Collapse
|
26
|
Fick RHJ, Tayart B, Bertrand C, Lang SC, Rey T, Ciompi F, Tilmant C, Farre I, Hadj SB. A Partial Label-Based Machine Learning Approach For Cervical Whole-Slide Image Classification: The Winning TissueNet Solution . Annu Int Conf IEEE Eng Med Biol Soc 2021; 2021:2127-2131. [PMID: 34891709 DOI: 10.1109/embc46164.2021.9631009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Cervical cancer is the fourth most common cancer in women worldwide. To determine early treatment for patients, it is critical to accurately classify the cervical intraepithelial lesion status based on a microscopic biopsy. Lesion classification is a 4-class problem, with biopsies being designated as benign or increasingly malignant as class 1-3, with 3 being invasive cancer. Unfortunately, traditional biopsy analysis by a pathologist is time-consuming and subject to intra- and inter-observer variability. For this reason, it is of interest to develop automatic analysis pipelines to classify lesion status directly from a digitalized whole slide image (WSI). The recent TissueNet Challenge was organized to find the best automatic detection pipeline for this task, using a dataset of 1015 annotated WSI slides. In this work, we present our winning end-to-end solution for cervical slide classification composed of a two-step classification model: First, we classify individual slide patches using an ensemble CNN, followed by an SVM-based slide classification using statistical features of the aggregated patch-level predictions. Importantly, we present the key innovation of our approach, which is a novel partial label-based loss function that allows us to supplement the supervised WSI patch annotations with weakly supervised patches based on the WSI class. This led to us not requiring additional expert tissue annotation, while still reaching the winning score of 94.7%. Our approach is a step towards the clinical inclusion of automatic pipelines for cervical cancer treatment planning.Clinical relevance- The explanation of the winning Tis-sueNet AI algorithm for automated cervical cancer classification, which may provide insights for the next generation of computer assisted tools in digital pathology.
Collapse
|
27
|
Marini N, Otálora S, Podareanu D, van Rijthoven M, van der Laak J, Ciompi F, Müller H, Atzori M. Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images. Front Comput Sci 2021. [DOI: 10.3389/fcomp.2021.684521] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023] Open
Abstract
Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.
Collapse
|
28
|
Munari E, Marconi M, Querzoli G, Lunardi G, Bertoglio P, Ciompi F, Tosadori A, Eccher A, Tumino N, Quatrini L, Vacca P, Rossi G, Cavazza A, Martignoni G, Brunelli M, Netto GJ, Moretta L, Zamboni G, Bogina G. Impact of PD-L1 and PD-1 Expression on the Prognostic Significance of CD8 + Tumor-Infiltrating Lymphocytes in Non-Small Cell Lung Cancer. Front Immunol 2021; 12:680973. [PMID: 34122444 PMCID: PMC8187779 DOI: 10.3389/fimmu.2021.680973] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 05/10/2021] [Indexed: 12/26/2022] Open
Abstract
The immune infiltrate within tumors has proved to be very powerful in the prognostic stratification of patients and much attention is also being paid towards its predictive value. In this work we therefore aimed at clarifying the significance and impact of PD-L1 and PD-1 expression on the prognostic value of CD8+ tumor infiltrating lymphocytes (TILs) in a cohort of consecutive patients with primary resected non-small cell lung cancer (NSCLC). Tissue microarrays (TMA) were built using one representative formalin fixed paraffin embedded block for every case, with 5 cores for each block. TMA sections were stained with PD-L1 (clone SP263), PD-1 (clone NAT105) and CD8 (clone SP57). Number of CD8+ cells per mm2 were automatically counted; median, 25th and 75th percentiles of CD8+ cells were used as threshold for statistical clinical outcome analysis and evaluated in patients subgroups defined by expression of PD-L1 and PD-1 within tumors. We found an overall strong prognostic value of CD8+ cells in our cohort of 314 resected NSCLC, especially in PD-L1 negative tumors lacking PD-1+ TILs, and demonstrated that in PD-L1 positive tumors a higher density of CD8+ lymphocytes is necessary to improve the prognosis. Our data strengthen the concept of the importance of the assessment and quantification of the immune contexture in cancer and, similarly to what has been carried on in colorectal cancer, promote the efforts for the establishment of an Immunoscore for NSCLC for prognostic and possibly predictive purposes.
Collapse
Affiliation(s)
- Enrico Munari
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy.,Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Marcella Marconi
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy
| | - Giulia Querzoli
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy
| | - Gianluigi Lunardi
- Clinical Analysis Laboratory and Transfusional Medicine, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy
| | - Pietro Bertoglio
- Division of Thoracic Surgery, IRCCS Maggiore Teaching Hospital and Sant'Orsola University Hospital, Bologna, Italy
| | - Francesco Ciompi
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, Nijmegen, Netherlands
| | - Alice Tosadori
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy
| | - Albino Eccher
- Pathology Unit, University and Hospital Trust of Verona, Verona, Italy
| | - Nicola Tumino
- Immunology Area, Bambino Gesù Children's Hospital (IRCCS), Rome, Italy
| | - Linda Quatrini
- Immunology Area, Bambino Gesù Children's Hospital (IRCCS), Rome, Italy
| | - Paola Vacca
- Immunology Area, Bambino Gesù Children's Hospital (IRCCS), Rome, Italy
| | - Giulio Rossi
- Pathology Unit, AUSL della Romagna, Ravenna, Italy
| | - Alberto Cavazza
- Pathology Unit, AUSL/IRCCS of Reggio Emilia, Reggio Emilia, Italy
| | - Guido Martignoni
- Department of Diagnostics and Public Health, University of Verona, Verona, Italy.,Pathology Unit, Pederzoli Hospital, Peschiera del Garda, Italy
| | - Matteo Brunelli
- Department of Diagnostics and Public Health, University of Verona, Verona, Italy
| | - George J Netto
- Department of Pathology, University of Alabama at Birmingham, Birmingham, AL, United States
| | - Lorenzo Moretta
- Immunology Area, Bambino Gesù Children's Hospital (IRCCS), Rome, Italy
| | - Giuseppe Zamboni
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy.,Department of Diagnostics and Public Health, University of Verona, Verona, Italy
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, Negrar di Valpolicella, Italy
| |
Collapse
|
29
|
Munari E, Mariotti FR, Quatrini L, Bertoglio P, Tumino N, Vacca P, Eccher A, Ciompi F, Brunelli M, Martignoni G, Bogina G, Moretta L. PD-1/PD-L1 in Cancer: Pathophysiological, Diagnostic and Therapeutic Aspects. Int J Mol Sci 2021; 22:ijms22105123. [PMID: 34066087 PMCID: PMC8151504 DOI: 10.3390/ijms22105123] [Citation(s) in RCA: 54] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 04/26/2021] [Accepted: 04/27/2021] [Indexed: 12/13/2022] Open
Abstract
Immune evasion is a key strategy adopted by tumor cells to escape the immune system while promoting their survival and metastatic spreading. Indeed, several mechanisms have been developed by tumors to inhibit immune responses. PD-1 is a cell surface inhibitory receptor, which plays a major physiological role in the maintenance of peripheral tolerance. In pathological conditions, activation of the PD-1/PD-Ls signaling pathway may block immune cell activation, a mechanism exploited by tumor cells to evade the antitumor immune control. Targeting the PD-1/PD-L1 axis has represented a major breakthrough in cancer treatment. Indeed, the success of PD-1 blockade immunotherapies represents an unprecedented success in the treatment of different cancer types. To improve the therapeutic efficacy, a deeper understanding of the mechanisms regulating PD-1 expression and signaling in the tumor context is required. We provide an overview of the current knowledge of PD-1 expression on both tumor-infiltrating T and NK cells, summarizing the recent evidence on the stimuli regulating its expression. We also highlight perspectives and limitations of the role of PD-L1 expression as a predictive marker, discuss well-established and novel potential approaches to improve patient selection and clinical outcome and summarize current indications for anti-PD1/PD-L1 immunotherapy.
Collapse
Affiliation(s)
- Enrico Munari
- Pathology Unit, Department of Molecular and Translational Medicine, University of Brescia, 25100 Brescia, Italy;
| | - Francesca R. Mariotti
- Immunology Area, Bambino Gesù Children’s Hospital, IRCCS, 00146 Rome, Italy; (F.R.M.); (L.Q.); (N.T.); (P.V.)
| | - Linda Quatrini
- Immunology Area, Bambino Gesù Children’s Hospital, IRCCS, 00146 Rome, Italy; (F.R.M.); (L.Q.); (N.T.); (P.V.)
| | - Pietro Bertoglio
- Division of Thoracic Surgery, IRCCS Maggiore Teaching Hospital and Sant’Orsola University Hospital, 40133 Bologna, Italy;
| | - Nicola Tumino
- Immunology Area, Bambino Gesù Children’s Hospital, IRCCS, 00146 Rome, Italy; (F.R.M.); (L.Q.); (N.T.); (P.V.)
| | - Paola Vacca
- Immunology Area, Bambino Gesù Children’s Hospital, IRCCS, 00146 Rome, Italy; (F.R.M.); (L.Q.); (N.T.); (P.V.)
| | - Albino Eccher
- Pathology Unit, University and Hospital Trust of Verona, 37134 Verona, Italy;
| | - Francesco Ciompi
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, 6543 SH Nijmegen, The Netherlands;
| | - Matteo Brunelli
- Department of Diagnostics and Public Health, University of Verona, 37134 Verona, Italy; (M.B.); (G.M.)
| | - Guido Martignoni
- Department of Diagnostics and Public Health, University of Verona, 37134 Verona, Italy; (M.B.); (G.M.)
- Pathology Unit, Pederzoli Hospital, 37019 Peschiera del Garda, Italy
| | - Giuseppe Bogina
- Pathology Unit, IRCCS Sacro Cuore Don Calabria, 37024 Negrar di Valpolicella, Italy;
| | - Lorenzo Moretta
- Immunology Area, Bambino Gesù Children’s Hospital, IRCCS, 00146 Rome, Italy; (F.R.M.); (L.Q.); (N.T.); (P.V.)
- Correspondence:
| |
Collapse
|
30
|
Abstract
Machine learning techniques have great potential to improve medical diagnostics, offering ways to improve accuracy, reproducibility and speed, and to ease workloads for clinicians. In the field of histopathology, deep learning algorithms have been developed that perform similarly to trained pathologists for tasks such as tumor detection and grading. However, despite these promising results, very few algorithms have reached clinical implementation, challenging the balance between hope and hype for these new techniques. This Review provides an overview of the current state of the field, as well as describing the challenges that still need to be addressed before artificial intelligence in histopathology can achieve clinical value.
Collapse
Affiliation(s)
- Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands. .,Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| |
Collapse
|
31
|
Balkenhol MC, Ciompi F, Świderska-Chadaj Ż, van de Loo R, Intezar M, Otte-Höller I, Geijs D, Lotz J, Weiss N, de Bel T, Litjens G, Bult P, van der Laak JA. Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics. Breast 2021; 56:78-87. [PMID: 33640523 PMCID: PMC7933536 DOI: 10.1016/j.breast.2021.02.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Revised: 02/05/2021] [Accepted: 02/08/2021] [Indexed: 12/29/2022] Open
Abstract
The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.
Collapse
Affiliation(s)
- Maschenka Ca Balkenhol
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands.
| | - Francesco Ciompi
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Żaneta Świderska-Chadaj
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands; Warsaw University of Technology, Faculty of Electrical Engineering, Warsaw, Poland
| | - Rob van de Loo
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Milad Intezar
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Irene Otte-Höller
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Daan Geijs
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Johannes Lotz
- Fraunhofer Institute for Image Computing MEVIS, Lübeck, Germany
| | - Nick Weiss
- Fraunhofer Institute for Image Computing MEVIS, Lübeck, Germany
| | - Thomas de Bel
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Geert Litjens
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Peter Bult
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands
| | - Jeroen Awm van der Laak
- Radboud University Medical Center, Radboud Institute for Health Sciences, Department of Pathology, Nijmegen, the Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| |
Collapse
|
32
|
Tellez D, Litjens G, van der Laak J, Ciompi F. Neural Image Compression for Gigapixel Histopathology Image Analysis. IEEE Trans Pattern Anal Mach Intell 2021; 43:567-578. [PMID: 31442971 DOI: 10.1109/tpami.2019.2936841] [Citation(s) in RCA: 48] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
We propose Neural Image Compression (NIC), a two-step method to build convolutional neural networks for gigapixel image analysis solely using weak image-level labels. First, gigapixel images are compressed using a neural network trained in an unsupervised fashion, retaining high-level information while suppressing pixel-level noise. Second, a convolutional neural network (CNN) is trained on these compressed image representations to predict image-level labels, avoiding the need for fine-grained manual annotations. We compared several encoding strategies, namely reconstruction error minimization, contrastive training and adversarial feature learning, and evaluated NIC on a synthetic task and two public histopathology datasets. We found that NIC can exploit visual cues associated with image-level labels successfully, integrating both global and local visual information. Furthermore, we visualized the regions of the input gigapixel images where the CNN attended to, and confirmed that they overlapped with annotations from human experts.
Collapse
|
33
|
Ciompi F, Veta M, van der Laak J, Rajpoot N. Editorial Computational Pathology. IEEE J Biomed Health Inform 2021. [DOI: 10.1109/jbhi.2021.3052029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
34
|
van Rijthoven M, Balkenhol M, Siliņa K, van der Laak J, Ciompi F. HookNet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images. Med Image Anal 2021; 68:101890. [DOI: 10.1016/j.media.2020.101890] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2019] [Revised: 10/22/2020] [Accepted: 10/23/2020] [Indexed: 10/23/2022]
|
35
|
Balkenhol M, Bult P, Tellez D, Vreuls W, Clahsen P, Ciompi F, Van der Laak J. Deep learning enables fully automated mitotic density assessment in breast cancer histopathology. Eur J Cancer 2020. [DOI: 10.1016/s0959-8049(20)30764-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
36
|
Kos Z, Roblin E, Kim RS, Michiels S, Gallas BD, Chen W, van de Vijver KK, Goel S, Adams S, Demaria S, Viale G, Nielsen TO, Badve SS, Symmans WF, Sotiriou C, Rimm DL, Hewitt S, Denkert C, Loibl S, Luen SJ, Bartlett JMS, Savas P, Pruneri G, Dillon DA, Cheang MCU, Tutt A, Hall JA, Kok M, Horlings HM, Madabhushi A, van der Laak J, Ciompi F, Laenkholm AV, Bellolio E, Gruosso T, Fox SB, Araya JC, Floris G, Hudeček J, Voorwerk L, Beck AH, Kerner J, Larsimont D, Declercq S, Van den Eynden G, Pusztai L, Ehinger A, Yang W, AbdulJabbar K, Yuan Y, Singh R, Hiley C, Bakir MA, Lazar AJ, Naber S, Wienert S, Castillo M, Curigliano G, Dieci MV, André F, Swanton C, Reis-Filho J, Sparano J, Balslev E, Chen IC, Stovgaard EIS, Pogue-Geile K, Blenman KRM, Penault-Llorca F, Schnitt S, Lakhani SR, Vincent-Salomon A, Rojo F, Braybrooke JP, Hanna MG, Soler-Monsó MT, Bethmann D, Castaneda CA, Willard-Gallo K, Sharma A, Lien HC, Fineberg S, Thagaard J, Comerma L, Gonzalez-Ericsson P, Brogi E, Loi S, Saltz J, Klaushen F, Cooper L, Amgad M, Moore DA, Salgado R. Pitfalls in assessing stromal tumor infiltrating lymphocytes (sTILs) in breast cancer. NPJ Breast Cancer 2020; 6:17. [PMID: 32411819 PMCID: PMC7217863 DOI: 10.1038/s41523-020-0156-0] [Citation(s) in RCA: 89] [Impact Index Per Article: 22.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2019] [Accepted: 03/02/2020] [Indexed: 02/08/2023] Open
Abstract
Stromal tumor-infiltrating lymphocytes (sTILs) are important prognostic and predictive biomarkers in triple-negative (TNBC) and HER2-positive breast cancer. Incorporating sTILs into clinical practice necessitates reproducible assessment. Previously developed standardized scoring guidelines have been widely embraced by the clinical and research communities. We evaluated sources of variability in sTIL assessment by pathologists in three previous sTIL ring studies. We identify common challenges and evaluate impact of discrepancies on outcome estimates in early TNBC using a newly-developed prognostic tool. Discordant sTIL assessment is driven by heterogeneity in lymphocyte distribution. Additional factors include: technical slide-related issues; scoring outside the tumor boundary; tumors with minimal assessable stroma; including lymphocytes associated with other structures; and including other inflammatory cells. Small variations in sTIL assessment modestly alter risk estimation in early TNBC but have the potential to affect treatment selection if cutpoints are employed. Scoring and averaging multiple areas, as well as use of reference images, improve consistency of sTIL evaluation. Moreover, to assist in avoiding the pitfalls identified in this analysis, we developed an educational resource available at www.tilsinbreastcancer.org/pitfalls.
Collapse
Affiliation(s)
- Zuzana Kos
- Department of Pathology, BC Cancer - Vancouver, Vancouver, BC Canada
| | - Elvire Roblin
- Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, Villejuif, France
- Oncostat U1018, Inserm, University Paris-Saclay, labeled Ligue Contre le Cancer, Villejuif, France
| | - Rim S. Kim
- National Surgical Adjuvant Breast and Bowel Project (NSABP)/NRG Oncology, Pittsburgh, PA USA
| | - Stefan Michiels
- Department of Biostatistics and Epidemiology, Gustave Roussy, University Paris-Saclay, Villejuif, France
- Oncostat U1018, Inserm, University Paris-Saclay, labeled Ligue Contre le Cancer, Villejuif, France
| | - Brandon D. Gallas
- Division of Imaging, Diagnostics, and Software Reliability (DIDSR); Office of Science and Engineering Laboratories (OSEL); Center for Devices and Radiological Health (CDRH), US Food and Drug Administration (US FDA), Silver Spring, MD USA
| | - Weijie Chen
- Division of Imaging, Diagnostics, and Software Reliability (DIDSR); Office of Science and Engineering Laboratories (OSEL); Center for Devices and Radiological Health (CDRH), US Food and Drug Administration (US FDA), Silver Spring, MD USA
| | - Koen K. van de Vijver
- Department of Pathology, University Hospital Antwerp, Antwerp, Belgium
- Department of Pathology, Ghent University Hospital, Cancer Research Institute Ghent (CRIG), Ghent, Belgium
| | - Shom Goel
- The Sir Peter MacCallum Cancer Centre, Melbourne, VIC Australia
- Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria Australia
| | - Sylvia Adams
- Perlmutter Cancer Center, New York University Medical School, New York, NY USA
| | - Sandra Demaria
- Departments of Radiation Oncology and Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, NY USA
| | - Giuseppe Viale
- Department of Pathology, Istituto Europeo di Oncologia, University of Milan, Milan, Italy
| | - Torsten O. Nielsen
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, Canada
| | - Sunil S. Badve
- Department of Pathology and Laboratory Medicine, Indiana University, Indianapolis, USA
| | - W. Fraser Symmans
- Department of Pathology, The University of Texas M.D. Anderson Cancer Center, Houston, TX USA
| | - Christos Sotiriou
- Department of Medical Oncology, Institut Jules Bordet, Université Libre de Bruxelles, Brussels, Belgium
| | - David L. Rimm
- Department of Pathology, Yale School of Medicine, New Haven, CT USA
| | - Stephen Hewitt
- Laboratory of Pathology, National Cancer Institute, NIH, Bethesda, MD USA
| | - Carsten Denkert
- Institute of Pathology, Universitätsklinikum Gießen und Marburg GmbH, Standort Marburg and Philipps-Universität Marburg, Marburg, Germany
| | | | - Stephen J. Luen
- Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria Australia
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC Australia
| | - John M. S. Bartlett
- Ontario Institute for Cancer Research, Toronto, ON Canada
- University of Edinburgh Cancer Research Centre, Edinburgh, UK
| | - Peter Savas
- Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria Australia
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC Australia
| | - Giancarlo Pruneri
- Department of Pathology, IRCCS Fondazione Instituto Nazionale Tumori and University of Milan, School of Medicine, Milan, Italy
| | - Deborah A. Dillon
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA USA
- Department of Pathology, Dana Farber Cancer Institute, Boston, MA USA
| | - Maggie Chon U. Cheang
- Institute of Cancer Research Clinical Trials and Statistics Unit, The Institute of Cancer Research, Surrey, UK
| | - Andrew Tutt
- Breast Cancer Now Toby Robins Research Centre, The Institute of Cancer Research, London, UK
| | | | - Marleen Kok
- Department of Medical Oncology and Division of Tumor Biology & Immunology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Hugo M. Horlings
- Department of Pathology, University Hospital Antwerp, Antwerp, Belgium
- Division of Molecular Pathology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH USA
- Louis Stokes Cleveland Veterans Affairs Medical Center, Cleveland, OH USA
| | - Jeroen van der Laak
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, Nijmegen, Netherlands
| | - Francesco Ciompi
- Computational Pathology Group, Department of Pathology, Radboud University Medical Center, Nijmegen, Netherlands
| | | | - Enrique Bellolio
- Departamento de Anatomía Patológica, Universidad de La Frontera, Temuco, Chile
| | | | - Stephen B. Fox
- The Sir Peter MacCallum Cancer Centre, Melbourne, VIC Australia
- Department of Pathology, Peter MacCallum Cancer Centre Department of Pathology, Melbourne, VIC Australia
| | | | - Giuseppe Floris
- KU Leuven- Univerisity of Leuven, Department of Imaging and Pathology, Laboratory of Translational Cell & Tissue Research and KU Leuven- University Hospitals Leuven, Department of Pathology, Leuven, Belgium
| | - Jan Hudeček
- Department of Research IT, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Leonie Voorwerk
- Division of Tumor Biology & Immunology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | | | - Denis Larsimont
- Department of Pathology, Jules Bordet Institute, Brussels, Belgium
| | | | | | - Lajos Pusztai
- Department of Internal Medicine, Section of Medical Oncology, Yale Cancer Center, Yale School of Medicine, New Haven, CT USA
| | - Anna Ehinger
- Department of Clinical Genetics and Pathology, Skåne University Hospital, Lund University, Lund, Sweden
| | - Wentao Yang
- Department of Pathology, Fudan University Shanghai Cancer Centre, Shanghai, China
| | - Khalid AbdulJabbar
- Centre for Evolution and Cancer; Division of Molecular Pathology, The Institute of Cancer Research, London, UK
| | - Yinyin Yuan
- Centre for Evolution and Cancer; Division of Molecular Pathology, The Institute of Cancer Research, London, UK
| | - Rajendra Singh
- Icahn School of Medicine at Mt. Sinai, New York, NY 10029 USA
| | - Crispin Hiley
- Cancer Research UK Lung Cancer Centre of Excellence, University College London Cancer Institute, University College London, London, UK
| | - Maise al Bakir
- Cancer Research UK Lung Cancer Centre of Excellence, University College London Cancer Institute, University College London, London, UK
| | - Alexander J. Lazar
- Departments of Pathology, Genomic Medicine, Dermatology, and Translational Molecular Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Stephen Naber
- Department of Pathology and Laboratory Medicine, Tufts Medical Center, Boston, USA
| | - Stephan Wienert
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Institute of Pathology, Charitéplatz 1, 10117 Berlin, Germany
| | - Miluska Castillo
- Department of Medical Oncology and Research, Instituto Nacional de Enfermedades Neoplasicas, Lima, 15038 Peru
| | | | - Maria-Vittoria Dieci
- Medical Oncology 2, Istituto Oncologico Veneto IOV - IRCCS, Padova, Italy
- Department of Surgery, Oncology and Gastroenterology, University of Padova, Padova, Italy
| | - Fabrice André
- Department of Medical Oncology, Institut Gustave Roussy, Villejuif, France
| | - Charles Swanton
- Cancer Research UK Lung Cancer Centre of Excellence, University College London Cancer Institute, University College London, London, UK
- Francis Crick Institute, Midland Road, London, UK
| | - Jorge Reis-Filho
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY USA
- Human Oncology and Pathogenesis Program, Memorial Sloan Kettering Cancer Center, New York, NY USA
| | - Joseph Sparano
- Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, NY USA
| | - Eva Balslev
- Department of Pathology, Herlev and Gentofte Hospital, Herlev, Denmark
| | - I-Chun Chen
- Department of Oncology, National Taiwan University Cancer Center, Taipei, Taiwan
- Department of Oncology, National Taiwan University Hospital, Taipei, Taiwan
- Graduate Institute of Oncology, College of Medicine, National Taiwan University, Taipei, Taiwan
| | | | - Katherine Pogue-Geile
- National Surgical Adjuvant Breast and Bowel Project (NSABP)/NRG Oncology, Pittsburgh, PA USA
| | - Kim R. M. Blenman
- Department of Internal Medicine, Section of Medical Oncology, Yale Cancer Center, Yale School of Medicine, New Haven, CT USA
| | | | - Stuart Schnitt
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA USA
| | - Sunil R. Lakhani
- The University of Queensland Centre for Clinical Research and Pathology Queensland, Brisbane, QLD Australia
| | - Anne Vincent-Salomon
- Institut Curie, Paris Sciences Lettres Université, Inserm U934, Department of Pathology, Paris, France
| | - Federico Rojo
- Pathology Department, Instituto de Investigación Sanitaria Fundación Jiménez Díaz (IIS-FJD) - CIBERONC, Madrid, Spain
- GEICAM-Spanish Breast Cancer Research Group, Madrid, Spain
| | - Jeremy P. Braybrooke
- Nuffield Department of Population Health, University of Oxford, Oxford and Department of Medical Oncology, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
| | - Matthew G. Hanna
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY USA
| | - M. Teresa Soler-Monsó
- Department of Pathology, Bellvitge University Hospital, IDIBELL. Breast Unit. Catalan Institut of Oncology. L ‘Hospitalet del Llobregat’, Barcelona, 08908 Catalonia Spain
| | - Daniel Bethmann
- University Hospital Halle (Saale), Institute of Pathology, Halle (Saale), Germany
| | - Carlos A. Castaneda
- Department of Medical Oncology and Research, Instituto Nacional de Enfermedades Neoplasicas, Lima, 15038 Peru
| | - Karen Willard-Gallo
- Molecular Immunology Unit, Institut Jules Bordet, Universitè Libre de Bruxelles, Brussels, Belgium
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University, Atlanta, GA USA
| | - Huang-Chun Lien
- Department of Pathology, National Taiwan University Hospital, Taipei, Taiwan
| | - Susan Fineberg
- Department of Pathology, Montefiore Medical Center and the Albert Einstein College of Medicine, Bronx, NY USA
| | - Jeppe Thagaard
- DTU Compute, Department of Applied Mathematics, Technical University of Denmark; Visiopharm A/S, Hørsholm, Denmark
| | - Laura Comerma
- GEICAM-Spanish Breast Cancer Research Group, Madrid, Spain
- Pathology Department, Hospital del Mar, Parc de Salut Mar, Barcelona, Spain
| | - Paula Gonzalez-Ericsson
- Breast Cancer Program, Vanderbilt-Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, TN USA
| | - Edi Brogi
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY USA
| | - Sherene Loi
- Peter MacCallum Department of Oncology, University of Melbourne, Melbourne, Victoria Australia
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC Australia
| | - Joel Saltz
- Biomedical Informatics Department, Stony Brook University, Stony Brook, NY USA
| | - Frederick Klaushen
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Lee Cooper
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL USA
| | - Mohamed Amgad
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA USA
| | - David A. Moore
- Department of Pathology, UCL Cancer Institute, UCL, London, UK
- University College Hospitals NHS Trust, London, UK
| | - Roberto Salgado
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Melbourne, VIC Australia
- Department of Pathology, GZA-ZNA, Antwerp, Belgium
| |
Collapse
|
37
|
Amgad M, Stovgaard ES, Balslev E, Thagaard J, Chen W, Dudgeon S, Sharma A, Kerner JK, Denkert C, Yuan Y, AbdulJabbar K, Wienert S, Savas P, Voorwerk L, Beck AH, Madabhushi A, Hartman J, Sebastian MM, Horlings HM, Hudeček J, Ciompi F, Moore DA, Singh R, Roblin E, Balancin ML, Mathieu MC, Lennerz JK, Kirtani P, Chen IC, Braybrooke JP, Pruneri G, Demaria S, Adams S, Schnitt SJ, Lakhani SR, Rojo F, Comerma L, Badve SS, Khojasteh M, Symmans WF, Sotiriou C, Gonzalez-Ericsson P, Pogue-Geile KL, Kim RS, Rimm DL, Viale G, Hewitt SM, Bartlett JMS, Penault-Llorca F, Goel S, Lien HC, Loibl S, Kos Z, Loi S, Hanna MG, Michiels S, Kok M, Nielsen TO, Lazar AJ, Bago-Horvath Z, Kooreman LFS, van der Laak JAWM, Saltz J, Gallas BD, Kurkure U, Barnes M, Salgado R, Cooper LAD. Report on computational assessment of Tumor Infiltrating Lymphocytes from the International Immuno-Oncology Biomarker Working Group. NPJ Breast Cancer 2020; 6:16. [PMID: 32411818 PMCID: PMC7217824 DOI: 10.1038/s41523-020-0154-2] [Citation(s) in RCA: 74] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Accepted: 02/18/2020] [Indexed: 02/07/2023] Open
Abstract
Assessment of tumor-infiltrating lymphocytes (TILs) is increasingly recognized as an integral part of the prognostic workflow in triple-negative (TNBC) and HER2-positive breast cancer, as well as many other solid tumors. This recognition has come about thanks to standardized visual reporting guidelines, which helped to reduce inter-reader variability. Now, there are ripe opportunities to employ computational methods that extract spatio-morphologic predictive features, enabling computer-aided diagnostics. We detail the benefits of computational TILs assessment, the readiness of TILs scoring for computational assessment, and outline considerations for overcoming key barriers to clinical translation in this arena. Specifically, we discuss: 1. ensuring computational workflows closely capture visual guidelines and standards; 2. challenges and thoughts standards for assessment of algorithms including training, preanalytical, analytical, and clinical validation; 3. perspectives on how to realize the potential of machine learning models and to overcome the perceptual and practical limits of visual scoring.
Collapse
Affiliation(s)
- Mohamed Amgad
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA USA
| | | | - Eva Balslev
- Department of Pathology, Herlev and Gentofte Hospital, University of Copenhagen, Herlev, Denmark
| | - Jeppe Thagaard
- DTU Compute, Department of Applied Mathematics, Technical University of Denmark, Lyngby, Denmark
- Visiopharm A/S, Hørsholm, Denmark
| | - Weijie Chen
- FDA/CDRH/OSEL/Division of Imaging, Diagnostics, and Software Reliability, Silver Spring, MD USA
| | - Sarah Dudgeon
- FDA/CDRH/OSEL/Division of Imaging, Diagnostics, and Software Reliability, Silver Spring, MD USA
| | - Ashish Sharma
- Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA USA
| | | | - Carsten Denkert
- Institut für Pathologie, Universitätsklinikum Gießen und Marburg GmbH, Standort Marburg, Philipps-Universität Marburg, Marburg, Germany
- Institute of Pathology, Philipps-University Marburg, Marburg, Germany
- German Cancer Consortium (DKTK), Partner Site Charité, Berlin, Germany
| | - Yinyin Yuan
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK
- Division of Molecular Pathology, The Institute of Cancer Research, London, UK
| | - Khalid AbdulJabbar
- Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK
- Division of Molecular Pathology, The Institute of Cancer Research, London, UK
| | - Stephan Wienert
- Institut für Pathologie, Universitätsklinikum Gießen und Marburg GmbH, Standort Marburg, Philipps-Universität Marburg, Marburg, Germany
| | - Peter Savas
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Victoria, Australia
- Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia
| | - Leonie Voorwerk
- Department of Tumor Biology & Immunology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Anant Madabhushi
- Case Western Reserve University, Department of Biomedical Engineering, Cleveland, OH USA
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH USA
| | - Johan Hartman
- Department of Oncology and Pathology, Karolinska Institutet and University Hospital, Solna, Sweden
| | - Manu M. Sebastian
- Departments of Epigenetics and Molecular Carcinogenesis, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Hugo M. Horlings
- Division of Molecular Pathology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Jan Hudeček
- Department of Research IT, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - David A. Moore
- Department of Pathology, UCL Cancer Institute, London, UK
| | - Rajendra Singh
- Department of Pathology and Laboratory Medicine, Icahn School of Medicine at Mount Sinai, New York, NY USA
| | - Elvire Roblin
- Université Paris-Saclay, Univ. Paris-Sud, Villejuif, France
| | - Marcelo Luiz Balancin
- Department of Pathology, Faculty of Medicine, University of São Paulo, São Paulo, Brazil
| | - Marie-Christine Mathieu
- Department of Medical Biology and Pathology, Gustave Roussy Cancer Campus, Villejuif, France
| | - Jochen K. Lennerz
- Department of Pathology, Massachusetts General Hospital, Boston, MA USA
| | - Pawan Kirtani
- Department of Histopathology, Manipal Hospitals Dwarka, New Delhi, India
| | - I-Chun Chen
- Department of Oncology, National Taiwan University Cancer Center, Taipei, Taiwan
| | - Jeremy P. Braybrooke
- Nuffield Department of Population Health, University of Oxford, Oxford, UK
- Department of Medical Oncology, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
| | - Giancarlo Pruneri
- Pathology Department, Fondazione IRCCS Istituto Nazionale Tumori and University of Milan, School of Medicine, Milan, Italy
| | | | - Sylvia Adams
- Laura and Isaac Perlmutter Cancer Center, NYU Langone Medical Center, New York, NY USA
| | - Stuart J. Schnitt
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA USA
| | - Sunil R. Lakhani
- The University of Queensland Centre for Clinical Research and Pathology Queensland, Brisbane, Australia
| | - Federico Rojo
- Pathology Department, CIBERONC-Instituto de Investigación Sanitaria Fundación Jiménez Díaz (IIS-FJD), Madrid, Spain
- GEICAM-Spanish Breast Cancer Research Group, Madrid, Spain
| | - Laura Comerma
- Pathology Department, CIBERONC-Instituto de Investigación Sanitaria Fundación Jiménez Díaz (IIS-FJD), Madrid, Spain
- GEICAM-Spanish Breast Cancer Research Group, Madrid, Spain
| | - Sunil S. Badve
- Department of Pathology and Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN USA
| | | | - W. Fraser Symmans
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Christos Sotiriou
- Breast Cancer Translational Research Laboratory, Institut Jules Bordet, Université Libre de Bruxelles (ULB), Brussels, Belgium
- ULB-Cancer Research Center (U-CRC) Université Libre de Bruxelles, Brussels, Belgium
| | - Paula Gonzalez-Ericsson
- Breast Cancer Program, Vanderbilt-Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, TN USA
| | | | | | - David L. Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT USA
| | - Giuseppe Viale
- Department of Pathology, IEO, European Institute of Oncology IRCCS & State University of Milan, Milan, Italy
| | - Stephen M. Hewitt
- Laboratory of Pathology, National Cancer Institute, National Institutes of Health, Bethesda, MD USA
| | - John M. S. Bartlett
- Ontario Institute for Cancer Research, Toronto, ON Canada
- Edinburgh Cancer Research Centre, Western General Hospital, Edinburgh, UK
| | - Frédérique Penault-Llorca
- Department of Pathology and Molecular Pathology, Centre Jean Perrin, Clermont-Ferrand, France
- UMR INSERM 1240, Universite Clermont Auvergne, Clermont-Ferrand, France
| | - Shom Goel
- Victorian Comprehensive Cancer Centre building, Peter MacCallum Cancer Centre, Melbourne, Victoria Australia
| | - Huang-Chun Lien
- Department of Pathology, National Taiwan University Hospital, Taipei, Taiwan
| | - Sibylle Loibl
- German Breast Group, c/o GBG-Forschungs GmbH, Neu-Isenburg, Germany
| | - Zuzana Kos
- Department of Pathology, BC Cancer, Vancouver, British Columbia Canada
| | - Sherene Loi
- Sir Peter MacCallum Department of Oncology, University of Melbourne, Parkville, Australia
- Peter MacCallum Cancer Centre, Melbourne, Australia
| | - Matthew G. Hanna
- Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY USA
| | - Stefan Michiels
- Gustave Roussy, Universite Paris-Saclay, Villejuif, France
- Université Paris-Sud, Institut National de la Santé et de la Recherche Médicale, Villejuif, France
| | - Marleen Kok
- Division of Molecular Oncology & Immunology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
- Department of Medical Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | | | - Alexander J. Lazar
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
- Department of Genomic Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX USA
- Department of Translational Molecular Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
- Department of Dermatology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | | | - Loes F. S. Kooreman
- GROW - School for Oncology and Developmental Biology, Maastricht University Medical Centre, Maastricht, The Netherlands
- Department of Pathology, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Jeroen A. W. M. van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY USA
| | - Brandon D. Gallas
- FDA/CDRH/OSEL/Division of Imaging, Diagnostics, and Software Reliability, Silver Spring, MD USA
| | - Uday Kurkure
- Roche Tissue Diagnostics, Digital Pathology, Santa Clara, CA USA
| | - Michael Barnes
- Roche Diagnostics Information Solutions, Belmont, CA USA
| | - Roberto Salgado
- Division of Research and Cancer Medicine, Peter MacCallum Cancer Centre, University of Melbourne, Victoria, Australia
- Department of Pathology, GZA-ZNA Ziekenhuizen, Antwerp, Belgium
| | - Lee A. D. Cooper
- Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, IL USA
| |
Collapse
|
38
|
Bokhorst JM, Blank A, Lugli A, Zlobec I, Dawson H, Vieth M, Rijstenberg LL, Brockmoeller S, Urbanowicz M, Flejou JF, Kirsch R, Ciompi F, van der Laak JAWM, Nagtegaal ID. Assessment of individual tumor buds using keratin immunohistochemistry: moderate interobserver agreement suggests a role for machine learning. Mod Pathol 2020; 33:825-833. [PMID: 31844269 PMCID: PMC7190566 DOI: 10.1038/s41379-019-0434-2] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2019] [Revised: 11/07/2019] [Accepted: 11/23/2019] [Indexed: 02/07/2023]
Abstract
Tumor budding is a promising and cost-effective biomarker with strong prognostic value in colorectal cancer. However, challenges related to interobserver variability persist. Such variability may be reduced by immunohistochemistry and computer-aided tumor bud selection. Development of computer algorithms for this purpose requires unequivocal examples of individual tumor buds. As such, we undertook a large-scale, international, and digital observer study on individual tumor bud assessment. From a pool of 46 colorectal cancer cases with tumor budding, 3000 tumor bud candidates were selected, largely based on digital image analysis algorithms. For each candidate bud, an image patch (size 256 × 256 µm) was extracted from a pan cytokeratin-stained whole-slide image. Members of an International Tumor Budding Consortium (n = 7) were asked to categorize each candidate as either (1) tumor bud, (2) poorly differentiated cluster, or (3) neither, based on current definitions. Agreement was assessed with Cohen's and Fleiss Kappa statistics. Fleiss Kappa showed moderate overall agreement between observers (0.42 and 0.51), while Cohen's Kappas ranged from 0.25 to 0.63. Complete agreement by all seven observers was present for only 34% of the 3000 tumor bud candidates, while 59% of the candidates were agreed on by at least five of the seven observers. Despite reports of moderate-to-substantial agreement with respect to tumor budding grade, agreement with respect to individual pan cytokeratin-stained tumor buds is moderate at most. A machine learning approach may prove especially useful for a more robust assessment of individual tumor buds.
Collapse
Affiliation(s)
- J. M. Bokhorst
- grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, Netherlands
| | - A. Blank
- grid.5734.50000 0001 0726 5157University of Bern, Bern, Switzerland
| | - A. Lugli
- grid.5734.50000 0001 0726 5157University of Bern, Bern, Switzerland
| | - I. Zlobec
- grid.5734.50000 0001 0726 5157University of Bern, Bern, Switzerland
| | - H. Dawson
- grid.5734.50000 0001 0726 5157University of Bern, Bern, Switzerland
| | - M. Vieth
- grid.7384.80000 0004 0467 6972University of Bayreuth, Bayreuth, Germany
| | - L. L. Rijstenberg
- grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, Netherlands
| | - S. Brockmoeller
- grid.9909.90000 0004 1936 8403University of Leeds, Leeds, UK
| | - M. Urbanowicz
- grid.418936.10000 0004 0610 0854EORTC Translational Research Unit, Brussels, Belgium
| | - J. F. Flejou
- grid.412370.30000 0004 1937 1100Saint-Antoine Hospital, Paris, France
| | - R. Kirsch
- grid.17063.330000 0001 2157 2938University of Toronto, Toronto, Canada
| | - F. Ciompi
- grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, Netherlands
| | - J. A. W. M. van der Laak
- grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, Netherlands ,grid.5640.70000 0001 2162 9922Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | - I. D. Nagtegaal
- grid.10417.330000 0004 0444 9382Radboud University Medical Center, Nijmegen, Netherlands
| |
Collapse
|
39
|
Tellez D, Litjens G, Bándi P, Bulten W, Bokhorst JM, Ciompi F, van der Laak J. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Med Image Anal 2019; 58:101544. [DOI: 10.1016/j.media.2019.101544] [Citation(s) in RCA: 179] [Impact Index Per Article: 35.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2019] [Revised: 06/25/2019] [Accepted: 08/19/2019] [Indexed: 11/17/2022]
|
40
|
Affiliation(s)
- Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center Nijmegen, Nijmegen, the Netherlands. .,Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center Nijmegen, Nijmegen, the Netherlands
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center Nijmegen, Nijmegen, the Netherlands
| |
Collapse
|
41
|
Swiderska-Chadaj Z, Pinckaers H, van Rijthoven M, Balkenhol M, Melnikova M, Geessink O, Manson Q, Sherman M, Polonia A, Parry J, Abubakar M, Litjens G, van der Laak J, Ciompi F. Learning to detect lymphocytes in immunohistochemistry with deep learning. Med Image Anal 2019; 58:101547. [PMID: 31476576 DOI: 10.1016/j.media.2019.101547] [Citation(s) in RCA: 65] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2019] [Revised: 08/12/2019] [Accepted: 08/20/2019] [Indexed: 12/17/2022]
Abstract
The immune system is of critical importance in the development of cancer. The evasion of destruction by the immune system is one of the emerging hallmarks of cancer. We have built a dataset of 171,166 manually annotated CD3+ and CD8+ cells, which we used to train deep learning algorithms for automatic detection of lymphocytes in histopathology images to better quantify immune response. Moreover, we investigate the effectiveness of four deep learning based methods when different subcompartments of the whole-slide image are considered: normal tissue areas, areas with immune cell clusters, and areas containing artifacts. We have compared the proposed methods in breast, colon and prostate cancer tissue slides collected from nine different medical centers. Finally, we report the results of an observer study on lymphocyte quantification, which involved four pathologists from different medical centers, and compare their performance with the automatic detection. The results give insights on the applicability of the proposed methods for clinical use. U-Net obtained the highest performance with an F1-score of 0.78 and the highest agreement with manual evaluation (κ=0.72), whereas the average pathologists agreement with reference standard was κ=0.64. The test set and the automatic evaluation procedure are publicly available at lyon19.grand-challenge.org.
Collapse
Affiliation(s)
| | - Hans Pinckaers
- Department of Pathology, Radboud University Medical Center, The Netherlands
| | - Mart van Rijthoven
- Department of Pathology, Radboud University Medical Center, The Netherlands
| | | | - Margarita Melnikova
- Department of Pathology, Radboud University Medical Center, The Netherlands; Department of Clinical Medicine, Aarhus University, Denmark; Institute of Pathology, Randers Regional Hospital, Denmark
| | - Oscar Geessink
- Department of Pathology, Radboud University Medical Center, The Netherlands
| | - Quirine Manson
- Department of Pathology, University Medical Center, Utrecht, The Netherlands
| | | | - Antonio Polonia
- Institute of Molecular Pathology and Immunology, University of Porto, Porto, Portugal
| | - Jeremy Parry
- Fiona Stanley Hospital, Murdoch, Perth, Western Australia
| | - Mustapha Abubakar
- Integrative Tumor Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, USA
| | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, The Netherlands
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, The Netherlands; Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden
| | - Francesco Ciompi
- Department of Pathology, Radboud University Medical Center, The Netherlands
| |
Collapse
|
42
|
Litjens G, Ciompi F, Wolterink JM, de Vos BD, Leiner T, Teuwen J, Išgum I. State-of-the-Art Deep Learning in Cardiovascular Image Analysis. JACC Cardiovasc Imaging 2019; 12:1549-1565. [DOI: 10.1016/j.jcmg.2019.06.009] [Citation(s) in RCA: 141] [Impact Index Per Article: 28.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/01/2018] [Revised: 05/13/2019] [Accepted: 06/13/2019] [Indexed: 02/07/2023]
|
43
|
Balkenhol MCA, Bult P, Tellez D, Vreuls W, Clahsen PC, Ciompi F, van der Laak JAWM. Deep learning and manual assessment show that the absolute mitotic count does not contain prognostic information in triple negative breast cancer. Cell Oncol (Dordr) 2019; 42:555-569. [DOI: 10.1007/s13402-019-00445-z] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/25/2019] [Indexed: 12/12/2022] Open
|
44
|
Balocco S, Ciompi F, Rigla J, Carrillo X, Mauri J, Radeva P. Assessment of intracoronary stent location and extension in intravascular ultrasound sequences. Med Phys 2018; 46:484-493. [PMID: 30383304 DOI: 10.1002/mp.13273] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2018] [Revised: 10/11/2018] [Accepted: 10/12/2018] [Indexed: 11/06/2022] Open
Abstract
PURPOSE An intraluminal coronary stent is a metal scaffold deployed in a stenotic artery during percutaneous coronary intervention (PCI). In order to have an effective deployment, a stent should be optimally placed with regard to anatomical structures such as bifurcations and stenoses. Intravascular ultrasound (IVUS) is a catheter-based imaging technique generally used for PCI guiding and assessing the correct placement of the stent. A novel approach that automatically detects the boundaries and the position of the stent along the IVUS pullback is presented. Such a technique aims at optimizing the stent deployment. METHODS The method requires the identification of the stable frames of the sequence and the reliable detection of stent struts. Using these data, a measure of likelihood for a frame to contain a stent is computed. Then, a robust binary representation of the presence of the stent in the pullback is obtained applying an iterative and multiscale quantization of the signal to symbols using the Symbolic Aggregate approXimation algorithm. RESULTS The technique was extensively validated on a set of 103 IVUS of sequences of in vivo coronary arteries containing metallic and bioabsorbable stents acquired through an international multicentric collaboration across five clinical centers. The method was able to detect the stent position with an overall F-measure of 86.4%, a Jaccard index score of 75% and a mean distance of 2.5 mm from manually annotated stent boundaries, and in bioabsorbable stents with an overall F-measure of 88.6%, a Jaccard score of 77.7 and a mean distance of 1.5 mm from manually annotated stent boundaries. Additionally, a map indicating the distance between the lumen and the stent along the pullback is created in order to show the angular sectors of the sequence in which the malapposition is present. CONCLUSIONS Results obtained comparing the automatic results vs the manual annotation of two observers shows that the method approaches the interobserver variability. Similar performances are obtained on both metallic and bioabsorbable stents, showing the flexibility and robustness of the method.
Collapse
Affiliation(s)
- Simone Balocco
- Department of Matematics and Informatics, University of Barcelona, Gran Via 585, 08007, Barcelona, Spain.,Computer Vision Center, 08193, Bellaterra, Spain
| | - Francesco Ciompi
- Department of Pathology University Medical Center, Nijmegen, The Netherlands.,Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Xavier Carrillo
- University Hospital Germans Trias i Pujol, 08916, Badalona, Spain
| | - Josepa Mauri
- University Hospital Germans Trias i Pujol, 08916, Badalona, Spain
| | - Petia Radeva
- Department of Matematics and Informatics, University of Barcelona, Gran Via 585, 08007, Barcelona, Spain.,Computer Vision Center, 08193, Bellaterra, Spain
| |
Collapse
|
45
|
Tellez D, Balkenhol M, Otte-Holler I, van de Loo R, Vogels R, Bult P, Wauters C, Vreuls W, Mol S, Karssemeijer N, Litjens G, van der Laak J, Ciompi F. Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks. IEEE Trans Med Imaging 2018; 37:2126-2136. [PMID: 29994086 DOI: 10.1109/tmi.2018.2820199] [Citation(s) in RCA: 92] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by noisy and expensive reference standards established by pathologists, lack of generalization due to staining variation across laboratories, and high computational requirements needed to process gigapixel whole-slide images (WSIs). In this paper, we present a method to train and evaluate CNNs to specifically solve these issues in the context of mitosis detection in breast cancer WSIs. First, by combining image analysis of mitotic activity in phosphohistone-H3 restained slides and registration, we built a reference standard for mitosis detection in entire H&E WSIs requiring minimal manual annotation effort. Second, we designed a data augmentation strategy that creates diverse and realistic H&E stain variations by modifying H&E color channels directly. Using it during training combined with network ensembling resulted in a stain invariant mitosis detector. Third, we applied knowledge distillation to reduce the computational requirements of the mitosis detection ensemble with a negligible loss of performance. The system was trained in a single-center cohort and evaluated in an independent multicenter cohort from the cancer genome atlas on the three tasks of the tumor proliferation assessment challenge. We obtained a performance within the top three best methods for most of the tasks of the challenge.
Collapse
|
46
|
Silva M, Prokop M, Jacobs C, Capretti G, Sverzellati N, Ciompi F, van Ginneken B, Schaefer-Prokop CM, Galeone C, Marchianò A, Pastorino U. Long-Term Active Surveillance of Screening Detected Subsolid Nodules is a Safe Strategy to Reduce Overtreatment. J Thorac Oncol 2018; 13:1454-1463. [PMID: 30026071 DOI: 10.1016/j.jtho.2018.06.013] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2018] [Revised: 06/12/2018] [Accepted: 06/12/2018] [Indexed: 12/17/2022]
Abstract
INTRODUCTION Lung cancer presenting as subsolid nodule (SSN) can show slow growth, hence treating SSN is controversial. Our aim was to determine the long-term outcome of subjects with unresected SSNs in lung cancer screening. METHODS Since 2005, the Multicenter Italian Lung Detection (MILD) screening trial implemented active surveillance for persistent SSN, as opposed to early resection. Presence of SSNs was related to diagnosis of cancer at the site of SSN, elsewhere in the lung, or in the body. The risk of overall mortality and lung cancer mortality was tested by Cox proportional hazards model. RESULTS SSNs were found in 16.9% (389 of 2303) of screenees. During 9.3 ± 1.2 years of follow-up, the hazard ratio of lung cancer diagnosis in subjects with SSN was 6.77 (95% confidence interval: 3.39-13.54), with 73% (22 of 30) of cancers not arising from SSN (median time to diagnosis 52 months from SSN). Lung cancer-specific mortality in subjects with SSN was significantly increased (hazard ratio = 3.80; 95% confidence interval: 1.24-11.65) compared to subjects without lung nodules. Lung cancer arising from SSN did not lead to death within the follow-up period. CONCLUSIONS Subjects with SSN in the MILD cohort showed a high risk of developing lung cancer elsewhere in the lung, with only a minority of cases arising from SSN, and never representing the cause of death. These results show the safety of active surveillance for conservative management of SSN until signs of solid component growth and the need for prolonged follow-up because of high risk of other cancers.
Collapse
Affiliation(s)
- Mario Silva
- Section of Radiology, Unit of Surgical Sciences, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy; Department of Thoracic Surgery, IRCCS Istituto Nazionale Tumori, Milan, Italy.
| | - Mathias Prokop
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, Netherlands
| | - Colin Jacobs
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, Netherlands
| | - Giovanni Capretti
- Section of Radiology, Unit of Surgical Sciences, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy
| | - Nicola Sverzellati
- Section of Radiology, Unit of Surgical Sciences, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy
| | - Francesco Ciompi
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, Netherlands
| | - Bram van Ginneken
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, Netherlands
| | - Cornelia M Schaefer-Prokop
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, Netherlands; Department of Radiology, Meander Medical Center, Amersfoort, Netherlands
| | - Carlotta Galeone
- Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milan, Italy
| | - Alfonso Marchianò
- Department of Radiology, IRCCS Istituto Nazionale Tumori, Milan, Italy
| | - Ugo Pastorino
- Department of Thoracic Surgery, IRCCS Istituto Nazionale Tumori, Milan, Italy
| |
Collapse
|
47
|
Chung K, Ciompi F, Scholten ET, Goo JM, Prokop M, Jacobs C, van Ginneken B, Schaefer-Prokop CM. Visual discrimination of screen-detected persistent from transient subsolid nodules: An observer study. PLoS One 2018; 13:e0191874. [PMID: 29438443 PMCID: PMC5810988 DOI: 10.1371/journal.pone.0191874] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2017] [Accepted: 01/12/2018] [Indexed: 12/18/2022] Open
Abstract
Purpose To evaluate whether, and to which extent, experienced radiologists are able to visually correctly differentiate transient from persistent subsolid nodules from a single CT examination alone and to determine CT morphological features to make this differentiation. Materials and methods We selected 86 transient and 135 persistent subsolid nodules from the National Lung Screening Trial (NLST) database. Four experienced radiologists visually assessed a predefined list of morphological features and gave a final judgment on a continuous scale (0–100). To assess observer performance, area under the receiver operating characteristic (ROC) curve was calculated. Statistical differences of morphological features between transient and persistent lesions were calculated using Chi-square. Inter-observer agreement of morphological features was evaluated by percentage agreement. Results Forty-nine lesions were excluded by at least 2 observers, leaving 172 lesions for analysis. On average observers were able to differentiate transient from persistent subsolid nodules ≥ 10 mm with an area under the curve of 0.75 (95% CI 0.67–0.82). Nodule type, lesion margin, presence of a well-defined border, and pleural retraction showed significant differences between transient and persistent lesions in two observers. Average pair-wise percentage agreement for these features was 81%, 64%, 47% and 89% respectively. Agreement for other morphological features varied from 53% to 95%. Conclusion The visual capacity of experienced radiologists to differentiate persistent and transient subsolid nodules is moderate in subsolid nodules larger than 10 mm. Performance of the visual assessment of CT morphology alone is not sufficient to generally abandon a short-term follow-up for subsolid nodules.
Collapse
Affiliation(s)
- Kaman Chung
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
- * E-mail:
| | - Francesco Ciompi
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Ernst T. Scholten
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jin Mo Goo
- Department of Radiology, Seoul National University College of Medicine, and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea
| | - Mathias Prokop
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Colin Jacobs
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Bram van Ginneken
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Cornelia M. Schaefer-Prokop
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
- Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands
| |
Collapse
|
48
|
Charbonnier JP, Chung K, Scholten ET, van Rikxoort EM, Jacobs C, Sverzellati N, Silva M, Pastorino U, van Ginneken B, Ciompi F. Automatic segmentation of the solid core and enclosed vessels in subsolid pulmonary nodules. Sci Rep 2018; 8:646. [PMID: 29330380 PMCID: PMC5766558 DOI: 10.1038/s41598-017-19101-3] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2017] [Accepted: 12/11/2017] [Indexed: 12/17/2022] Open
Abstract
Subsolid pulmonary nodules are commonly encountered in lung cancer screening and clinical routine. Compared to other nodule types, subsolid nodules are associated with a higher malignancy probability for which the size and mass of the nodule and solid core are important indicators. However, reliably measuring these characteristics on computed tomography (CT) can be hampered by the presence of vessels encompassed by the nodule, since vessels have similar CT attenuation as solid cores. This can affect treatment decisions and patient management. We present a method based on voxel classification to automatically identify vessels and solid cores in given subsolid nodules on CT. Three experts validated our method on 170 screen-detected subsolid nodules from the Multicentric Italian Lung Disease trial. The agreement between the proposed method and the observers was substantial for vessel detection and moderate for solid core detection, which was similar to the inter-observer agreement. We found a relatively high variability in the inter-observer agreement and low method-observer agreements for delineating the borders of vessels and solid cores, illustrating the difficulty of this task. However, 92.4% of the proposed vessel and 80.6% of the proposed solid core segmentations were labeled as usable in clinical practice by the majority of experts.
Collapse
Affiliation(s)
- Jean-Paul Charbonnier
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Kaman Chung
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Ernst T Scholten
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Eva M van Rikxoort
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Colin Jacobs
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Mario Silva
- Section of Radiology, University Hospital of Parma, Parma, Italy
| | - Ugo Pastorino
- Section of Thoracic Surgery, Fondazione IRCCS Istituto Nazionale Tumori, Milano, Italy
| | - Bram van Ginneken
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Francesco Ciompi
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|
49
|
Bokhorst JM, Rijstenberg L, Goudkade D, Nagtegaal I, van der Laak J, Ciompi F. Automatic Detection of Tumor Budding in Colorectal Carcinoma with Deep Learning. Computational Pathology and Ophthalmic Medical Image Analysis 2018. [DOI: 10.1007/978-3-030-00949-6_16] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
|
50
|
Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42:60-88. [PMID: 28778026 DOI: 10.1016/j.media.2017.07.005] [Citation(s) in RCA: 4117] [Impact Index Per Article: 588.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2017] [Revised: 07/24/2017] [Accepted: 07/25/2017] [Indexed: 02/07/2023]
Affiliation(s)
- Geert Litjens
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands.
| | - Thijs Kooi
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | | | - Francesco Ciompi
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Mohsen Ghafoorian
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | | | - Bram van Ginneken
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Clara I Sánchez
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen, The Netherlands
| |
Collapse
|