1
|
Klauschen F, Dippel J, Keyl P, Jurmeister P, Bockmayr M, Mock A, Buchstab O, Alber M, Ruff L, Montavon G, Müller KR. [Explainable artificial intelligence in pathology]. Pathologie (Heidelb) 2024; 45:133-139. [PMID: 38315198 DOI: 10.1007/s00292-024-01308-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/19/2024] [Indexed: 02/07/2024]
Abstract
With the advancements in precision medicine, the demands on pathological diagnostics have increased, requiring standardized, quantitative, and integrated assessments of histomorphological and molecular pathological data. Great hopes are placed in artificial intelligence (AI) methods, which have demonstrated the ability to analyze complex clinical, histological, and molecular data for disease classification, biomarker quantification, and prognosis estimation. This paper provides an overview of the latest developments in pathology AI, discusses the limitations, particularly concerning the black box character of AI, and describes solutions to make decision processes more transparent using methods of so-called explainable AI (XAI).
Collapse
Affiliation(s)
- Frederick Klauschen
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland.
- Institut für Pathologie, Charité - Universitätsmedizin Berlin, Berlin, Deutschland.
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Deutschland.
- Deutsches Krebsforschungszentrum (DKTK/DKFZ), Partnerstandort München, München, Deutschland.
| | - Jonas Dippel
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Deutschland
- Machine Learning Group, Fachbereich Elektrotechnik und Informatik, Technische Universität Berlin, Berlin, Deutschland
| | - Philipp Keyl
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland
| | - Philipp Jurmeister
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland
- Deutsches Krebsforschungszentrum (DKTK/DKFZ), Partnerstandort München, München, Deutschland
| | - Michael Bockmayr
- Institut für Pathologie, Charité - Universitätsmedizin Berlin, Berlin, Deutschland
- Pädiatrische Hämatologie und Onkologie, Universitätsklinikum Hamburg-Eppendorf, Hamburg, Deutschland
- Forschungsinstitut Kinderkrebs-Zentrum Hamburg, Hamburg, Deutschland
| | - Andreas Mock
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland
- Deutsches Krebsforschungszentrum (DKTK/DKFZ), Partnerstandort München, München, Deutschland
| | - Oliver Buchstab
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland
| | - Maximilian Alber
- Institut für Pathologie, Charité - Universitätsmedizin Berlin, Berlin, Deutschland
- Aignostics GmbH, Berlin, Deutschland
| | | | - Grégoire Montavon
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Deutschland
- Machine Learning Group, Fachbereich Elektrotechnik und Informatik, Technische Universität Berlin, Berlin, Deutschland
- Fachbereich Mathematik und Informatik, Freie Universität Berlin, Berlin, Deutschland
| | - Klaus-Robert Müller
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Deutschland.
- Machine Learning Group, Fachbereich Elektrotechnik und Informatik, Technische Universität Berlin, Berlin, Deutschland.
- Department of Artificial Intelligence, Korea University, Seoul, Südkorea.
- Max-Planck-Institut für Informatik, Saarbrücken, Deutschland.
- Machine Learning/Intelligent Data Analysis (IDA), Technische Universität Berlin, Marchstr. 23, 10587, Berlin, Deutschland.
| |
Collapse
|
2
|
Li JJX, Tse GM. Immunocytochemical markers, molecular testing and digital cytopathology for aspiration cytology of metastatic breast carcinoma. Cytopathology 2024; 35:218-225. [PMID: 37985397 DOI: 10.1111/cyt.13333] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 08/28/2023] [Accepted: 11/02/2023] [Indexed: 11/22/2023]
Abstract
Fine-needle aspiration cytology (FNAC) is a versatile diagnostic procedure uniquely suited for tissue biopsy of breast carcinomas and axillary metastases and/or recurrences. With the expanding treatment options and accompanying theragnostic tests, it is crucial to recognize the developments on ancillary testing and digital cytopathology techniques related to aspiration cytology of metastatic breast carcinoma. In this review, we aim to summarize and update the evidence of immunocytochemistry, for the detection of carcinoma cells (epithelial markers), confirmation of breast primary (breast-specific markers), assessment of surrogate immunostains (hormone receptors, ki-67 proliferative index and HER2) and theragnostic biomarkers, with discussion on potential diagnostic pitfalls, followed by the application of molecular tests, and digital cytopathologic techniques for assessing metastatic breast carcinoma in cytology.
Collapse
Affiliation(s)
- Joshua J X Li
- Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong, Shatin, Hong Kong
| | - Gary M Tse
- Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong, Shatin, Hong Kong
| |
Collapse
|
3
|
Truhn D, Tayebi Arasteh S, Saldanha OL, Müller-Franzes G, Khader F, Quirke P, West NP, Gray R, Hutchins GGA, James JA, Loughrey MB, Salto-Tellez M, Brenner H, Brobeil A, Yuan T, Chang-Claude J, Hoffmeister M, Foersch S, Han T, Keil S, Schulze-Hagen M, Isfort P, Bruners P, Kaissis G, Kuhl C, Nebelung S, Kather JN. Encrypted federated learning for secure decentralized collaboration in cancer image analysis. Med Image Anal 2024; 92:103059. [PMID: 38104402 PMCID: PMC10804934 DOI: 10.1016/j.media.2023.103059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 04/28/2023] [Accepted: 12/05/2023] [Indexed: 12/19/2023]
Abstract
Artificial intelligence (AI) has a multitude of applications in cancer research and oncology. However, the training of AI systems is impeded by the limited availability of large datasets due to data protection requirements and other regulatory obstacles. Federated and swarm learning represent possible solutions to this problem by collaboratively training AI models while avoiding data transfer. However, in these decentralized methods, weight updates are still transferred to the aggregation server for merging the models. This leaves the possibility for a breach of data privacy, for example by model inversion or membership inference attacks by untrusted servers. Somewhat-homomorphically-encrypted federated learning (SHEFL) is a solution to this problem because only encrypted weights are transferred, and model updates are performed in the encrypted space. Here, we demonstrate the first successful implementation of SHEFL in a range of clinically relevant tasks in cancer image analysis on multicentric datasets in radiology and histopathology. We show that SHEFL enables the training of AI models which outperform locally trained models and perform on par with models which are centrally trained. In the future, SHEFL can enable multiple institutions to co-train AI models without forsaking data governance and without ever transmitting any decryptable data to untrusted servers.
Collapse
Affiliation(s)
- Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany.
| | - Soroosh Tayebi Arasteh
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Oliver Lester Saldanha
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Gustav Müller-Franzes
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Firas Khader
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Philip Quirke
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Nicholas P West
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Richard Gray
- Clinical Trial Service Unit, University of Oxford, Oxford, United Kingdom
| | - Gordon G A Hutchins
- Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom
| | - Jacqueline A James
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, United Kingdom; Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast, United Kingdom; The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom
| | - Maurice B Loughrey
- The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom; Department of Cellular Pathology, Belfast Health and Social Care Trust, Belfast, United Kingdom; Centre for Public Health, Queen's University Belfast, Belfast, United Kingdom
| | - Manuel Salto-Tellez
- Precision Medicine Centre of Excellence, Health Sciences Building, The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, Belfast, United Kingdom; Regional Molecular Diagnostic Service, Belfast Health and Social Care Trust, Belfast, United Kingdom; The Patrick G Johnston Centre for Cancer Research, Queen's University Belfast, United Kingdom
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany; German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Brobeil
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany; Tissue Bank, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Tanwei Yuan
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany; Medical Faculty Heidelberg, Heidelberg University, Heidelberg, Germany
| | - Jenny Chang-Claude
- Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; Division of Cancer Epidemiology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sebastian Foersch
- Institute of Pathology, University Medical Center Mainz, Mainz, Germany
| | - Tianyu Han
- Physics of Molecular Imaging Systems, Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany
| | - Sebastian Keil
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Maximilian Schulze-Hagen
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Peter Isfort
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Philipp Bruners
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Georgios Kaissis
- Institute of Diagnostic and Interventional Radiology, Technical University of Munich, Munich, Germany; Artificial Intelligence in Medicine and Healthcare, Technical University of Munich, Munich, Germany; Department of Computing, Imperial College London, London, United Kingdom
| | - Christiane Kuhl
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Sven Nebelung
- Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany; Division of Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, United Kingdom; Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|
4
|
Abstract
Flexible electronics have recently gained considerable attention due to their potential to provide new and innovative solutions to a wide range of challenges in various electronic fields. These electronics require specific material properties and performance because they need to be integrated into a variety of surfaces or folded and rolled for newly formatted electronics. Two-dimensional (2D) materials have emerged as promising candidates for flexible electronics due to their unique mechanical, electrical, and optical properties, as well as their compatibility with other materials, enabling the creation of various flexible electronic devices. This article provides a comprehensive review of the progress made in developing flexible electronic devices using 2D materials. In addition, it highlights the key aspects of materials, scalable material production, and device fabrication processes for flexible applications, along with important examples of demonstrations that achieved breakthroughs in various flexible and wearable electronic applications. Finally, we discuss the opportunities, current challenges, potential solutions, and future investigative directions about this field.
Collapse
Affiliation(s)
- Ajit Kumar Katiyar
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Anh Tuan Hoang
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Duo Xu
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Juyeong Hong
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Beom Jin Kim
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Seunghyeon Ji
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Jong-Hyun Ahn
- School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, Republic of Korea
| |
Collapse
|
5
|
Klauschen F, Dippel J, Keyl P, Jurmeister P, Bockmayr M, Mock A, Buchstab O, Alber M, Ruff L, Montavon G, Müller KR. Toward Explainable Artificial Intelligence for Precision Pathology. Annu Rev Pathol 2024; 19:541-570. [PMID: 37871132 DOI: 10.1146/annurev-pathmechdis-051222-113147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
The rapid development of precision medicine in recent years has started to challenge diagnostic pathology with respect to its ability to analyze histological images and increasingly large molecular profiling data in a quantitative, integrative, and standardized way. Artificial intelligence (AI) and, more precisely, deep learning technologies have recently demonstrated the potential to facilitate complex data analysis tasks, including clinical, histological, and molecular data for disease classification; tissue biomarker quantification; and clinical outcome prediction. This review provides a general introduction to AI and describes recent developments with a focus on applications in diagnostic pathology and beyond. We explain limitations including the black-box character of conventional AI and describe solutions to make machine learning decisions more transparent with so-called explainable AI. The purpose of the review is to foster a mutual understanding of both the biomedical and the AI side. To that end, in addition to providing an overview of the relevant foundations in pathology and machine learning, we present worked-through examples for a better practical understanding of what AI can achieve and how it should be done.
Collapse
Affiliation(s)
- Frederick Klauschen
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Jonas Dippel
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
| | - Philipp Keyl
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
| | - Philipp Jurmeister
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Michael Bockmayr
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Department of Pediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Research Institute Children's Cancer Center Hamburg, Hamburg, Germany
| | - Andreas Mock
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Oliver Buchstab
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
| | - Maximilian Alber
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Aignostics, Berlin, Germany
| | | | - Grégoire Montavon
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
- Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany
| | - Klaus-Robert Müller
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
- Department of Artificial Intelligence, Korea University, Seoul, Korea
- Max Planck Institute for Informatics, Saarbrücken, Germany
| |
Collapse
|
6
|
Chatterjee S, Mishra J, Sundram F, Roop P. Towards Personalised Mood Prediction and Explanation for Depression from Biophysical Data. Sensors (Basel) 2023; 24:164. [PMID: 38203024 PMCID: PMC10781272 DOI: 10.3390/s24010164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 11/30/2023] [Accepted: 12/19/2023] [Indexed: 01/12/2024]
Abstract
Digital health applications using Artificial Intelligence (AI) are a promising opportunity to address the widening gap between available resources and mental health needs globally. Increasingly, passively acquired data from wearables are augmented with carefully selected active data from depressed individuals to develop Machine Learning (ML) models of depression based on mood scores. However, most ML models are black box in nature, and hence the outputs are not explainable. Depression is also multimodal, and the reasons for depression may vary significantly between individuals. Explainable and personalised models will thus be beneficial to clinicians to determine the main features that lead to a decline in the mood state of a depressed individual, thus enabling suitable personalised therapy. This is currently lacking. Therefore, this study presents a methodology for developing personalised and accurate Deep Learning (DL)-based predictive mood models for depression, along with novel methods for identifying the key facets that lead to the exacerbation of depressive symptoms. We illustrate our approach by using an existing multimodal dataset containing longitudinal Ecological Momentary Assessments of depression, lifestyle data from wearables and neurocognitive assessments for 14 mild to moderately depressed participants over one month. We develop classification- and regression-based DL models to predict participants' current mood scores-a discrete score given to a participant based on the severity of their depressive symptoms. The models are trained inside eight different evolutionary-algorithm-based optimisation schemes that optimise the model parameters for a maximum predictive performance. A five-fold cross-validation scheme is used to verify the DL model's predictive performance against 10 classical ML-based models, with a model error as low as 6% for some participants. We use the best model from the optimisation process to extract indicators, using SHAP, ALE and Anchors from explainable AI literature to explain why certain predictions are made and how they affect mood. These feature insights can assist health professionals in incorporating personalised interventions into a depressed individual's treatment regimen.
Collapse
Affiliation(s)
- Sobhan Chatterjee
- Department of Electrical, Computer and Software Engineering, Faculty of Engineering, University of Auckland, Auckland 1010, New Zealand
| | - Jyoti Mishra
- Neural Engineering and Translation Labs, Department of Psychiatry, University of California, San Diego, CA 92093, USA;
| | - Frederick Sundram
- Department of Psychological Medicine, Faculty of Medical and Health Sciences, University of Auckland, Auckland 1023, New Zealand;
| | - Partha Roop
- Department of Electrical, Computer and Software Engineering, Faculty of Engineering, University of Auckland, Auckland 1010, New Zealand
| |
Collapse
|
7
|
Sehring J, Dohmen H, Selignow C, Schmid K, Grau S, Stein M, Uhl E, Mukhopadhyay A, Németh A, Amsel D, Acker T. Leveraging Attention-Based Convolutional Neural Networks for Meningioma Classification in Computational Histopathology. Cancers (Basel) 2023; 15:5190. [PMID: 37958364 PMCID: PMC10647687 DOI: 10.3390/cancers15215190] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 10/23/2023] [Accepted: 10/25/2023] [Indexed: 11/15/2023] Open
Abstract
Convolutional neural networks (CNNs) are becoming increasingly valuable tools for advanced computational histopathology, promoting precision medicine through exceptional visual decoding abilities. Meningiomas, the most prevalent primary intracranial tumors, necessitate accurate grading and classification for informed clinical decision-making. Recently, DNA methylation-based molecular classification of meningiomas has proven to be more effective in predicting tumor recurrence than traditional histopathological methods. However, DNA methylation profiling is expensive, labor-intensive, and not widely accessible. Consequently, a digital histology-based prediction of DNA methylation classes would be advantageous, complementing molecular classification. In this study, we developed and rigorously assessed an attention-based multiple-instance deep neural network for predicting meningioma methylation classes using tumor methylome data from 142 (+51) patients and corresponding hematoxylin-eosin-stained histological sections. Pairwise analysis of sample cohorts from three meningioma methylation classes demonstrated high accuracy in two combinations. The performance of our approach was validated using an independent set of 51 meningioma patient samples. Importantly, attention map visualization revealed that the algorithm primarily focuses on tumor regions deemed significant by neuropathologists, offering insights into the decision-making process of the CNN. Our findings highlight the capacity of CNNs to effectively harness phenotypic information from histological sections through computerized images for precision medicine. Notably, this study is the first demonstration of predicting clinically relevant DNA methylome information using computer vision applied to standard histopathology. The introduced AI framework holds great potential in supporting, augmenting, and expediting meningioma classification in the future.
Collapse
Affiliation(s)
- Jannik Sehring
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Hildegard Dohmen
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Carmen Selignow
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Kai Schmid
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Stefan Grau
- Department of Neurosurgery, Hospital Fulda, Pacelliallee 4, D-36043 Fulda, Germany
| | - Marco Stein
- Department of Neurosurgery, University Hospital Gießen, Klinikstr. 33, D-35392 Giessen, Germany
| | - Eberhard Uhl
- Department of Neurosurgery, University Hospital Gießen, Klinikstr. 33, D-35392 Giessen, Germany
| | - Anirban Mukhopadhyay
- Department of Computer Science, Technical University of Darmstadt, Fraunhoferstraße 5, D-64283 Darmstadt, Germany
| | - Attila Németh
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Daniel Amsel
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| | - Till Acker
- Institute of Neuropathology, Justus Liebig University Giessen, Arndtstr. 16, D-35392 Giessen, Germany; (J.S.)
| |
Collapse
|
8
|
Edsjö A, Holmquist L, Geoerger B, Nowak F, Gomon G, Alix-Panabières C, Ploeger C, Lassen U, Le Tourneau C, Lehtiö J, Ott PA, von Deimling A, Fröhling S, Voest E, Klauschen F, Dienstmann R, Alshibany A, Siu LL, Stenzinger A. Precision cancer medicine: Concepts, current practice, and future developments. J Intern Med 2023; 294:455-481. [PMID: 37641393 DOI: 10.1111/joim.13709] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
Precision cancer medicine is a multidisciplinary team effort that requires involvement and commitment of many stakeholders including the society at large. Building on the success of significant advances in precision therapy for oncological patients over the last two decades, future developments will be significantly shaped by improvements in scalable molecular diagnostics in which increasingly complex multilayered datasets require transformation into clinically useful information guiding patient management at fast turnaround times. Adaptive profiling strategies involving tissue- and liquid-based testing that account for the immense plasticity of cancer during the patient's journey and also include early detection approaches are already finding their way into clinical routine and will become paramount. A second major driver is the development of smart clinical trials and trial concepts which, complemented by real-world evidence, rapidly broaden the spectrum of therapeutic options. Tight coordination with regulatory agencies and health technology assessment bodies is crucial in this context. Multicentric networks operating nationally and internationally are key in implementing precision oncology in clinical practice and support developing and improving the ecosystem and framework needed to turn invocation into benefits for patients. The review provides an overview of the diagnostic tools, innovative clinical studies, and collaborative efforts needed to realize precision cancer medicine.
Collapse
Affiliation(s)
- Anders Edsjö
- Department of Clinical Genetics, Pathology and Molecular Diagnostics, Office for Medical Services, Region Skåne, Lund, Sweden
- Division of Pathology, Department of Clinical Sciences, Lund University, Lund, Sweden
- Genomic Medicine Sweden (GMS), Kristianstad, Sweden
| | - Louise Holmquist
- Department of Clinical Genetics, Pathology and Molecular Diagnostics, Office for Medical Services, Region Skåne, Lund, Sweden
- Genomic Medicine Sweden (GMS), Kristianstad, Sweden
| | - Birgit Geoerger
- Department of Pediatric and Adolescent Oncology, Gustave Roussy Cancer Campus, Université Paris-Saclay, Villejuif, France
- INSERM U1015, Gustave Roussy Cancer Campus, Université Paris-Saclay, Villejuif, France
| | | | - Georgy Gomon
- Department of Molecular Oncology and Immunology, The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Department of Medical Oncology, Leiden University Medical Centre, Leiden, The Netherlands
| | - Catherine Alix-Panabières
- Laboratory of Rare Human Circulating Cells, University Medical Center of Montpellier, Montpellier, France
- CREEC, MIVEGEC, University of Montpellier, Montpellier, France
| | - Carolin Ploeger
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany
- Centers for Personalized Medicine (ZPM), Heidelberg, Germany
| | - Ulrik Lassen
- Department of Oncology, Copenhagen University Hospital, Copenhagen, Denmark
| | - Christophe Le Tourneau
- Department of Drug Development and Innovation (D3i), Institut Curie, Paris, France
- INSERM U900 Research Unit, Saint-Cloud, France
- Faculty of Medicine, Paris-Saclay University, Paris, France
| | - Janne Lehtiö
- Department of Oncology Pathology, Karolinska Institutet, Science for Life Laboratory, Stockholm, Sweden
| | - Patrick A Ott
- Department of Medical Oncology, Dana-Farber Cancer Institute and Harvard Medical School, Boston, Massachusetts, USA
| | - Andreas von Deimling
- Clinical Cooperation Unit Neuropathology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- Department of Neuropathology, Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Stefan Fröhling
- Division of Translational Medical Oncology, National Center for Tumor Diseases (NCT) Heidelberg and German Cancer Research Center (DKFZ), Heidelberg, Germany
- German Cancer Consortium (DKTK), Heidelberg, Germany
| | - Emile Voest
- Department of Molecular Oncology and Immunology, The Netherlands Cancer Institute, Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
- Oncode Institute, The Netherlands Cancer Institute, Amsterdam, The Netherlands
| | - Frederick Klauschen
- Institute of Pathology, Charite - Universitätsmedizin Berlin, Berlin, Germany
- German Cancer Consortium (DKTK), Partner Site Berlin, and German Cancer Research Center (DKFZ), Heidelberg, Germany
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Institute of Pathology, Ludwig-Maximilians-University, Munich, Germany
- German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Munich Partner Site, Heidelberg, Germany
| | | | | | - Lillian L Siu
- Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Albrecht Stenzinger
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany
- Centers for Personalized Medicine (ZPM), Heidelberg, Germany
| |
Collapse
|
9
|
Liu Y, Han D, Parwani AV, Li Z. Applications of Artificial Intelligence in Breast Pathology. Arch Pathol Lab Med 2023; 147:1003-1013. [PMID: 36800539 DOI: 10.5858/arpa.2022-0457-ra] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/15/2022] [Indexed: 02/19/2023]
Abstract
CONTEXT.— Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology. OBJECTIVE.— To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes. DATA SOURCES.— We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience. CONCLUSIONS.— With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.
Collapse
Affiliation(s)
- Yueping Liu
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Dandan Han
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| | - Anil V Parwani
- The Department of Pathology, The Ohio State University, Columbus (Parwani, Li)
| | - Zaibo Li
- From the Department of Pathology, The Fourth Hospital of Hebei Medical University, Shijiazhuang, China (Liu, Han)
| |
Collapse
|
10
|
McDonnell KJ. Leveraging the Academic Artificial Intelligence Silecosystem to Advance the Community Oncology Enterprise. J Clin Med 2023; 12:4830. [PMID: 37510945 PMCID: PMC10381436 DOI: 10.3390/jcm12144830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Revised: 07/05/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
Over the last 75 years, artificial intelligence has evolved from a theoretical concept and novel paradigm describing the role that computers might play in our society to a tool with which we daily engage. In this review, we describe AI in terms of its constituent elements, the synthesis of which we refer to as the AI Silecosystem. Herein, we provide an historical perspective of the evolution of the AI Silecosystem, conceptualized and summarized as a Kuhnian paradigm. This manuscript focuses on the role that the AI Silecosystem plays in oncology and its emerging importance in the care of the community oncology patient. We observe that this important role arises out of a unique alliance between the academic oncology enterprise and community oncology practices. We provide evidence of this alliance by illustrating the practical establishment of the AI Silecosystem at the City of Hope Comprehensive Cancer Center and its team utilization by community oncology providers.
Collapse
Affiliation(s)
- Kevin J McDonnell
- Center for Precision Medicine, Department of Medical Oncology & Therapeutics Research, City of Hope Comprehensive Cancer Center, Duarte, CA 91010, USA
| |
Collapse
|
11
|
Reis-Filho JS, Kather JN. Overcoming the challenges to implementation of artificial intelligence in pathology. J Natl Cancer Inst 2023; 115:608-612. [PMID: 36929936 PMCID: PMC10248832 DOI: 10.1093/jnci/djad048] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 03/02/2023] [Accepted: 03/11/2023] [Indexed: 03/18/2023] Open
Abstract
Pathologists worldwide are facing remarkable challenges with increasing workloads and lack of time to provide consistently high-quality patient care. The application of artificial intelligence (AI) to digital whole-slide images has the potential of democratizing the access to expert pathology and affordable biomarkers by supporting pathologists in the provision of timely and accurate diagnosis as well as supporting oncologists by directly extracting prognostic and predictive biomarkers from tissue slides. The long-awaited adoption of AI in pathology, however, has not materialized, and the transformation of pathology is happening at a much slower pace than that observed in other fields (eg, radiology). Here, we provide a critical summary of the developments in digital and computational pathology in the last 10 years, outline key hurdles and ways to overcome them, and provide a perspective for AI-supported precision oncology in the future.
Collapse
Affiliation(s)
- Jorge S Reis-Filho
- Experimental Pathology, Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Jakob Nikolas Kather
- Department of Medicine I, University Hospital and Faculty of Medicine, Technical University Dresden, Dresden, Germany
- Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James’s, University of Leeds, Leeds, UK
| |
Collapse
|
12
|
Mondol RK, Millar EKA, Graham PH, Browne L, Sowmya A, Meijering E. hist2RNA: An Efficient Deep Learning Architecture to Predict Gene Expression from Breast Cancer Histopathology Images. Cancers (Basel) 2023; 15:cancers15092569. [PMID: 37174035 PMCID: PMC10177559 DOI: 10.3390/cancers15092569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 04/23/2023] [Accepted: 04/28/2023] [Indexed: 05/15/2023] Open
Abstract
Gene expression can be used to subtype breast cancer with improved prediction of risk of recurrence and treatment responsiveness over that obtained using routine immunohistochemistry (IHC). However, in the clinic, molecular profiling is primarily used for ER+ breast cancer, which is costly, tissue destructive, requires specialised platforms, and takes several weeks to obtain a result. Deep learning algorithms can effectively extract morphological patterns in digital histopathology images to predict molecular phenotypes quickly and cost-effectively. We propose a new, computationally efficient approach called hist2RNA inspired by bulk RNA sequencing techniques to predict the expression of 138 genes (incorporated from 6 commercially available molecular profiling tests), including luminal PAM50 subtype, from hematoxylin and eosin (H&E)-stained whole slide images (WSIs). The training phase involves the aggregation of extracted features for each patient from a pretrained model to predict gene expression at the patient level using annotated H&E images from The Cancer Genome Atlas (TCGA, n = 335). We demonstrate successful gene prediction on a held-out test set (n = 160, corr = 0.82 across patients, corr = 0.29 across genes) and perform exploratory analysis on an external tissue microarray (TMA) dataset (n = 498) with known IHC and survival information. Our model is able to predict gene expression and luminal PAM50 subtype (Luminal A versus Luminal B) on the TMA dataset with prognostic significance for overall survival in univariate analysis (c-index = 0.56, hazard ratio = 2.16 (95% CI 1.12-3.06), p < 5 × 10-3), and independent significance in multivariate analysis incorporating standard clinicopathological variables (c-index = 0.65, hazard ratio = 1.87 (95% CI 1.30-2.68), p < 5 × 10-3). The proposed strategy achieves superior performance while requiring less training time, resulting in less energy consumption and computational cost compared to patch-based models. Additionally, hist2RNA predicts gene expression that has potential to determine luminal molecular subtypes which correlates with overall survival, without the need for expensive molecular testing.
Collapse
Affiliation(s)
- Raktim Kumar Mondol
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW 2052, Australia
| | - Ewan K A Millar
- Department of Anatomical Pathology, NSW Health Pathology, St. George Hospital, Kogarah, NSW 2217, Australia
- St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW 2052, Australia
- Faculty of Medicine and Health Sciences, Sydney Western University, Campbelltown, NSW 2560, Australia
- University of Technology Sydney, Ultimo, NSW 2007, Australia
| | - Peter H Graham
- St. George and Sutherland Clinical School, UNSW Sydney, Kensington, NSW 2052, Australia
- Cancer Care Centre, St George Hospital, Sydney, NSW 2217, Australia
| | - Lois Browne
- Cancer Care Centre, St George Hospital, Sydney, NSW 2217, Australia
| | - Arcot Sowmya
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW 2052, Australia
| | - Erik Meijering
- School of Computer Science and Engineering, UNSW Sydney, Kensington, NSW 2052, Australia
| |
Collapse
|
13
|
Mamalakis M, Dwivedi K, Sharkey M, Alabed S, Kiely D, Swift AJ. A transparent artificial intelligence framework to assess lung disease in pulmonary hypertension. Sci Rep 2023; 13:3812. [PMID: 36882484 PMCID: PMC9990015 DOI: 10.1038/s41598-023-30503-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 02/24/2023] [Indexed: 03/09/2023] Open
Abstract
Recent studies have recognized the importance of characterizing the extent of lung disease in pulmonary hypertension patients by using Computed Tomography. The trustworthiness of an artificial intelligence system is linked with the depth of the evaluation in functional, operational, usability, safety and validation dimensions. The safety and validation of an artificial tool is linked to the uncertainty estimation of the model's prediction. On the other hand, the functionality, operation and usability can be achieved by explainable deep learning approaches which can verify the learning patterns and use of the network from a generalized point of view. We developed an artificial intelligence framework to map the 3D anatomical models of patients with lung disease in pulmonary hypertension. To verify the trustworthiness of the framework we studied the uncertainty estimation of the network's prediction, and we explained the learning patterns of the network. Therefore, a new generalized technique combining local explainable and interpretable dimensionality reduction approaches (PCA-GradCam, PCA-Shape) was developed. Our open-source software framework was evaluated in unbiased validation datasets achieving accurate, robust and generalized results.
Collapse
Affiliation(s)
- Michail Mamalakis
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK.
- Department of Computer Science, University of Sheffield, 211 Portobello, Sheffield, S1 4DP, UK.
- Insigneo Institute for in silico Medicine, University of Sheffield, The Pam Liversidge Building, Sheffield, S1 3JD, UK.
| | - Krit Dwivedi
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK
- Insigneo Institute for in silico Medicine, University of Sheffield, The Pam Liversidge Building, Sheffield, S1 3JD, UK
| | - Michael Sharkey
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK
| | - Samer Alabed
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK
- Insigneo Institute for in silico Medicine, University of Sheffield, The Pam Liversidge Building, Sheffield, S1 3JD, UK
| | - David Kiely
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK
- Department of Cardiology, University of Sheffield, Sheffield Teaching Hospitals Sheffield, Sheffield, S5 7AU, UK
| | - Andrew J Swift
- Department of Infection, Immunity and Cardiovascular Disease, University of Sheffield, Beech Hill Rd, Sheffield, S10 2RX, UK.
- Insigneo Institute for in silico Medicine, University of Sheffield, The Pam Liversidge Building, Sheffield, S1 3JD, UK.
| |
Collapse
|
14
|
Keyl P, Bischoff P, Dernbach G, Bockmayr M, Fritz R, Horst D, Blüthgen N, Montavon G, Müller KR, Klauschen F. Single-cell gene regulatory network prediction by explainable AI. Nucleic Acids Res 2023; 51:e20. [PMID: 36629274 PMCID: PMC9976884 DOI: 10.1093/nar/gkac1212] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 11/16/2022] [Accepted: 12/06/2022] [Indexed: 01/12/2023] Open
Abstract
The molecular heterogeneity of cancer cells contributes to the often partial response to targeted therapies and relapse of disease due to the escape of resistant cell populations. While single-cell sequencing has started to improve our understanding of this heterogeneity, it offers a mostly descriptive view on cellular types and states. To obtain more functional insights, we propose scGeneRAI, an explainable deep learning approach that uses layer-wise relevance propagation (LRP) to infer gene regulatory networks from static single-cell RNA sequencing data for individual cells. We benchmark our method with synthetic data and apply it to single-cell RNA sequencing data of a cohort of human lung cancers. From the predicted single-cell networks our approach reveals characteristic network patterns for tumor cells and normal epithelial cells and identifies subnetworks that are observed only in (subgroups of) tumor cells of certain patients. While current state-of-the-art methods are limited by their ability to only predict average networks for cell populations, our approach facilitates the reconstruction of networks down to the level of single cells which can be utilized to characterize the heterogeneity of gene regulation within and across tumors.
Collapse
Affiliation(s)
- Philipp Keyl
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany
| | - Philip Bischoff
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Anna-Louisa-Karsch-Straße 2, 10178 Berlin, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Berlin partner site, Germany
| | - Gabriel Dernbach
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
| | - Michael Bockmayr
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,Department of Pediatric Hematology and Oncolog, University Medical Center Hamburg-Eppendorf, Martinistr. 52, 20246 Hamburg, Germany.,Mildred Scheel Cancer Career Center HaTriCS4, University Medical Center Hamburg-Eppendorf Martinistr. 52, 20246 Hamburg, Germany
| | - Rebecca Fritz
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany
| | - David Horst
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Berlin partner site, Germany
| | - Nils Blüthgen
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,Institut für Biologie, Humboldt University, Free University of Berlin, Unter den Linden 6, 10099 Berlin, Germany
| | - Grégoire Montavon
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.,Machine Learning Group, Technical University of Berlin, Marchstr. 23, 10587 Berlin, Germany
| | - Klaus-Robert Müller
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.,Machine Learning Group, Technical University of Berlin, Marchstr. 23, 10587 Berlin, Germany.,Department of Artificial Intelligence, Korea University, Seoul 136-713, South Korea.,Max-Planck-Institute for Informatics, Stuhlsatzenhausweg 4, 66123 Saarbrücken, Germany
| | - Frederick Klauschen
- Institute of Pathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Berlin, Charitéplatz 1, 10117 Berlin, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Berlin partner site, Germany.,BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.,Institute of Pathology, Ludwig-Maximilians-University Munich, Thalkirchner Str. 36, 80337 München, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Munich partner site, Germany
| |
Collapse
|
15
|
Zeng Y, Xu X. Label Diffusion Graph Learning network for semi-supervised breast histological image recognition. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
16
|
Martin-Leo H, Frederick K, Wojciech S, Klaus-Robert M, Emmanuel D, Sonja S, Sylvia H, Ina K, Jörg A, Liron P, Hendrik S, Patrick W. Imaging bridges pathology and radiology. J Pathol Inform 2023; 14:100298. [PMID: 36851923 PMCID: PMC9958472 DOI: 10.1016/j.jpi.2023.100298] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 01/23/2023] [Accepted: 01/25/2023] [Indexed: 02/01/2023] Open
Abstract
In recent years, medical disciplines have moved closer together and rigid borders have been increasingly dissolved. The synergetic advantage of combining multiple disciplines is particularly important for radiology, nuclear medicine, and pathology to perform integrative diagnostics. In this review, we discuss how medical subdisciplines can be reintegrated in the future using state-of-the-art methods of digitization, data science, and machine learning. Integration of methods is made possible by the digitalization of radiological and nuclear medical images, as well as pathological images. 3D histology can become a valuable tool, not only for integration into radiological images but also for the visualization of cellular interactions, the so-called connectomes. In human pathology, it has recently become possible to image and calculate the movements and contacts of immunostained cells in fresh tissue explants. Recording the movement of a living cell is proving to be informative and makes it possible to study dynamic connectomes in the diagnosis of lymphoid tissue. By applying computational methods including data science and machine learning, new perspectives for analyzing and understanding diseases become possible.
Collapse
Affiliation(s)
- Hansmann Martin-Leo
- Frankfurt Institute for Advanced Studies, Frankfurt/Main, Germany
- Institute for Pharmacology and Toxicology, Goethe University, Frankfurt/Main, Germany
| | - Klauschen Frederick
- Charité University Hospital, Berlin, Germany
- German Cancer Consortium (DKTK), Munich partner site, and German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- BIFOLD -- Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Ludwig-Maximilians-Universität, Munich, Germany
- Aignostics GmbH, Berlin, Germany
| | - Samek Wojciech
- BIFOLD -- Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Fraunhofer Heinrich Hertz Institute, Berlin, Germany
- Technical University Berlin, Berlin, Germany
| | - Müller Klaus-Robert
- BIFOLD -- Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Aignostics GmbH, Berlin, Germany
- Technical University Berlin, Berlin, Germany
- Korea University, Seoul, South Korea
- Max-Planck-Institut für Informatik, Saarbrücken, Germany
| | - Donnadieu Emmanuel
- Université Paris Cité, CNRS, INSERM, Equipe Labellisée Ligue Contre le Cancer, Institut Cochin, F-75014 Paris, France
| | - Scharf Sonja
- Frankfurt Institute for Advanced Studies, Frankfurt/Main, Germany
- Institute for Pharmacology and Toxicology, Goethe University, Frankfurt/Main, Germany
- Department of Molecular Bioinformatics, Goethe University Frankfurt, Frankfurt/Main, Germany
| | - Hartmann Sylvia
- Dr. Senckenberg Institute of Pathology, Goethe University Frankfurt, Frankfurt/Main, Germany
| | - Koch Ina
- Department of Molecular Bioinformatics, Goethe University Frankfurt, Frankfurt/Main, Germany
| | - Ackermann Jörg
- Department of Molecular Bioinformatics, Goethe University Frankfurt, Frankfurt/Main, Germany
| | - Pantanowitz Liron
- Department of Pathology, University of Michigan, Ann Arbor, Michigan
| | - Schäfer Hendrik
- Institute for Pharmacology and Toxicology, Goethe University, Frankfurt/Main, Germany
- Charité University Hospital, Berlin, Germany
| | - Wurzel Patrick
- Frankfurt Institute for Advanced Studies, Frankfurt/Main, Germany
- Institute for Pharmacology and Toxicology, Goethe University, Frankfurt/Main, Germany
- Department of Molecular Bioinformatics, Goethe University Frankfurt, Frankfurt/Main, Germany
| |
Collapse
|
17
|
Ogier du Terrail J, Leopold A, Joly C, Béguier C, Andreux M, Maussion C, Schmauch B, Tramel EW, Bendjebbar E, Zaslavskiy M, Wainrib G, Milder M, Gervasoni J, Guerin J, Durand T, Livartowski A, Moutet K, Gautier C, Djafar I, Moisson AL, Marini C, Galtier M, Balazard F, Dubois R, Moreira J, Simon A, Drubay D, Lacroix-Triki M, Franchet C, Bataillon G, Heudel PE. Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat Med 2023; 29:135-146. [PMID: 36658418 DOI: 10.1038/s41591-022-02155-w] [Citation(s) in RCA: 25] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Accepted: 11/23/2022] [Indexed: 01/21/2023]
Abstract
Triple-negative breast cancer (TNBC) is a rare cancer, characterized by high metastatic potential and poor prognosis, and has limited treatment options. The current standard of care in nonmetastatic settings is neoadjuvant chemotherapy (NACT), but treatment efficacy varies substantially across patients. This heterogeneity is still poorly understood, partly due to the paucity of curated TNBC data. Here we investigate the use of machine learning (ML) leveraging whole-slide images and clinical information to predict, at diagnosis, the histological response to NACT for early TNBC women patients. To overcome the biases of small-scale studies while respecting data privacy, we conducted a multicentric TNBC study using federated learning, in which patient data remain secured behind hospitals' firewalls. We show that local ML models relying on whole-slide images can predict response to NACT but that collaborative training of ML models further improves performance, on par with the best current approaches in which ML models are trained using time-consuming expert annotations. Our ML model is interpretable and is sensitive to specific histological patterns. This proof of concept study, in which federated learning is applied to real-world datasets, paves the way for future biomarker discovery using unprecedentedly large datasets.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Camille Franchet
- Institut Universitaire du Cancer de Toulouse (IUCT) Oncopole, Toulouse, France
| | | | | |
Collapse
|
18
|
Farahani FV, Fiok K, Lahijanian B, Karwowski W, Douglas PK. Explainable AI: A review of applications to neuroimaging data. Front Neurosci 2022; 16:906290. [PMID: 36583102 PMCID: PMC9793854 DOI: 10.3389/fnins.2022.906290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 11/14/2022] [Indexed: 12/05/2022] Open
Abstract
Deep neural networks (DNNs) have transformed the field of computer vision and currently constitute some of the best models for representations learned via hierarchical processing in the human brain. In medical imaging, these models have shown human-level performance and even higher in the early diagnosis of a wide range of diseases. However, the goal is often not only to accurately predict group membership or diagnose but also to provide explanations that support the model decision in a context that a human can readily interpret. The limited transparency has hindered the adoption of DNN algorithms across many domains. Numerous explainable artificial intelligence (XAI) techniques have been developed to peer inside the "black box" and make sense of DNN models, taking somewhat divergent approaches. Here, we suggest that these methods may be considered in light of the interpretation goal, including functional or mechanistic interpretations, developing archetypal class instances, or assessing the relevance of certain features or mappings on a trained model in a post-hoc capacity. We then focus on reviewing recent applications of post-hoc relevance techniques as applied to neuroimaging data. Moreover, this article suggests a method for comparing the reliability of XAI methods, especially in deep neural networks, along with their advantages and pitfalls.
Collapse
Affiliation(s)
- Farzad V. Farahani
- Department of Biostatistics, Johns Hopkins University, Baltimore, MD, United States
- Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, United States
| | - Krzysztof Fiok
- Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, United States
| | - Behshad Lahijanian
- Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, United States
- H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | - Waldemar Karwowski
- Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, United States
| | - Pamela K. Douglas
- School of Modeling, Simulation, and Training, University of Central Florida, Orlando, FL, United States
| |
Collapse
|
19
|
Fähndrich J, Povalej R, Rittelmeier H, Berner S. Interview: AI Expert Prof. Müller on XAI. Künstl Intell 2022. [DOI: 10.1007/s13218-022-00776-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
20
|
Kim I, Kang K, Song Y, Kim TJ. Application of Artificial Intelligence in Pathology: Trends and Challenges. Diagnostics (Basel) 2022; 12:diagnostics12112794. [PMID: 36428854 PMCID: PMC9688959 DOI: 10.3390/diagnostics12112794] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 11/03/2022] [Accepted: 11/11/2022] [Indexed: 11/16/2022] Open
Abstract
Given the recent success of artificial intelligence (AI) in computer vision applications, many pathologists anticipate that AI will be able to assist them in a variety of digital pathology tasks. Simultaneously, tremendous advancements in deep learning have enabled a synergy with artificial intelligence (AI), allowing for image-based diagnosis on the background of digital pathology. There are efforts for developing AI-based tools to save pathologists time and eliminate errors. Here, we describe the elements in the development of computational pathology (CPATH), its applicability to AI development, and the challenges it faces, such as algorithm validation and interpretability, computing systems, reimbursement, ethics, and regulations. Furthermore, we present an overview of novel AI-based approaches that could be integrated into pathology laboratory workflows.
Collapse
Affiliation(s)
- Inho Kim
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Kyungmin Kang
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Youngjae Song
- College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seocho-gu, Seoul 06591, Republic of Korea
| | - Tae-Jung Kim
- Department of Hospital Pathology, Yeouido St. Mary’s Hospital, College of Medicine, The Catholic University of Korea, 10, 63-ro, Yeongdeungpo-gu, Seoul 07345, Republic of Korea
- Correspondence: ; Tel.: +82-2-3779-2157
| |
Collapse
|
21
|
Wagner P, Strodthoff N, Wurzel P, Marban A, Scharf S, Schäfer H, Seegerer P, Loth A, Hartmann S, Klauschen F, Müller KR, Samek W, Hansmann ML. New definitions of human lymphoid and follicular cell entities in lymphatic tissue by machine learning. Sci Rep 2022; 12:18991. [PMID: 36347879 DOI: 10.1038/s41598-022-18097-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Accepted: 08/05/2022] [Indexed: 11/09/2022] Open
Abstract
Histological sections of the lymphatic system are usually the basis of static (2D) morphological investigations. Here, we performed a dynamic (4D) analysis of human reactive lymphoid tissue using confocal fluorescent laser microscopy in combination with machine learning. Based on tracks for T-cells (CD3), B-cells (CD20), follicular T-helper cells (PD1) and optical flow of follicular dendritic cells (CD35), we put forward the first quantitative analysis of movement-related and morphological parameters within human lymphoid tissue. We identified correlations of follicular dendritic cell movement and the behavior of lymphocytes in the microenvironment. In addition, we investigated the value of movement and/or morphological parameters for a precise definition of cell types (CD clusters). CD-clusters could be determined based on movement and/or morphology. Differentiating between CD3- and CD20 positive cells is most challenging and long term-movement characteristics are indispensable. We propose morphological and movement-related prototypes of cell entities applying machine learning models. Finally, we define beyond CD clusters new subgroups within lymphocyte entities based on long term movement characteristics. In conclusion, we showed that the combination of 4D imaging and machine learning is able to define characteristics of lymphocytes not visible in 2D histology.
Collapse
|
22
|
Loh HW, Ooi CP, Seoni S, Barua PD, Molinari F, Acharya UR. Application of explainable artificial intelligence for healthcare: A systematic review of the last decade (2011-2022). Comput Methods Programs Biomed 2022; 226:107161. [PMID: 36228495 DOI: 10.1016/j.cmpb.2022.107161] [Citation(s) in RCA: 66] [Impact Index Per Article: 33.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 09/16/2022] [Accepted: 09/25/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVES Artificial intelligence (AI) has branched out to various applications in healthcare, such as health services management, predictive medicine, clinical decision-making, and patient data and diagnostics. Although AI models have achieved human-like performance, their use is still limited because they are seen as a black box. This lack of trust remains the main reason for their low use in practice, especially in healthcare. Hence, explainable artificial intelligence (XAI) has been introduced as a technique that can provide confidence in the model's prediction by explaining how the prediction is derived, thereby encouraging the use of AI systems in healthcare. The primary goal of this review is to provide areas of healthcare that require more attention from the XAI research community. METHODS Multiple journal databases were thoroughly searched using PRISMA guidelines 2020. Studies that do not appear in Q1 journals, which are highly credible, were excluded. RESULTS In this review, we surveyed 99 Q1 articles covering the following XAI techniques: SHAP, LIME, GradCAM, LRP, Fuzzy classifier, EBM, CBR, rule-based systems, and others. CONCLUSION We discovered that detecting abnormalities in 1D biosignals and identifying key text in clinical notes are areas that require more attention from the XAI research community. We hope this is review will encourage the development of a holistic cloud system for a smart city.
Collapse
Affiliation(s)
- Hui Wen Loh
- School of Science and Technology, Singapore University of Social Sciences, Singapore
| | - Chui Ping Ooi
- School of Science and Technology, Singapore University of Social Sciences, Singapore
| | - Silvia Seoni
- Department of Electronics and Telecommunications, Biolab, Politecnico di Torino, Torino 10129, Italy
| | - Prabal Datta Barua
- Faculty of Engineering and Information Technology, University of Technology Sydney, Australia; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Australia
| | - Filippo Molinari
- Department of Electronics and Telecommunications, Biolab, Politecnico di Torino, Torino 10129, Italy
| | - U Rajendra Acharya
- School of Science and Technology, Singapore University of Social Sciences, Singapore; School of Business (Information Systems), Faculty of Business, Education, Law & Arts, University of Southern Queensland, Australia; School of Engineering, Ngee Ann Polytechnic, Singapore; Department of Bioinformatics and Medical Engineering, Asia University, Taiwan; Research Organization for Advanced Science and Technology (IROAST), Kumamoto University, Kumamoto, Japan.
| |
Collapse
|
23
|
Bai B, Wang H, Li Y, de Haan K, Colonnese F, Wan Y, Zuo J, Doan NB, Zhang X, Zhang Y, Li J, Yang X, Dong W, Darrow MA, Kamangar E, Lee HS, Rivenson Y, Ozcan A. Label-Free Virtual HER2 Immunohistochemical Staining of Breast Tissue using Deep Learning. BME Front 2022; 2022:9786242. [PMID: 37850170 PMCID: PMC10521710 DOI: 10.34133/2022/9786242] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 08/25/2022] [Indexed: 10/19/2023] Open
Abstract
The immunohistochemical (IHC) staining of the human epidermal growth factor receptor 2 (HER2) biomarker is widely practiced in breast tissue analysis, preclinical studies, and diagnostic decisions, guiding cancer treatment and investigation of pathogenesis. HER2 staining demands laborious tissue treatment and chemical processing performed by a histotechnologist, which typically takes one day to prepare in a laboratory, increasing analysis time and associated costs. Here, we describe a deep learning-based virtual HER2 IHC staining method using a conditional generative adversarial network that is trained to rapidly transform autofluorescence microscopic images of unlabeled/label-free breast tissue sections into bright-field equivalent microscopic images, matching the standard HER2 IHC staining that is chemically performed on the same tissue sections. The efficacy of this virtual HER2 staining framework was demonstrated by quantitative analysis, in which three board-certified breast pathologists blindly graded the HER2 scores of virtually stained and immunohistochemically stained HER2 whole slide images (WSIs) to reveal that the HER2 scores determined by inspecting virtual IHC images are as accurate as their immunohistochemically stained counterparts. A second quantitative blinded study performed by the same diagnosticians further revealed that the virtually stained HER2 images exhibit a comparable staining quality in the level of nuclear detail, membrane clearness, and absence of staining artifacts with respect to their immunohistochemically stained counterparts. This virtual HER2 staining framework bypasses the costly, laborious, and time-consuming IHC staining procedures in laboratory and can be extended to other types of biomarkers to accelerate the IHC tissue staining used in life sciences and biomedical workflow.
Collapse
Affiliation(s)
- Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Hongda Wang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Kevin de Haan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | | | - Yujie Wan
- Physics and Astronomy Department, University of California, Los Angeles, CA 90095, USA
| | - Jingyi Zuo
- Computer Science Department, University of California, Los Angeles, CA, USA
| | - Ngan B. Doan
- Translational Pathology Core Laboratory, University of California, Los Angeles, CA 90095, USA
| | - Xiaoran Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Jingxi Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Xilin Yang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Wenjie Dong
- Statistics Department, University of California, Los Angeles, CA 90095, USA
| | - Morgan Angus Darrow
- Department of Pathology and Laboratory Medicine, University of California at Davis, Sacramento, CA 95817, USA
| | - Elham Kamangar
- Department of Pathology and Laboratory Medicine, University of California at Davis, Sacramento, CA 95817, USA
| | - Han Sung Lee
- Department of Pathology and Laboratory Medicine, University of California at Davis, Sacramento, CA 95817, USA
| | - Yair Rivenson
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
- Bioengineering Department, University of California, Los Angeles 90095, USA
- California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
- Department of Surgery, University of California, Los Angeles, CA 90095, USA
| |
Collapse
|
24
|
Lee LY, Yang CH, Lin YC, Hsieh YH, Chen YA, Chang MDT, Lin YY, Liao CT. A domain knowledge enhanced yield based deep learning classifier identifies perineural invasion in oral cavity squamous cell carcinoma. Front Oncol 2022; 12:951560. [DOI: 10.3389/fonc.2022.951560] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 10/10/2022] [Indexed: 11/13/2022] Open
Abstract
BackgroundPerineural invasion (PNI), a form of local invasion defined as the ability of cancer cells to invade in, around, and through nerves, has a negative prognostic impact in oral cavity squamous cell carcinoma (OCSCC). Unfortunately, the diagnosis of PNI suffers from a significant degree of intra- and interobserver variability. The aim of this pilot study was to develop a deep learning-based human-enhanced tool, termed domain knowledge enhanced yield (Domain-KEY) algorithm, for identifying PNI in digital slides.MethodsHematoxylin and eosin (H&E)-stained whole-slide images (WSIs, n = 85) were obtained from 80 patients with OCSCC. The model structure consisted of two parts to simulate human decision-making skills in diagnostic pathology. To this aim, two semantic segmentation models were constructed (i.e., identification of nerve fibers followed by the diagnosis of PNI). The inferred results were subsequently subjected to post-processing of generated decision rules for diagnostic labeling. Ten H&E-stained WSIs not previously used in the study were read and labeled by the Domain-KEY algorithm. Thereafter, labeling correctness was visually inspected by two independent pathologists.ResultsThe Domain-KEY algorithm was found to outperform the ResnetV2_50 classifier for the detection of PNI (diagnostic accuracy: 89.01% and 61.94%, respectively). On analyzing WSIs, the algorithm achieved a mean diagnostic accuracy as high as 97.50% versus traditional pathology. The observed accuracy in a validation dataset of 25 WSIs obtained from seven patients with oropharyngeal (cancer of the tongue base, n = 1; tonsil cancer, n = 1; soft palate cancer, n = 1) and hypopharyngeal (cancer of posterior wall, n = 2; pyriform sinus cancer, n = 2) malignancies was 96%. Notably, the algorithm was successfully applied in the analysis of WSIs to shorten the time required to reach a diagnosis. The addition of the hybrid intelligence model decreased the mean time required to reach a diagnosis by 15.0% and 23.7% for the first and second pathologists, respectively. On analyzing digital slides, the tool was effective in supporting human diagnostic thinking.ConclusionsThe Domain-KEY algorithm successfully mimicked human decision-making skills and supported expert pathologists in the routine diagnosis of PNI.
Collapse
|
25
|
Srinivasan V, Strodthoff N, Ma J, Binder A, Müller KR, Samek W. To pretrain or not? A systematic analysis of the benefits of pretraining in diabetic retinopathy. PLoS One 2022; 17:e0274291. [PMID: 36256665 PMCID: PMC9578637 DOI: 10.1371/journal.pone.0274291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2022] [Accepted: 08/26/2022] [Indexed: 11/06/2022] Open
Abstract
There is an increasing number of medical use cases where classification algorithms based on deep neural networks reach performance levels that are competitive with human medical experts. To alleviate the challenges of small dataset sizes, these systems often rely on pretraining. In this work, we aim to assess the broader implications of these approaches in order to better understand what type of pretraining works reliably (with respect to performance, robustness, learned representation etc.) in practice and what type of pretraining dataset is best suited to achieve good performance in small target dataset size scenarios. Considering diabetic retinopathy grading as an exemplary use case, we compare the impact of different training procedures including recently established self-supervised pretraining methods based on contrastive learning. To this end, we investigate different aspects such as quantitative performance, statistics of the learned feature representations, interpretability and robustness to image distortions. Our results indicate that models initialized from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions. In particular, self-supervised models show further benefits to supervised models. Self-supervised models with initialization from ImageNet pretraining not only report higher performance, they also reduce overfitting to large lesions along with improvements in taking into account minute lesions indicative of the progression of the disease. Understanding the effects of pretraining in a broader sense that goes beyond simple performance comparisons is of crucial importance for the broader medical imaging community beyond the use case considered in this work.
Collapse
Affiliation(s)
- Vignesh Srinivasan
- Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany
| | - Nils Strodthoff
- School of Medicine and Health Services, Oldenburg University, Oldenburg, Germany
| | - Jackie Ma
- Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany
| | - Alexander Binder
- Singapore Institute of Technology, ICT Cluster, Singapore, Singapore
- Department of Informatics, Oslo University, Oslo, Norway
| | - Klaus-Robert Müller
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
- Department of Artificial Intelligence, Korea University, Seoul, South Korea
- Max Planck Institute for Informatics, Saarbrücken, Germany
- * E-mail: (KRM); (WS)
| | - Wojciech Samek
- Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany
- BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
- * E-mail: (KRM); (WS)
| |
Collapse
|
26
|
Lipkova J, Chen RJ, Chen B, Lu MY, Barbieri M, Shao D, Vaidya AJ, Chen C, Zhuang L, Williamson DFK, Shaban M, Chen TY, Mahmood F. Artificial intelligence for multimodal data integration in oncology. Cancer Cell 2022; 40:1095-1110. [PMID: 36220072 PMCID: PMC10655164 DOI: 10.1016/j.ccell.2022.09.012] [Citation(s) in RCA: 68] [Impact Index Per Article: 34.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 07/12/2022] [Accepted: 09/15/2022] [Indexed: 02/07/2023]
Abstract
In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.
Collapse
Affiliation(s)
- Jana Lipkova
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Richard J Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Bowen Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Harvard University, Cambridge, MA, USA
| | - Ming Y Lu
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, MA, USA
| | - Matteo Barbieri
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Daniel Shao
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA
| | - Anurag J Vaidya
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Harvard-MIT Health Sciences and Technology (HST), Cambridge, MA, USA
| | - Chengkuan Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Luoting Zhuang
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA
| | - Drew F K Williamson
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Muhammad Shaban
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Tiffany Y Chen
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Faisal Mahmood
- Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Harvard Data Science Initiative, Harvard University, Cambridge, MA, USA.
| |
Collapse
|
27
|
Lin J, Han G, Pan X, Liu Z, Chen H, Li D, Jia X, Shi Z, Wang Z, Cui Y, Li H, Liang C, Liang L, Wang Y, Han C. PDBL: Improving Histopathological Tissue Classification With Plug-and-Play Pyramidal Deep-Broad Learning. IEEE Trans Med Imaging 2022; 41:2252-2262. [PMID: 35320093 DOI: 10.1109/tmi.2022.3161787] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Histopathological tissue classification is a simpler way to achieve semantic segmentation for the whole slide images, which can alleviate the requirement of pixel-level dense annotations. Existing works mostly leverage the popular CNN classification backbones in computer vision to achieve histopathological tissue classification. In this paper, we propose a super lightweight plug-and-play module, named Pyramidal Deep-Broad Learning (PDBL), for any well-trained classification backbone to improve the classification performance without a re-training burden. For each patch, we construct a multi-resolution image pyramid to obtain the pyramidal contextual information. For each level in the pyramid, we extract the multi-scale deep-broad features by our proposed Deep-Broad block (DB-block). We equip PDBL in three popular classification backbones, ShuffLeNetV2, EfficientNetb0, and ResNet50 to evaluate the effectiveness and efficiency of our proposed module on two datasets (Kather Multiclass Dataset and the LC25000 Dataset). Experimental results demonstrate the proposed PDBL can steadily improve the tissue-level classification performance for any CNN backbones, especially for the lightweight models when given a small among of training samples (less than 10%). It greatly saves the computational resources and annotation efforts. The source code is available at: https://github.com/linjiatai/PDBL.
Collapse
|
28
|
Shmatko A, Ghaffari Laleh N, Gerstung M, Kather JN. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. Nat Cancer 2022; 3:1026-1038. [PMID: 36138135 DOI: 10.1038/s43018-022-00436-4] [Citation(s) in RCA: 87] [Impact Index Per Article: 43.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Accepted: 08/03/2022] [Indexed: 06/16/2023]
Abstract
Artificial intelligence (AI) methods have multiplied our capabilities to extract quantitative information from digital histopathology images. AI is expected to reduce workload for human experts, improve the objectivity and consistency of pathology reports, and have a clinical impact by extracting hidden information from routinely available data. Here, we describe how AI can be used to predict cancer outcome, treatment response, genetic alterations and gene expression from digitized histopathology slides. We summarize the underlying technologies and emerging approaches, noting limitations, including the need for data sharing and standards. Finally, we discuss the broader implications of AI in cancer research and oncology.
Collapse
Affiliation(s)
- Artem Shmatko
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK
| | | | - Moritz Gerstung
- Division of AI in Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
- European Molecular Biology Laboratory, European Bioinformatics Institute, Cambridge, UK.
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- Medical Oncology, National Center for Tumor Diseases, University Hospital Heidelberg, Heidelberg, Germany.
- Pathology and Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
- Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany.
| |
Collapse
|
29
|
Mayer RS, Gretser S, Heckmann LE, Ziegler PK, Walter B, Reis H, Bankov K, Becker S, Triesch J, Wild PJ, Flinner N. How to learn with intentional mistakes: NoisyEnsembles to overcome poor tissue quality for deep learning in computational pathology. Front Med (Lausanne) 2022; 9:959068. [PMID: 36106328 PMCID: PMC9464871 DOI: 10.3389/fmed.2022.959068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Accepted: 08/01/2022] [Indexed: 11/13/2022] Open
Abstract
There is a lot of recent interest in the field of computational pathology, as many algorithms are introduced to detect, for example, cancer lesions or molecular features. However, there is a large gap between artificial intelligence (AI) technology and practice, since only a small fraction of the applications is used in routine diagnostics. The main problems are the transferability of convolutional neural network (CNN) models to data from other sources and the identification of uncertain predictions. The role of tissue quality itself is also largely unknown. Here, we demonstrated that samples of the TCGA ovarian cancer (TCGA-OV) dataset from different tissue sources have different quality characteristics and that CNN performance is linked to this property. CNNs performed best on high-quality data. Quality control tools were partially able to identify low-quality tiles, but their use did not increase the performance of the trained CNNs. Furthermore, we trained NoisyEnsembles by introducing label noise during training. These NoisyEnsembles could improve CNN performance for low-quality, unknown datasets. Moreover, the performance increases as the ensemble become more consistent, suggesting that incorrect predictions could be discarded efficiently to avoid wrong diagnostic decisions.
Collapse
Affiliation(s)
- Robin S. Mayer
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Steffen Gretser
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Lara E. Heckmann
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Paul K. Ziegler
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Britta Walter
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Henning Reis
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Katrin Bankov
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Sven Becker
- Department of Gynecology and Obstetrics, University Hospital Frankfurt, Frankfurt am Main, Germany
| | - Jochen Triesch
- Frankfurt Institute for Advanced Studies (FIAS), Frankfurt am Main, Germany
| | - Peter J. Wild
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
- Frankfurt Institute for Advanced Studies (FIAS), Frankfurt am Main, Germany
- Wildlab, University Hospital Frankfurt MVZ GmbH, Frankfurt am Main, Germany
- Frankfurt Cancer Institute (FCI), Frankfurt am Main, Germany
- University Cancer Center (UCT) Frankfurt-Marburg, Frankfurt am Main, Germany
| | - Nadine Flinner
- Dr. Senckenberg Institute of Pathology, University Hospital Frankfurt, Frankfurt am Main, Germany
- Frankfurt Institute for Advanced Studies (FIAS), Frankfurt am Main, Germany
- Frankfurt Cancer Institute (FCI), Frankfurt am Main, Germany
- University Cancer Center (UCT) Frankfurt-Marburg, Frankfurt am Main, Germany
- *Correspondence: Nadine Flinner
| |
Collapse
|
30
|
Hofmann SM, Beyer F, Lapuschkin S, Goltermann O, Loeffler M, Müller KR, Villringer A, Samek W, Witte AV. Towards the Interpretability of Deep Learning Models for Multi-modal Neuroimaging: Finding Structural Changes of the Ageing Brain. Neuroimage 2022; 261:119504. [PMID: 35882272 DOI: 10.1016/j.neuroimage.2022.119504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Revised: 07/15/2022] [Accepted: 07/21/2022] [Indexed: 11/17/2022] Open
Abstract
Brain-age (BA) estimates based on deep learning are increasingly used as neuroimaging biomarker for brain health; however, the underlying neural features have remained unclear. We combined ensembles of convolutional neural networks with Layer-wise Relevance Propagation (LRP) to detect which brain features contribute to BA. Trained on magnetic resonance imaging (MRI) data of a population-based study (n=2637, 18-82 years), our models estimated age accurately based on single and multiple modalities, regionally restricted and whole-brain images (mean absolute errors 3.37-3.86 years). We find that BA estimates capture aging at both small and large-scale changes, revealing gross enlargements of ventricles and subarachnoid spaces, as well as white matter lesions, and atrophies that appear throughout the brain. Divergence from expected aging reflected cardiovascular risk factors and accelerated aging was more pronounced in the frontal lobe. Applying LRP, our study demonstrates how superior deep learning models detect brain-aging in healthy and at-risk individuals throughout adulthood.
Collapse
Affiliation(s)
- Simon M Hofmann
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany; Department of Artificial Intelligence, Fraunhofer Institute Heinrich Hertz, 10587 Berlin, Germany; Clinic for Cognitive Neurology, University of Leipzig Medical Center, 04103 Leipzig, Germany.
| | - Frauke Beyer
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany; Clinic for Cognitive Neurology, University of Leipzig Medical Center, 04103 Leipzig, Germany
| | - Sebastian Lapuschkin
- Department of Artificial Intelligence, Fraunhofer Institute Heinrich Hertz, 10587 Berlin, Germany
| | - Ole Goltermann
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany; Max Planck School of Cognition, 04103 Leipzig, Germany; Institute of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Germany
| | | | - Klaus-Robert Müller
- Department of Electrical Engineering and Computer Science, Technical University Berlin, 10623 Berlin, Germany; Department of Artificial Intelligence, Korea University, 02841 Seoul, Korea (the Republic of); Brain Team, Google Research, 10117 Berlin, Germany; Max Planck Institute for Informatics, 66123 Saarbrücken, Germany; BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
| | - Arno Villringer
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany; Clinic for Cognitive Neurology, University of Leipzig Medical Center, 04103 Leipzig, Germany; MindBrainBody Institute, Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, 10099 Berlin, Germany; Center for Stroke Research, Charité - Universitätsmedizin Berlin, 10117 Berlin, Germany
| | - Wojciech Samek
- Department of Artificial Intelligence, Fraunhofer Institute Heinrich Hertz, 10587 Berlin, Germany; Department of Electrical Engineering and Computer Science, Technical University Berlin, 10623 Berlin, Germany; BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
| | - A Veronica Witte
- Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany; Clinic for Cognitive Neurology, University of Leipzig Medical Center, 04103 Leipzig, Germany
| |
Collapse
|
31
|
Keyl P, Bockmayr M, Heim D, Dernbach G, Montavon G, Müller K, Klauschen F. Patient-level proteomic network prediction by explainable artificial intelligence. NPJ Precis Oncol 2022; 6. [PMID: 35672443 PMCID: PMC9174200 DOI: 10.1038/s41698-022-00278-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Accepted: 04/15/2022] [Indexed: 11/08/2022] Open
Abstract
Understanding the pathological properties of dysregulated protein networks in individual patients’ tumors is the basis for precision therapy. Functional experiments are commonly used, but cover only parts of the oncogenic signaling networks, whereas methods that reconstruct networks from omics data usually only predict average network features across tumors. Here, we show that the explainable AI method layer-wise relevance propagation (LRP) can infer protein interaction networks for individual patients from proteomic profiling data. LRP reconstructs average and individual interaction networks with an AUC of 0.99 and 0.93, respectively, and outperforms state-of-the-art network prediction methods for individual tumors. Using data from The Cancer Proteome Atlas, we identify known and potentially novel oncogenic network features, among which some are cancer-type specific and show only minor variation among patients, while others are present across certain tumor types but differ among individual patients. Our approach may therefore support predictive diagnostics in precision oncology by inferring “patient-level” oncogenic mechanisms.
Collapse
|
32
|
Abstract
Given the rapid developments, there is no doubt that artificial intelligence (AI) will substantially impact pathological diagnostics. However, it remains an open question if AI will primarily be another diagnostic tool, such as immunohistochemistry, or if AI will also be able to replace human expertise. Most current studies on AI in histopathology deal with relatively simple diagnostic problems and are not yet capable of coping with the complexity of routine diagnostics. While some methods in molecular pathology would already be unthinkable without AI, it remains to be shown how AI will also be able to help with difficult histomorphological differential diagnoses in the future.
Collapse
Affiliation(s)
- Philipp Jurmeister
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland
- Institut für Pathologie, Charité - Universitätsmedizin Berlin, Berlin, Deutschland
| | - Klaus-Robert Müller
- Technische Universität Berlin, Berlin, Deutschland
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Deutschland
| | - Frederick Klauschen
- Pathologisches Institut, Ludwig-Maximilians-Universität München, Thalkirchner Str. 36, 80337, München, Deutschland.
- Institut für Pathologie, Charité - Universitätsmedizin Berlin, Berlin, Deutschland.
- Partnerstandort München, Deutsches Konsortium für Translationale Krebsforschung (DKTK) und Deutsches Krebsforschungszentrum (DKFZ), München, Deutschland.
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Deutschland.
| |
Collapse
|
33
|
Saldanha OL, Quirke P, West NP, James JA, Loughrey MB, Grabsch HI, Salto-Tellez M, Alwers E, Cifci D, Ghaffari Laleh N, Seibel T, Gray R, Hutchins GGA, Brenner H, van Treeck M, Yuan T, Brinker TJ, Chang-Claude J, Khader F, Schuppert A, Luedde T, Trautwein C, Muti HS, Foersch S, Hoffmeister M, Truhn D, Kather JN. Swarm learning for decentralized artificial intelligence in cancer histopathology. Nat Med 2022. [PMID: 35469069 DOI: 10.1038/s41591-022-01768-5] [Citation(s) in RCA: 45] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 03/02/2022] [Indexed: 02/07/2023]
Abstract
Artificial intelligence (AI) can predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets for which data collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL), in which partners jointly train AI models while avoiding data transfer and monopolistic data governance. Here, we demonstrate the successful use of SL in large, multicentric datasets of gigapixel histopathology images from over 5,000 patients. We show that AI models trained using SL can predict BRAF mutational status and microsatellite instability directly from hematoxylin and eosin (H&E)-stained pathology slides of colorectal cancer. We trained AI models on three patient cohorts from Northern Ireland, Germany and the United States, and validated the prediction performance in two independent datasets from the United Kingdom. Our data show that SL-trained AI models outperform most locally trained models, and perform on par with models that are trained on the merged datasets. In addition, we show that SL-based AI models are data efficient. In the future, SL can be used to train distributed AI models for any histopathology image analysis task, eliminating the need for data transfer. A decentralized, privacy-preserving machine learning framework used to train a clinically relevant AI system identifies actionable molecular alterations in patients with colorectal cancer by use of routine histopathology slides collected in real-world settings.
Collapse
|
34
|
Leitheiser M, Capper D, Seegerer P, Lehmann A, Schüller U, Müller KR, Klauschen F, Jurmeister P, Bockmayr M. Machine Learning Models Predict the Primary Sites of Head and Neck Squamous Cell Carcinoma Metastases Based on DNA Methylation. J Pathol 2021; 256:378-387. [PMID: 34878655 DOI: 10.1002/path.5845] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Revised: 10/24/2021] [Accepted: 12/06/2021] [Indexed: 11/10/2022]
Abstract
In head and neck squamous cell cancers (HNSCs) that present as metastases with an unknown primary (HNSC-CUPs), the identification of a primary tumor improves therapy options and increases patient survival. However, the currently available diagnostic methods are laborious and do not offer a sufficient detection rate. Predictive machine learning models based on DNA methylation profiles have recently emerged as a promising technique for tumor classification. We applied this technique to HNSC to develop a tool that can improve the diagnostic workup for HNSC-CUPs. On a reference cohort of 405 primary HNSC samples, we developed four classifiers based on different machine learning models (random forest (RF), neural network (NN), elastic net penalized logistic regression (LOGREG), support vector machine (SVM)) that predict the primary site of HNSC tumors from their DNA methylation profile. The classifiers achieved high classification accuracies (RF=83%, NN=88%, LOGREG=SVM=89%) on an independent cohort of 64 HNSC metastases. Further, the NN, LOGREG, and SVM models significantly outperformed p16 status as a marker for an origin in the oropharynx. In conclusion, the DNA methylation profiles of HNSC metastases are characteristic for their primary sites and the classifiers developed in this study, which are made available to the scientific community, can provide valuable information to guide the diagnostic workup of HNSC-CUP. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Maximilian Leitheiser
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany
| | - David Capper
- Department of Neuropathology, Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany.,German Cancer Consortium (DKTK), Partner Site Berlin, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Philipp Seegerer
- Machine-Learning Group, Department of Software Engineering and Theoretical Computer Science, Technical University of Berlin, Berlin, Germany.,Aignostics GmbH, Berlin, Germany
| | - Annika Lehmann
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany
| | - Ulrich Schüller
- Department of Pediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.,Institute of Neuropathology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.,Research Institute Children's Cancer Center Hamburg, Hamburg, Germany
| | - Klaus-Robert Müller
- Machine-Learning Group, Department of Software Engineering and Theoretical Computer Science, Technical University of Berlin, Berlin, Germany.,Department of Artificial Intelligence, Korea University, Seoul, South Korea.,Max-Planck-Institute for Informatics, Saarbrücken, Germany.,BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
| | - Frederick Klauschen
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany.,Aignostics GmbH, Berlin, Germany.,BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany.,LMU München, Institute of Pathology, Faculty of Medicine, LMU Munich, Munich, Germany
| | - Philipp Jurmeister
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany.,German Cancer Consortium (DKTK), Partner Site Berlin, German Cancer Research Center (DKFZ), Heidelberg, Germany.,LMU München, Institute of Pathology, Faculty of Medicine, LMU Munich, Munich, Germany
| | - Michael Bockmayr
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Institute of Pathology, Berlin, Germany.,Department of Pediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.,Research Institute Children's Cancer Center Hamburg, Hamburg, Germany.,Mildred Scheel Cancer Career Center HaTriCS4, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
35
|
Yousif M, van Diest PJ, Laurinavicius A, Rimm D, van der Laak J, Madabhushi A, Schnitt S, Pantanowitz L. Artificial intelligence applied to breast pathology. Virchows Arch 2021; 480:191-209. [PMID: 34791536 DOI: 10.1007/s00428-021-03213-3] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 09/12/2021] [Accepted: 09/27/2021] [Indexed: 12/12/2022]
Abstract
The convergence of digital pathology and computer vision is increasingly enabling computers to perform tasks performed by humans. As a result, artificial intelligence (AI) is having an astoundingly positive effect on the field of pathology, including breast pathology. Research using machine learning and the development of algorithms that learn patterns from labeled digital data based on "deep learning" neural networks and feature-engineered approaches to analyze histology images have recently provided promising results. Thus far, image analysis and more complex AI-based tools have demonstrated excellent success performing tasks such as the quantification of breast biomarkers and Ki67, mitosis detection, lymph node metastasis recognition, tissue segmentation for diagnosing breast carcinoma, prognostication, computational assessment of tumor-infiltrating lymphocytes, and prediction of molecular expression as well as treatment response and benefit of therapy from routine H&E images. This review critically examines the literature regarding these applications of AI in the area of breast pathology.
Collapse
Affiliation(s)
- Mustafa Yousif
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA.
- Department of Pathology, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Paul J van Diest
- Department of Pathology, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Arvydas Laurinavicius
- Department of Pathology, Pharmacology and Forensic Medicine, Faculty of Medicine, Vilnius University, and National Center of Pathology, Affiliate of Vilnius University Hospital Santaros Klinikos, Vilnius, Lithuania
| | - David Rimm
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| | - Jeroen van der Laak
- Department of Pathology, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Anant Madabhushi
- Department of Biomedical Engineering, Case Western Reserve University, and Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
- Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA
| | - Stuart Schnitt
- Department of Pathology, Brigham and Women's Hospital and Harvard Medical School, Breast Oncology Program, Dana-Farber/Brigham and Women's Cancer Center, Boston, MA, USA
| | | |
Collapse
|
36
|
Brockmoeller S, Echle A, Ghaffari Laleh N, Eiholm S, Malmstrøm ML, Plato Kuhlmann T, Levic K, Grabsch HI, West NP, Saldanha OL, Kouvidi K, Bono A, Heij LR, Brinker TJ, Gögenür I, Quirke P, Kather JN. Deep Learning identifies inflamed fat as a risk factor for lymph node metastasis in early colorectal cancer. J Pathol 2021; 256:269-281. [PMID: 34738636 DOI: 10.1002/path.5831] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Revised: 10/18/2021] [Accepted: 11/01/2021] [Indexed: 11/07/2022]
Abstract
The spread of early-stage (T1 and T2) adenocarcinomas to loco-regional lymph nodes is a key event in disease progression of colorectal cancer (CRC). The cellular mechanisms behind this event are not completely understood and existing predictive biomarkers are imperfect. Here, we used an end-to-end Deep Learning algorithm to identify risk factors for lymph node metastasis (LNM) status in digitized histopathology slides of the primary CRC and its surrounding tissue. In two large population-based cohorts, we show that this system can predict the presence of more than one LNM in pT2 CRC patients with an area under the receiver operating curve (AUROC) of 0.733 (0.67-0.758) and patients with any LNM with an AUROC of 0.711 (0.597-0.797). Similarly, in pT1 CRC patients, the presence of more than one LNM or any LNM was predictable with an AUROC of 0.733 (0.644-0.778) and 0.567 (0.542-0.597), respectively. Based on these findings, we used the Deep Learning system to guide human pathology experts towards highly predictive regions for LNM in the whole slide images. This hybrid human observer and Deep Learning approach identified inflamed adipose tissue as the highest predictive feature for LNM presence. Our study is a first proof of concept that artificial intelligence (AI) systems may be able to discover potentially new biological mechanisms in cancer progression. Our Deep Learning algorithm is publicly available and can be used for biomarker discovery in any disease setting. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Scarlet Brockmoeller
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Amelie Echle
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | | | - Susanne Eiholm
- Department of Pathology, Zealand University Hospital, University of Copenhagen, Roskilde, Denmark
| | | | | | - Katarina Levic
- Department of Surgery, Herlev University Hospital, Copenhagen, Denmark
| | - Heike Irmgard Grabsch
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
| | - Nicholas P West
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | | | - Katerina Kouvidi
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Aurora Bono
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Lara R Heij
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands
- Institute of Pathology, University Hospital RWTH Aachen, Aachen, Germany
- Department of Surgery and Transplantation, University Hospital RWTH Aachen, Aachen, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, National Center for Tumour Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Ismayil Gögenür
- Department of Surgery, Zealand University Hospital, University of Copenhagen, Køge, Denmark
- Gastrounit - Surgical Division, Center for Surgical Research, Copenhagen University Hospital Hvidovre, Copenhagen, Denmark
| | - Philip Quirke
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Jakob Nikolas Kather
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
- Medical Oncology, National Center of Tumour Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| |
Collapse
|
37
|
Abstract
Ferroptosis is a regulated form of non-apoptotic cell death implicated in pathological settings. To be exploited clinically, ferroptosis requires reagents that unequivocally detect ferroptosis in human and animal tissues. Such tools may enable development of ferroptosis-based medicines for diverse diseases.
Collapse
|
38
|
Pourasad Y, Zarouri E, Salemizadeh Parizi M, Salih Mohammed A. Presentation of Novel Architecture for Diagnosis and Identifying Breast Cancer Location Based on Ultrasound Images Using Machine Learning. Diagnostics (Basel) 2021; 11:diagnostics11101870. [PMID: 34679568 PMCID: PMC8534593 DOI: 10.3390/diagnostics11101870] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 10/02/2021] [Accepted: 10/03/2021] [Indexed: 12/14/2022] Open
Abstract
Breast cancer is one of the main causes of death among women worldwide. Early detection of this disease helps reduce the number of premature deaths. This research aims to design a method for identifying and diagnosing breast tumors based on ultrasound images. For this purpose, six techniques have been performed to detect and segment ultrasound images. Features of images are extracted using the fractal method. Moreover, k-nearest neighbor, support vector machine, decision tree, and Naïve Bayes classification techniques are used to classify images. Then, the convolutional neural network (CNN) architecture is designed to classify breast cancer based on ultrasound images directly. The presented model obtains the accuracy of the training set to 99.8%. Regarding the test results, this diagnosis validation is associated with 88.5% sensitivity. Based on the findings of this study, it can be concluded that the proposed high-potential CNN algorithm can be used to diagnose breast cancer from ultrasound images. The second presented CNN model can identify the original location of the tumor. The results show 92% of the images in the high-performance region with an AUC above 0.6. The proposed model can identify the tumor's location and volume by morphological operations as a post-processing algorithm. These findings can also be used to monitor patients and prevent the growth of the infected area.
Collapse
Affiliation(s)
- Yaghoub Pourasad
- Department of Electrical Engineering, Urmia University of Technology (UUT), Urmia 57166-93188, Iran
- Correspondence: ; Tel.: +98-4431980226
| | - Esmaeil Zarouri
- School of Electrical Engineering, Electronic Engineering, Iran University of Science and Technology—IUST, Tehran 16846-13114, Iran;
| | | | - Amin Salih Mohammed
- Department of Computer Engineering, College of Engineering and Computer Science, Lebanese French University, Erbil 44001, Iraq;
- Department of Software and Informatics Engineering, Salahaddin University, Erbil 44002, Iraq
| |
Collapse
|
39
|
Schrammen PL, Ghaffari Laleh N, Echle A, Truhn D, Schulz V, Brinker TJ, Brenner H, Chang-Claude J, Alwers E, Brobeil A, Kloor M, Heij LR, Jäger D, Trautwein C, Grabsch HI, Quirke P, West NP, Hoffmeister M, Kather JN. Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology. J Pathol 2021; 256:50-60. [PMID: 34561876 DOI: 10.1002/path.5800] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2021] [Revised: 08/01/2021] [Accepted: 09/03/2021] [Indexed: 12/17/2022]
Abstract
Deep learning is a powerful tool in computational pathology: it can be used for tumor detection and for predicting genetic alterations based on histopathology images alone. Conventionally, tumor detection and prediction of genetic alterations are two separate workflows. Newer methods have combined them, but require complex, manually engineered computational pipelines, restricting reproducibility and robustness. To address these issues, we present a new method for simultaneous tumor detection and prediction of genetic alterations: The Slide-Level Assessment Model (SLAM) uses a single off-the-shelf neural network to predict molecular alterations directly from routine pathology slides without any manual annotations, improving upon previous methods by automatically excluding normal and non-informative tissue regions. SLAM requires only standard programming libraries and is conceptually simpler than previous approaches. We have extensively validated SLAM for clinically relevant tasks using two large multicentric cohorts of colorectal cancer patients, Darmkrebs: Chancen der Verhütung durch Screening (DACHS) from Germany and Yorkshire Cancer Research Bowel Cancer Improvement Programme (YCR-BCIP) from the UK. We show that SLAM yields reliable slide-level classification of tumor presence with an area under the receiver operating curve (AUROC) of 0.980 (confidence interval 0.975, 0.984; n = 2,297 tumor and n = 1,281 normal slides). In addition, SLAM can detect microsatellite instability (MSI)/mismatch repair deficiency (dMMR) or microsatellite stability/mismatch repair proficiency with an AUROC of 0.909 (0.888, 0.929; n = 2,039 patients) and BRAF mutational status with an AUROC of 0.821 (0.786, 0.852; n = 2,075 patients). The improvement with respect to previous methods was validated in a large external testing cohort in which MSI/dMMR status was detected with an AUROC of 0.900 (0.864, 0.931; n = 805 patients). In addition, SLAM provides human-interpretable visualization maps, enabling the analysis of multiplexed network predictions by human experts. In summary, SLAM is a new simple and powerful method for computational pathology that could be applied to multiple disease contexts. © 2021 The Authors. The Journal of Pathology published by John Wiley & Sons, Ltd. on behalf of The Pathological Society of Great Britain and Ireland.
Collapse
Affiliation(s)
| | | | - Amelie Echle
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Daniel Truhn
- Department of Radiology, University Hospital RWTH Aachen, Aachen, Germany
| | - Volkmar Schulz
- Department of Physics of Molecular Imaging Systems, Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany.,Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany.,Comprehensive Diagnostic Center Aachen (CDCA), University Hospital Aachen, Aachen, Germany.,Hyperion Hybrid Imaging Systems GmbH, Aachen, Germany
| | - Titus J Brinker
- Digital Biomarkers for Oncology Group, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Hermann Brenner
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Division of Preventive Oncology, German Cancer Research Center (DKFZ) and National Center for Tumor Diseases (NCT), Heidelberg, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jenny Chang-Claude
- Division of Cancer Epidemiology, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Cancer Epidemiology Group, University Cancer Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Elizabeth Alwers
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Brobeil
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany.,Tumor Bank Unit, Tissue Bank of the National Center for Tumor Diseases, Heidelberg, Germany
| | - Matthias Kloor
- Institute of Pathology, University Hospital Heidelberg, Heidelberg, Germany
| | - Lara R Heij
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands.,Institute of Pathology, University Hospital RWTH Aachen, Aachen, Germany.,Department of Surgery and Transplantation, University Hospital RWTH Aachen, Aachen, Germany
| | - Dirk Jäger
- Medical Oncology, National Center of Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany
| | - Christian Trautwein
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany
| | - Heike I Grabsch
- Department of Pathology, GROW School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht, The Netherlands.,Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Philip Quirke
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Nicholas P West
- Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Michael Hoffmeister
- Division of Clinical Epidemiology and Aging Research, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Jakob Nikolas Kather
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.,German Cancer Consortium (DKTK), German Cancer Research Center (DKFZ), Heidelberg, Germany.,Medical Oncology, National Center of Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.,Pathology & Data Analytics, Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| |
Collapse
|
40
|
Keith JA, Vassilev-Galindo V, Cheng B, Chmiela S, Gastegger M, Müller KR, Tkatchenko A. Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems. Chem Rev 2021; 121:9816-9872. [PMID: 34232033 PMCID: PMC8391798 DOI: 10.1021/acs.chemrev.1c00107] [Citation(s) in RCA: 164] [Impact Index Per Article: 54.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2021] [Indexed: 12/23/2022]
Abstract
Machine learning models are poised to make a transformative impact on chemical sciences by dramatically accelerating computational algorithms and amplifying insights available from computational chemistry methods. However, achieving this requires a confluence and coaction of expertise in computer science and physical sciences. This Review is written for new and experienced researchers working at the intersection of both fields. We first provide concise tutorials of computational chemistry and machine learning methods, showing how insights involving both can be achieved. We follow with a critical review of noteworthy applications that demonstrate how computational chemistry and machine learning can be used together to provide insightful (and useful) predictions in molecular and materials modeling, retrosyntheses, catalysis, and drug design.
Collapse
Affiliation(s)
- John A. Keith
- Department
of Chemical and Petroleum Engineering Swanson School of Engineering, University of Pittsburgh, Pittsburgh, Pennsylvania 15261, United States
| | - Valentin Vassilev-Galindo
- Department
of Physics and Materials Science, University
of Luxembourg, L-1511 Luxembourg City, Luxembourg
| | - Bingqing Cheng
- Accelerate
Programme for Scientific Discovery, Department
of Computer Science and Technology, 15 J. J. Thomson Avenue, Cambridge CB3 0FD, United Kingdom
| | - Stefan Chmiela
- Department
of Software Engineering and Theoretical Computer Science, Technische Universität Berlin, 10587, Berlin, Germany
| | - Michael Gastegger
- Department
of Software Engineering and Theoretical Computer Science, Technische Universität Berlin, 10587, Berlin, Germany
| | - Klaus-Robert Müller
- Machine
Learning Group, Technische Universität
Berlin, 10587, Berlin, Germany
- Department
of Artificial Intelligence, Korea University, Anam-dong, Seongbuk-gu, Seoul, 02841, Korea
- Max-Planck-Institut für Informatik, 66123 Saarbrücken, Germany
- Google Research, Brain Team, 10117 Berlin, Germany
| | - Alexandre Tkatchenko
- Department
of Physics and Materials Science, University
of Luxembourg, L-1511 Luxembourg City, Luxembourg
| |
Collapse
|
41
|
Abstract
Interpreting the effects of genetic variants is key to understanding individual susceptibility to disease and designing personalized therapeutic approaches. Modern experimental technologies are enabling the generation of massive compendia of human genome sequence data and associated molecular and phenotypic traits, together with genome-scale expression, epigenomics and other functional genomic data. Integrative computational models can leverage these data to understand variant impact, elucidate the effect of dysregulated genes on biological pathways in specific disease and tissue contexts, and interpret disease risk beyond what is feasible with experiments alone. In this Review, we discuss recent developments in machine learning algorithms for genome interpretation and for integrative molecular-level modelling of cells, tissues and organs relevant to disease. More specifically, we highlight existing methods and key challenges and opportunities in identifying specific disease-causing genetic variants and linking them to molecular pathways and, ultimately, to disease phenotypes.
Collapse
|
42
|
Jin Y, Junren W, Jingwen J, Yajing S, Xi C, Ke Q. Research on the Construction and Application of Breast Cancer-Specific Database System Based on Full Data Lifecycle. Front Public Health 2021; 9:712827. [PMID: 34322474 PMCID: PMC8311352 DOI: 10.3389/fpubh.2021.712827] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2021] [Accepted: 06/14/2021] [Indexed: 02/05/2023] Open
Abstract
Relying on the Biomedical Big Data Center of West China Hospital, this paper makes an in-depth research on the construction method and application of breast cancer-specific database system based on full data lifecycle, including the establishment of data standards, data fusion and governance, multi-modal knowledge graph, data security sharing and value application of breast cancer-specific database. The research was developed by establishing the breast cancer master data and metadata standards, then collecting, mapping and governing the structured and unstructured clinical data, and parsing and processing the electronic medical records with NLP natural language processing method or other applicable methods, as well as constructing the breast cancer-specific database system to support the application of data in clinical practices, scientific research, and teaching in hospitals, giving full play to the value of medical big data of the Biomedical Big Data Center of West China Hospital.
Collapse
Affiliation(s)
- Yin Jin
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Wang Junren
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
- Medical Big Data Center, Sichuan University, Chengdu, China
| | - Jiang Jingwen
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
- Medical Big Data Center, Sichuan University, Chengdu, China
| | - Sun Yajing
- West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
- Medical Big Data Center, Sichuan University, Chengdu, China
| | - Chen Xi
- Chengdu Zhixin Electronic Technology Co., Ltd, Chengdu, China
| | - Qin Ke
- School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
43
|
Shaban-Nejad A, Michalowski M, Brownstein J, Buckeridge D. Guest Editorial Explainable AI: Towards Fairness, Accountability, Transparency and Trust in Healthcare. IEEE J Biomed Health Inform 2021. [DOI: 10.1109/jbhi.2021.3088832] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
44
|
Altaf MM. A hybrid deep learning model for breast cancer diagnosis based on transfer learning and pulse-coupled neural networks. Math Biosci Eng 2021; 18:5029-5046. [PMID: 34517476 DOI: 10.3934/mbe.2021256] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Radiology experts often face difficulties in mammography mass lesion labeling, which may lead to conclusive yet unnecessary and expensive breast biopsies. This paper focuses on building an automated diagnosis tool that supports radiologists in identifying and classifying mammography mass lesions. The paper's main contribution is to design a hybrid model based on Pulse-Coupled Neural Networks (PCNN) and Deep Convolutional Neural Networks (CNN). Due to the need for large datasets to train and tune CNNs, which are not available for medical images, Transfer Learning (TL) was exploited in this research. TL can be an effective approach when working with small-sized datasets. The paper's implementation was tested on three public benchmark datasets: DDMS, INbreast, and BCDR datasets for training and testing and MIAS for testing only. The results indicated the enhancement that PCNN provides when combined with CNN compared to other methods for the same public datasets. The hybrid model achieved 98.72% accuracy for DDMS, 97.5% for INbreast, and 96.94% for BCDR. To avoid overfitting, the proposed hybrid model was tested on an unseen MIAS dataset, achieving 98.77% accuracy. Other evaluation metrics are reported in the results section.
Collapse
Affiliation(s)
- Meteb M Altaf
- National Center for Robotics Technology and Internet of Things, King Abdulaziz City for Science and Technology, Riyadh, Saudi Arabia
| |
Collapse
|