1
|
Shao W, Shi H, Liu J, Zuo Y, Sun L, Xia T, Chen W, Wan P, Sheng J, Zhu Q, Zhang D. Multi-Instance Multi-Task Learning for Joint Clinical Outcome and Genomic Profile Predictions From the Histopathological Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2266-2278. [PMID: 38319755 DOI: 10.1109/tmi.2024.3362852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
With the remarkable success of digital histopathology and the deep learning technology, many whole-slide pathological images (WSIs) based deep learning models are designed to help pathologists diagnose human cancers. Recently, rather than predicting categorical variables as in cancer diagnosis, several deep learning studies are also proposed to estimate the continuous variables such as the patients' survival or their transcriptional profile. However, most of the existing studies focus on conducting these predicting tasks separately, which overlooks the useful intrinsic correlation among them that can boost the prediction performance of each individual task. In addition, it is sill challenge to design the WSI-based deep learning models, since a WSI is with huge size but annotated with coarse label. In this study, we propose a general multi-instance multi-task learning framework (HistMIMT) for multi-purpose prediction from WSIs. Specifically, we firstly propose a novel multi-instance learning module (TMICS) considering both common and specific task information across different tasks to generate bag representation for each individual task. Then, a soft-mask based fusion module with channel attention (SFCA) is developed to leverage useful information from the related tasks to help improve the prediction performance on target task. We evaluate our method on three cancer cohorts derived from the Cancer Genome Atlas (TCGA). For each cohort, our multi-purpose prediction tasks range from cancer diagnosis, survival prediction and estimating the transcriptional profile of gene TP53. The experimental results demonstrated that HistMIMT can yield better outcome on all clinical prediction tasks than its competitors.
Collapse
|
2
|
Son R, Yamazawa K, Oguchi A, Suga M, Tamura M, Yanagita M, Murakawa Y, Kume S. Morphomics via next-generation electron microscopy. J Mol Cell Biol 2024; 15:mjad081. [PMID: 38148118 PMCID: PMC11167312 DOI: 10.1093/jmcb/mjad081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Revised: 10/02/2022] [Accepted: 12/23/2023] [Indexed: 12/28/2023] Open
Abstract
The living body is composed of innumerable fine and complex structures. Although these structures have been studied in the past, a vast amount of information pertaining to them still remains unknown. When attempting to observe these ultra-structures, the use of electron microscopy (EM) has become indispensable. However, conventional EM settings are limited to a narrow tissue area, which can bias observations. Recently, new trends in EM research have emerged, enabling coverage of far broader, nano-scale fields of view for two-dimensional wide areas and three-dimensional large volumes. Moreover, cutting-edge bioimage informatics conducted via deep learning has accelerated the quantification of complex morphological bioimages. Taken together, these technological and analytical advances have led to the comprehensive acquisition and quantification of cellular morphology, which now arises as a new omics science termed 'morphomics'.
Collapse
Affiliation(s)
- Raku Son
- RIKEN-IFOM Joint Laboratory for Cancer Genomics, RIKEN Center for Integrative Medical Sciences, Yokohama 230-0045, Japan
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Kenji Yamazawa
- Advanced Manufacturing Support Team, RIKEN Center for Advanced Photonics, Wako 351-0198, Japan
| | - Akiko Oguchi
- RIKEN-IFOM Joint Laboratory for Cancer Genomics, RIKEN Center for Integrative Medical Sciences, Yokohama 230-0045, Japan
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
| | - Mitsuo Suga
- Multimodal Microstructure Analysis Unit, RIKEN–JEOL Collaboration Center, Kobe 650-0047, Japan
| | - Masaru Tamura
- Technology and Development Team for Mouse Phenotype Analysis, RIKEN BioResource Research Center, Tsukuba 305-0074, Japan
| | - Motoko Yanagita
- Department of Nephrology, Graduate School of Medicine, Kyoto University, Kyoto 606-8507, Japan
- Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto 606-8501, Japan
| | - Yasuhiro Murakawa
- RIKEN-IFOM Joint Laboratory for Cancer Genomics, RIKEN Center for Integrative Medical Sciences, Yokohama 230-0045, Japan
- Institute for the Advanced Study of Human Biology (ASHBi), Kyoto University, Kyoto 606-8501, Japan
- IFOM—The FIRC Institute of Molecular Oncology, Milan 20139, Italy
| | - Satoshi Kume
- Laboratory for Pathophysiological and Health Science, RIKEN Center for Biosystems Dynamics Research, Kobe 650-0047, Japan
- Center for Health Science Innovation, Osaka City University, Osaka 530-0011, Japan
- Osaka Electro-Communication University, Neyagawa 572-8530, Japan
| |
Collapse
|
3
|
Pateras IS, Igea A, Nikas IP, Leventakou D, Koufopoulos NI, Ieronimaki AI, Bergonzini A, Ryu HS, Chatzigeorgiou A, Frisan T, Kittas C, Panayiotides IG. Diagnostic Challenges during Inflammation and Cancer: Current Biomarkers and Future Perspectives in Navigating through the Minefield of Reactive versus Dysplastic and Cancerous Lesions in the Digestive System. Int J Mol Sci 2024; 25:1251. [PMID: 38279253 PMCID: PMC10816510 DOI: 10.3390/ijms25021251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 01/12/2024] [Accepted: 01/16/2024] [Indexed: 01/28/2024] Open
Abstract
In the setting of pronounced inflammation, changes in the epithelium may overlap with neoplasia, often rendering it impossible to establish a diagnosis with certainty in daily clinical practice. Here, we discuss the underlying molecular mechanisms driving tissue response during persistent inflammatory signaling along with the potential association with cancer in the gastrointestinal tract, pancreas, extrahepatic bile ducts, and liver. We highlight the histopathological challenges encountered in the diagnosis of chronic inflammation in routine practice and pinpoint tissue-based biomarkers that could complement morphology to differentiate reactive from dysplastic or cancerous lesions. We refer to the advantages and limitations of existing biomarkers employing immunohistochemistry and point to promising new markers, including the generation of novel antibodies targeting mutant proteins, miRNAs, and array assays. Advancements in experimental models, including mouse and 3D models, have improved our understanding of tissue response. The integration of digital pathology along with artificial intelligence may also complement routine visual inspections. Navigating through tissue responses in various chronic inflammatory contexts will help us develop novel and reliable biomarkers that will improve diagnostic decisions and ultimately patient treatment.
Collapse
Affiliation(s)
- Ioannis S. Pateras
- 2nd Department of Pathology, “Attikon” University Hospital, Medical School, National and Kapodistrian University of Athens, 124 62 Athens, Greece; (D.L.); (N.I.K.); (A.I.I.); (I.G.P.)
| | - Ana Igea
- Instituto de Investigación Sanitaria de Santiago de Compostela (IDIS), 15706 Santiago de Compostela, Spain;
- Mobile Genomes, Centre for Research in Molecular Medicine and Chronic Diseases (CiMUS), University of Santiago de Compostela (USC), 15706 Santiago de Compostela, Spain
| | - Ilias P. Nikas
- Medical School, University of Cyprus, 2029 Nicosia, Cyprus
| | - Danai Leventakou
- 2nd Department of Pathology, “Attikon” University Hospital, Medical School, National and Kapodistrian University of Athens, 124 62 Athens, Greece; (D.L.); (N.I.K.); (A.I.I.); (I.G.P.)
| | - Nektarios I. Koufopoulos
- 2nd Department of Pathology, “Attikon” University Hospital, Medical School, National and Kapodistrian University of Athens, 124 62 Athens, Greece; (D.L.); (N.I.K.); (A.I.I.); (I.G.P.)
| | - Argyro Ioanna Ieronimaki
- 2nd Department of Pathology, “Attikon” University Hospital, Medical School, National and Kapodistrian University of Athens, 124 62 Athens, Greece; (D.L.); (N.I.K.); (A.I.I.); (I.G.P.)
| | - Anna Bergonzini
- Center for Infectious Medicine, Department of Medicine Huddinge, Karolinska Institutet, Karolinska University Hospital, Alfred Nobels Allé 8, 141 52 Stockholm, Sweden;
- Department of Molecular Biology and Umeå Centre for Microbial Research (UCMR), Umeå University, 901 87 Umeå, Sweden;
| | - Han Suk Ryu
- Department of Pathology, Seoul National University Hospital, Seoul 03080, Republic of Korea;
| | - Antonios Chatzigeorgiou
- Department of Physiology, Medical School, National and Kapodistrian University of Athens, 115 27 Athens, Greece;
| | - Teresa Frisan
- Department of Molecular Biology and Umeå Centre for Microbial Research (UCMR), Umeå University, 901 87 Umeå, Sweden;
| | - Christos Kittas
- Department of Histopathology, Biomedicine Group of Health Company, 156 26 Athens, Greece;
| | - Ioannis G. Panayiotides
- 2nd Department of Pathology, “Attikon” University Hospital, Medical School, National and Kapodistrian University of Athens, 124 62 Athens, Greece; (D.L.); (N.I.K.); (A.I.I.); (I.G.P.)
| |
Collapse
|
4
|
Tak S, Han G, Leem SH, Lee SY, Paek K, Kim JA. Prediction of anticancer drug resistance using a 3D microfluidic bladder cancer model combined with convolutional neural network-based image analysis. Front Bioeng Biotechnol 2024; 11:1302983. [PMID: 38268938 PMCID: PMC10806080 DOI: 10.3389/fbioe.2023.1302983] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Accepted: 12/28/2023] [Indexed: 01/26/2024] Open
Abstract
Bladder cancer is the most common urological malignancy worldwide, and its high recurrence rate leads to poor survival outcomes. The effect of anticancer drug treatment varies significantly depending on individual patients and the extent of drug resistance. In this study, we developed a validation system based on an organ-on-a-chip integrated with artificial intelligence technologies to predict resistance to anticancer drugs in bladder cancer. As a proof-of-concept, we utilized the gemcitabine-resistant bladder cancer cell line T24 with four distinct levels of drug resistance (parental, early, intermediate, and late). These cells were co-cultured with endothelial cells in a 3D microfluidic chip. A dataset comprising 2,674 cell images from the chips was analyzed using a convolutional neural network (CNN) to distinguish the extent of drug resistance among the four cell groups. The CNN achieved 95.2% accuracy upon employing data augmentation and a step decay learning rate with an initial value of 0.001. The average diagnostic sensitivity and specificity were 90.5% and 96.8%, respectively, and all area under the curve (AUC) values were over 0.988. Our proposed method demonstrated excellent performance in accurately identifying the extent of drug resistance, which can assist in the prediction of drug responses and in determining the appropriate treatment for bladder cancer patients.
Collapse
Affiliation(s)
- Sungho Tak
- Research Center for Bioconvergence Analysis, Korea Basic Science Institute, Cheongju, Republic of Korea
- Graduate School of Analytical Science and Technology, Chungnam National University, Daejeon, Republic of Korea
| | - Gyeongjin Han
- Research Center for Bioconvergence Analysis, Korea Basic Science Institute, Cheongju, Republic of Korea
| | - Sun-Hee Leem
- Department of Biomedical Sciences, Dong-A University, Busan, Republic of Korea
- Department of Health Sciences, The Graduate School of Dong-A University, Busan, Republic of Korea
| | - Sang-Yeop Lee
- Research Center for Bioconvergence Analysis, Korea Basic Science Institute, Cheongju, Republic of Korea
| | - Kyurim Paek
- Center for Scientific Instrumentation, Korea Basic Science Institute, Daejeon, Republic of Korea
| | - Jeong Ah Kim
- Center for Scientific Instrumentation, Korea Basic Science Institute, Daejeon, Republic of Korea
- Department of Bio-Analytical Science, University of Science and Technology, Daejeon, Republic of Korea
- Chung-Ang University Hospital, Chung-Ang University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
5
|
Zhang D, Schroeder A, Yan H, Yang H, Hu J, Lee MYY, Cho KS, Susztak K, Xu GX, Feldman MD, Lee EB, Furth EE, Wang L, Li M. Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology. Nat Biotechnol 2024:10.1038/s41587-023-02019-9. [PMID: 38168986 DOI: 10.1038/s41587-023-02019-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 10/04/2023] [Indexed: 01/05/2024]
Abstract
Spatial transcriptomics (ST) has demonstrated enormous potential for generating intricate molecular maps of cells within tissues. Here we present iStar, a method based on hierarchical image feature extraction that integrates ST data and high-resolution histology images to predict spatial gene expression with super-resolution. Our method enhances gene expression resolution to near-single-cell levels in ST and enables gene expression prediction in tissue sections where only histology images are available.
Collapse
Affiliation(s)
- Daiwei Zhang
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| | - Amelia Schroeder
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Hanying Yan
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Haochen Yang
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Jian Hu
- Department of Human Genetics, School of Medicine, Emory University, Atlanta, GA, USA
| | - Michelle Y Y Lee
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Graduate Group in Genomics and Computational Biology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Kyung S Cho
- Department of Genomic Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Katalin Susztak
- Renal, Electrolyte, and Hypertension Division, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - George X Xu
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael D Feldman
- Department of Pathology and Laboratory Medicine, School of Medicine, Indiana University, Indianapolis, IN, USA
| | - Edward B Lee
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Emma E Furth
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Linghua Wang
- Department of Genomic Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mingyao Li
- Statistical Center for Single-Cell and Spatial Genomics, Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| |
Collapse
|
6
|
Xiao X, Kong Y, Li R, Wang Z, Lu H. Transformer with convolution and graph-node co-embedding: An accurate and interpretable vision backbone for predicting gene expressions from local histopathological image. Med Image Anal 2024; 91:103040. [PMID: 38007979 DOI: 10.1016/j.media.2023.103040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 11/04/2023] [Accepted: 11/17/2023] [Indexed: 11/28/2023]
Abstract
Inferring gene expressions from histopathological images has long been a fascinating yet challenging task, primarily due to the substantial disparities between the two modality. Existing strategies using local or global features of histological images are suffering model complexity, GPU consumption, low interpretability, insufficient encoding of local features, and over-smooth prediction of gene expressions among neighboring sites. In this paper, we develop TCGN (Transformer with Convolution and Graph-Node co-embedding method) for gene expression estimation from H&E-stained pathological slide images. TCGN comprises a combination of convolutional layers, transformer encoders, and graph neural networks, and is the first to integrate these blocks in a general and interpretable computer vision backbone. Notably, TCGN uniquely operates with just a single spot image as input for histopathological image analysis, simplifying the process while maintaining interpretability. We validate TCGN on three publicly available spatial transcriptomic datasets. TCGN consistently exhibited the best performance (with median PCC 0.232). TCGN offers superior accuracy while keeping parameters to a minimum (just 86.241 million), and it consumes minimal memory, allowing it to run smoothly even on personal computers. Moreover, TCGN can be extended to handle bulk RNA-seq data while providing the interpretability. Enhancing the accuracy of omics information prediction from pathological images not only establishes a connection between genotype and phenotype, enabling the prediction of costly-to-measure biomarkers from affordable histopathological images, but also lays the groundwork for future multi-modal data modeling. Our results confirm that TCGN is a powerful tool for inferring gene expressions from histopathological images in precision health applications.
Collapse
Affiliation(s)
- Xiao Xiao
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China; SJTU-Yale Joint Center for Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China; Department of Biostatistics, Yale School of Public Health, Yale University, New Haven, CT, United States
| | - Yan Kong
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China; SJTU-Yale Joint Center for Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
| | - Ronghan Li
- SJTU-Yale Joint Center for Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China; Zhiyuan College, Shanghai Jiao Tong University, Shanghai, China
| | - Zuoheng Wang
- Department of Biostatistics, Yale School of Public Health, Yale University, New Haven, CT, United States
| | - Hui Lu
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, Department of Bioinformatics and Biostatistics, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China; SJTU-Yale Joint Center for Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China; NHC Key Laboratory of Medical Embryogenesis and Developmental Molecular Biology & Shanghai Key Laboratory of Embryo and Reproduction Engineering, Shanghai Engineering Research Center for Big Data in Pediatric Precision Medicine, Shanghai, China.
| |
Collapse
|
7
|
Duroux D, Wohlfart C, Van Steen K, Vladimirova A, King M. Graph-based multi-modality integration for prediction of cancer subtype and severity. Sci Rep 2023; 13:19653. [PMID: 37949935 PMCID: PMC10638406 DOI: 10.1038/s41598-023-46392-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 10/31/2023] [Indexed: 11/12/2023] Open
Abstract
Personalised cancer screening before therapy paves the way toward improving diagnostic accuracy and treatment outcomes. Most approaches are limited to a single data type and do not consider interactions between features, leaving aside the complementary insights that multimodality and systems biology can provide. In this project, we demonstrate the use of graph theory for data integration via individual networks where nodes and edges are individual-specific. We showcase the consequences of early, intermediate, and late graph-based fusion of RNA-Seq data and histopathology whole-slide images for predicting cancer subtypes and severity. The methodology developed is as follows: (1) we create individual networks; (2) we compute the similarity between individuals from these graphs; (3) we train our model on the similarity matrices; (4) we evaluate the performance using the macro F1 score. Pros and cons of elements of the pipeline are evaluated on publicly available real-life datasets. We find that graph-based methods can increase performance over methods that do not study interactions. Additionally, merging multiple data sources often improves classification compared to models based on single data, especially through intermediate fusion. The proposed workflow can easily be adapted to other disease contexts to accelerate and enhance personalized healthcare.
Collapse
Affiliation(s)
- Diane Duroux
- BIO3 - Systems Genetics, GIGA-R Medical Genomics, University of Liège, 4000, Liège, Belgium.
- Post-Doctoral Fellow, ETH AI center, Zürich, Switzerland.
| | | | - Kristel Van Steen
- BIO3 - Systems Genetics, GIGA-R Medical Genomics, University of Liège, 4000, Liège, Belgium
- Department of Human Genetics, BIO3 - Systems Medicine, 3000, Leuven, Belgium
| | - Antoaneta Vladimirova
- Roche Information Solutions, Roche Diagnostics Corporation, Santa Clara, California, United States of America
| | | |
Collapse
|
8
|
Garrido-Martín D, Calvo M, Reverter F, Guigó R. A fast non-parametric test of association for multiple traits. Genome Biol 2023; 24:230. [PMID: 37828616 PMCID: PMC10571397 DOI: 10.1186/s13059-023-03076-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Accepted: 09/27/2023] [Indexed: 10/14/2023] Open
Abstract
The increasing availability of multidimensional phenotypic data in large cohorts of genotyped individuals requires efficient methods to identify genetic effects on multiple traits. Permutational multivariate analysis of variance (PERMANOVA) offers a powerful non-parametric approach. However, it relies on permutations to assess significance, which hinders the analysis of large datasets. Here, we derive the limiting null distribution of the PERMANOVA test statistic, providing a framework for the fast computation of asymptotic p values. Our asymptotic test presents controlled type I error and high power, often outperforming parametric approaches. We illustrate its applicability in the context of QTL mapping and GWAS.
Collapse
Affiliation(s)
- Diego Garrido-Martín
- Department of Genetics, Microbiology and Statistics, Universitat de Barcelona (UB), Av. Diagonal 643, Barcelona, 08028, Spain.
- Centre for Genomic Regulation (CRG), The Barcelona Institute of Science and Technology, Dr. Aiguader 88, Barcelona, 08003, Catalonia, Spain.
| | - Miquel Calvo
- Department of Genetics, Microbiology and Statistics, Universitat de Barcelona (UB), Av. Diagonal 643, Barcelona, 08028, Spain
| | - Ferran Reverter
- Department of Genetics, Microbiology and Statistics, Universitat de Barcelona (UB), Av. Diagonal 643, Barcelona, 08028, Spain
| | - Roderic Guigó
- Centre for Genomic Regulation (CRG), The Barcelona Institute of Science and Technology, Dr. Aiguader 88, Barcelona, 08003, Catalonia, Spain
- Universitat Pompeu Fabra (UPF), Barcelona, Catalonia, Spain
| |
Collapse
|
9
|
Wang JM, Hong R, Demicco EG, Tan J, Lazcano R, Moreira AL, Li Y, Calinawan A, Razavian N, Schraink T, Gillette MA, Omenn GS, An E, Rodriguez H, Tsirigos A, Ruggles KV, Ding L, Robles AI, Mani DR, Rodland KD, Lazar AJ, Liu W, Fenyö D. Deep learning integrates histopathology and proteogenomics at a pan-cancer level. Cell Rep Med 2023; 4:101173. [PMID: 37582371 PMCID: PMC10518635 DOI: 10.1016/j.xcrm.2023.101173] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2022] [Revised: 07/31/2023] [Accepted: 08/04/2023] [Indexed: 08/17/2023]
Abstract
We introduce a pioneering approach that integrates pathology imaging with transcriptomics and proteomics to identify predictive histology features associated with critical clinical outcomes in cancer. We utilize 2,755 H&E-stained histopathological slides from 657 patients across 6 cancer types from CPTAC. Our models effectively recapitulate distinctions readily made by human pathologists: tumor vs. normal (AUROC = 0.995) and tissue-of-origin (AUROC = 0.979). We further investigate predictive power on tasks not normally performed from H&E alone, including TP53 prediction and pathologic stage. Importantly, we describe predictive morphologies not previously utilized in a clinical setting. The incorporation of transcriptomics and proteomics identifies pathway-level signatures and cellular processes driving predictive histology features. Model generalizability and interpretability is confirmed using TCGA. We propose a classification system for these tasks, and suggest potential clinical applications for this integrated human and machine learning approach. A publicly available web-based platform implements these models.
Collapse
Affiliation(s)
- Joshua M Wang
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Runyu Hong
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Elizabeth G Demicco
- Department of Pathology and Laboratory Medicine, Mount Sinai Hospital and Laboratory Medicine and Pathobiology, University of Toronto, Toronto M5G 1X5, ON, Canada
| | - Jimin Tan
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA; Division of Precision Medicine, Department of Medicine, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Rossana Lazcano
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA
| | - Andre L Moreira
- Department of Pathology, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Yize Li
- Department of Medicine, Washington University in St. Louis, St. Louis, MO 63110, USA; McDonnell Genome Institute, Washington University in St. Louis, St. Louis, MO 63108, USA
| | - Anna Calinawan
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Icahn Institute for Data Science and Genomic Technology, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Narges Razavian
- Department of Population Health, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Radiology, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Tobias Schraink
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA; Division of Precision Medicine, Department of Medicine, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Michael A Gillette
- The Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA; Massachusetts General Hospital Division of Pulmonary and Critical Care Medicine, Boston, MA 02114, USA; Harvard Medical School, Boston, MA 02115, USA
| | - Gilbert S Omenn
- Departments of Computational Medicine & Bioinformatics, Internal Medicine, Human Genetics, and School of Public Health, University of Michigan, Ann Arbor, MI 48109, USA
| | - Eunkyung An
- Office of Cancer Clinical Proteomics Research, National Cancer Institute, Rockville, MD 20850, USA
| | - Henry Rodriguez
- Office of Cancer Clinical Proteomics Research, National Cancer Institute, Rockville, MD 20850, USA
| | - Aristotelis Tsirigos
- Department of Pathology, NYU Grossman School of Medicine, New York, NY 10016, USA; Division of Precision Medicine, Department of Medicine, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Kelly V Ruggles
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Division of Precision Medicine, Department of Medicine, NYU Grossman School of Medicine, New York, NY 10016, USA
| | - Li Ding
- Department of Medicine and Genetics, Siteman Cancer Center, Washington University in St. Louis, St. Louis, MO 63110, USA; McDonnell Genome Institute, Washington University in St. Louis, St. Louis, MO 63108, USA
| | - Ana I Robles
- Office of Cancer Clinical Proteomics Research, National Cancer Institute, Rockville, MD 20850, USA
| | - D R Mani
- The Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA
| | - Karin D Rodland
- Biological Sciences Division, Pacific Northwest National Laboratory, Richland, WA 99354, USA; Department of Cell, Developmental, and Cancer Biology, Oregon Health & Science University, Portland, OR 97221, USA
| | - Alexander J Lazar
- Department of Pathology, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA; Department of Genomic Medicine, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA.
| | - Wenke Liu
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA.
| | - David Fenyö
- Institute for Systems Genetics, NYU Grossman School of Medicine, New York, NY 10016, USA; Department of Biochemistry and Molecular Pharmacology, NYU Grossman School of Medicine, New York, NY 10016, USA.
| |
Collapse
|
10
|
Halder A, Bhowmick C, Dutta PK, Mahadevappa M. Identification and Analysis of Imaging-Genomic Signatures to Study Recurrence in Breast Cancers. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083388 DOI: 10.1109/embc40787.2023.10339965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
One of the main causes of breast cancer related death is its recurrence. In this study, we investigate the association of gene expression and pathological image features to understand breast cancer recurrence. A total of 172 breast cancer patient data was downloaded from the TCGA-BRCA database. The dataset contained diagnostic whole slide images and RNA-seq data of 80 recurrent and 92 disease-free breast cancer patients. We performed genomic analysis on RNA-seq data to obtain the hub genes related to recurrent breast cancer. We extracted relevant pathomic features from histopathology images. The discriminative ability of the hub genes and pathomic features were evaluated using machine learning classifiers. We used Spearman rank correlation analysis to find statistically significant association between gene expression and pathomic features. We identified that, genes which are related to breast cancer progression is significantly associated (adjusted p-value<0.05) with several pathomic features.Clinical Relevance- Histopathology is the gold standard for cancer detection. It provides us with cellular level information. A strong association between a pathomic feature and a gene expression will help clinicians understand the cellular and molecular mechanism of cancer for better prognosis.
Collapse
|
11
|
Wilson A, Babadi M. SynapseCLR: Uncovering features of synapses in primary visual cortex through contrastive representation learning. PATTERNS 2023; 4:100693. [PMID: 37123442 PMCID: PMC10140600 DOI: 10.1016/j.patter.2023.100693] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 11/23/2022] [Accepted: 01/25/2023] [Indexed: 03/09/2023]
Abstract
3D electron microscopy (EM) connectomics image volumes are surpassing 1 mm3, providing information-dense, multi-scale visualizations of brain circuitry and necessitating scalable analysis techniques. We present SynapseCLR, a self-supervised contrastive learning method for 3D EM data, and use it to extract features of synapses from mouse visual cortex. SynapseCLR feature representations separate synapses by appearance and functionally important structural annotations. We demonstrate SynapseCLR's utility for valuable downstream tasks, including one-shot identification of defective synapse segmentations, dataset-wide similarity-based querying, and accurate imputation of annotations for unlabeled synapses, using manual annotation of only 0.2% of the dataset's synapses. In particular, excitatory versus inhibitory neuronal types can be assigned with >99.8% accuracy to individual synapses and highly truncated neurites, enabling neurite-enhanced connectomics analysis. Finally, we present a data-driven, unsupervised study of synaptic structural variation on the representation manifold, revealing its intrinsic axes of variation and showing that representations contain inhibitory subtype information.
Collapse
|
12
|
Mou T, Liang J, Vu TN, Tian M, Gao Y. A Comprehensive Landscape of Imaging Feature-Associated RNA Expression Profiles in Human Breast Tissue. SENSORS (BASEL, SWITZERLAND) 2023; 23:1432. [PMID: 36772473 PMCID: PMC9921444 DOI: 10.3390/s23031432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Revised: 01/15/2023] [Accepted: 01/20/2023] [Indexed: 06/18/2023]
Abstract
The expression abundance of transcripts in nondiseased breast tissue varies among individuals. The association study of genotypes and imaging phenotypes may help us to understand this individual variation. Since existing reports mainly focus on tumors or lesion areas, the heterogeneity of pathological image features and their correlations with RNA expression profiles for nondiseased tissue are not clear. The aim of this study is to discover the association between the nucleus features and the transcriptome-wide RNAs. We analyzed both microscopic histology images and RNA-sequencing data of 456 breast tissues from the Genotype-Tissue Expression (GTEx) project and constructed an automatic computational framework. We classified all samples into four clusters based on their nucleus morphological features and discovered feature-specific gene sets. The biological pathway analysis was performed on each gene set. The proposed framework evaluates the morphological characteristics of the cell nucleus quantitatively and identifies the associated genes. We found image features that capture population variation in breast tissue associated with RNA expressions, suggesting that the variation in expression pattern affects population variation in the morphological traits of breast tissue. This study provides a comprehensive transcriptome-wide view of imaging-feature-specific RNA expression for healthy breast tissue. Such a framework could also be used for understanding the connection between RNA expression and morphology in other tissues and organs. Pathway analysis indicated that the gene sets we identified were involved in specific biological processes, such as immune processes.
Collapse
Affiliation(s)
- Tian Mou
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518000, China
| | - Jianwen Liang
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518000, China
| | - Trung Nghia Vu
- Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, SE 17177 Stockholm, Sweden
| | - Mu Tian
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518000, China
| | - Yi Gao
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518000, China
| |
Collapse
|
13
|
García-Pérez R, Ramirez JM, Ripoll-Cladellas A, Chazarra-Gil R, Oliveros W, Soldatkina O, Bosio M, Rognon PJ, Capella-Gutierrez S, Calvo M, Reverter F, Guigó R, Aguet F, Ferreira PG, Ardlie KG, Melé M. The landscape of expression and alternative splicing variation across human traits. CELL GENOMICS 2022; 3:100244. [PMID: 36777183 PMCID: PMC9903719 DOI: 10.1016/j.xgen.2022.100244] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 11/08/2022] [Accepted: 12/07/2022] [Indexed: 12/31/2022]
Abstract
Understanding the consequences of individual transcriptome variation is fundamental to deciphering human biology and disease. We implement a statistical framework to quantify the contributions of 21 individual traits as drivers of gene expression and alternative splicing variation across 46 human tissues and 781 individuals from the Genotype-Tissue Expression project. We demonstrate that ancestry, sex, age, and BMI make additive and tissue-specific contributions to expression variability, whereas interactions are rare. Variation in splicing is dominated by ancestry and is under genetic control in most tissues, with ribosomal proteins showing a strong enrichment of tissue-shared splicing events. Our analyses reveal a systemic contribution of types 1 and 2 diabetes to tissue transcriptome variation with the strongest signal in the nerve, where histopathology image analysis identifies novel genes related to diabetic neuropathy. Our multi-tissue and multi-trait approach provides an extensive characterization of the main drivers of human transcriptome variation in health and disease.
Collapse
Affiliation(s)
- Raquel García-Pérez
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Jose Miguel Ramirez
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Aida Ripoll-Cladellas
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Ruben Chazarra-Gil
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Winona Oliveros
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Oleksandra Soldatkina
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Mattia Bosio
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Paul Joris Rognon
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain,Department of Economics and Business, Universitat Pompeu Fabra, Barcelona, Catalonia 08005, Spain,Department of Statistics and Operations Research, Universitat Politècnica de Catalunya, Barcelona, Catalonia 08034, Spain
| | - Salvador Capella-Gutierrez
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain
| | - Miquel Calvo
- Statistics Section, Faculty of Biology, Universitat de Barcelona (UB), Barcelona, Catalonia 08028, Spain
| | - Ferran Reverter
- Statistics Section, Faculty of Biology, Universitat de Barcelona (UB), Barcelona, Catalonia 08028, Spain
| | - Roderic Guigó
- Bioinformatics and Genomics, Center for Genomic Regulation, Barcelona, Catalonia 08003, Spain
| | | | - Pedro G. Ferreira
- Department of Computer Science, Faculty of Sciences, University of Porto, Rua do Campo Alegre, 4169-007 Porto, Portugal,Laboratory of Artificial Intelligence and Decision Support, INESC TEC, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal,Institute of Molecular Pathology and Immunology of the University of Porto, Institute for Research and Innovation in Health (i3s), R. Alfredo Allen 208, 4200-135 Porto, Portugal
| | | | - Marta Melé
- Department of Life Sciences, Barcelona Supercomputing Center (BCN-CNS), Barcelona, Catalonia 08034, Spain,Corresponding author
| |
Collapse
|
14
|
Kim J, Rustam S, Mosquera JM, Randell SH, Shaykhiev R, Rendeiro AF, Elemento O. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat Methods 2022; 19:1653-1661. [PMID: 36316562 PMCID: PMC11102857 DOI: 10.1038/s41592-022-01657-2] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 09/21/2022] [Indexed: 11/05/2022]
Abstract
Multiplexed imaging and spatial transcriptomics enable highly resolved spatial characterization of cellular phenotypes, but still largely depend on laborious manual annotation to understand higher-order patterns of tissue organization. As a result, higher-order patterns of tissue organization are poorly understood and not systematically connected to disease pathology or clinical outcomes. To address this gap, we developed an approach called UTAG to identify and quantify microanatomical tissue structures in multiplexed images without human intervention. Our method combines information on cellular phenotypes with the physical proximity of cells to accurately identify organ-specific microanatomical domains in healthy and diseased tissue. We apply our method to various types of images across healthy and disease states to show that it can consistently detect higher-level architectures in human tissues, quantify structural differences between healthy and diseased tissue, and reveal tissue organization patterns at the organ scale.
Collapse
Affiliation(s)
- Junbum Kim
- Institute for Computational Biomedicine, Department of Physiology and Biophysics, Weill Cornell Medicine, New York, NY, USA
| | - Samir Rustam
- Department of Medicine, Weill Cornell Medicine, New York, NY, USA
| | - Juan Miguel Mosquera
- Department of Pathology and Laboratory Medicine, Caryl and Israel Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, NY, USA
| | - Scott H Randell
- Marsico Lung Institute, The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Renat Shaykhiev
- Department of Medicine, Weill Cornell Medicine, New York, NY, USA
| | - André F Rendeiro
- Institute for Computational Biomedicine, Department of Physiology and Biophysics, Weill Cornell Medicine, New York, NY, USA.
- Caryl and Israel Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, NY, USA.
- CeMM Research Center for Molecular Medicine of the Austrian Academy of Sciences, Vienna, Austria.
| | - Olivier Elemento
- Institute for Computational Biomedicine, Department of Physiology and Biophysics, Weill Cornell Medicine, New York, NY, USA.
- Caryl and Israel Englander Institute for Precision Medicine, Weill Cornell Medicine, New York, NY, USA.
| |
Collapse
|
15
|
Liu TYA, Chen H, Gomez C, Correa ZM, Unberath M. Direct Gene Expression Profile Prediction for Uveal Melanoma from Digital Cytopathology Images via Deep Learning and Salient Image Region Identification. OPHTHALMOLOGY SCIENCE 2022; 3:100240. [PMID: 36561353 PMCID: PMC9764247 DOI: 10.1016/j.xops.2022.100240] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Revised: 10/16/2022] [Accepted: 10/24/2022] [Indexed: 12/14/2022]
Abstract
Objective To demonstrate that deep learning (DL) methods can produce robust prediction of gene expression profile (GEP) in uveal melanoma (UM) based on digital cytopathology images. Design Evaluation of a diagnostic test or technology. Subjects Participants and Controls Deidentified smeared cytology slides stained with hematoxylin and eosin obtained from a fine needle aspirated from UM. Methods Digital whole-slide images were generated by fine-needle aspiration biopsies of UM tumors that underwent GEP testing. A multistage DL system was developed with automatic region-of-interest (ROI) extraction from digital cytopathology images, an attention-based neural network, ROI feature aggregation, and slide-level data augmentation. Main Outcome Measures The ability of our DL system in predicting GEP on a slide (patient) level. Data were partitioned at the patient level (73% training; 27% testing). Results In total, our study included 89 whole-slide images from 82 patients and 121 388 unique ROIs. The testing set included 24 slides from 24 patients (12 class 1 tumors; 12 class 2 tumors; 1 slide per patient). Our DL system for GEP prediction achieved an area under the receiver operating characteristic curve of 0.944, an accuracy of 91.7%, a sensitivity of 91.7%, and a specificity of 91.7% on a slide-level analysis. The incorporation of slide-level feature aggregation and data augmentation produced a more predictive DL model (P = 0.0031). Conclusions Our current work established a complete pipeline for GEP prediction in UM tumors: from automatic ROI extraction from digital cytopathology whole-slide images to slide-level predictions. Our DL system demonstrated robust performance and, if validated prospectively, could serve as an image-based alternative to GEP testing.
Collapse
Key Words
- ANN, artificial neural network
- Artificial intelligence
- CAM, class activation map
- Cytopathology
- DA, data augmentation
- DL, deep learning
- Deep learning
- FA, feature aggregation
- FNAB, fine-needle aspiration biopsy
- GEP, gene expression profile
- Gene expression profile
- ML, machine learning
- ROI, region-of-interest
- UM, uveal melanoma
- Uveal melanoma
Collapse
Affiliation(s)
- T. Y. Alvin Liu
- Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland
| | - Haomin Chen
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, Maryland
| | - Catalina Gomez
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, Maryland
| | - Zelia M. Correa
- Bascom Palmer Eye Institute, University of Miami, Miami, Florida,Correspondence: Zelia M. Correa, MD, PhD, 900 Northwest 17th Street, Floor 1,Miami, FL 33136.
| | - Mathias Unberath
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, Maryland
| |
Collapse
|
16
|
Alsaleh L, Li C, Couetil JL, Ye Z, Huang K, Zhang J, Chen C, Johnson TS. Spatial Transcriptomic Analysis Reveals Associations between Genes and Cellular Topology in Breast and Prostate Cancers. Cancers (Basel) 2022; 14:4856. [PMID: 36230778 PMCID: PMC9562681 DOI: 10.3390/cancers14194856] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Revised: 09/26/2022] [Accepted: 09/28/2022] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Cancer is the leading cause of death worldwide with breast and prostate cancer the most common among women and men, respectively. Gene expression and image features are independently prognostic of patient survival; but until the advent of spatial transcriptomics (ST), it was not possible to determine how gene expression of cells was tied to their spatial relationships (i.e., topology). METHODS We identify topology-associated genes (TAGs) that correlate with 700 image topological features (ITFs) in breast and prostate cancer ST samples. Genes and image topological features are independently clustered and correlated with each other. Themes among genes correlated with ITFs are investigated by functional enrichment analysis. RESULTS Overall, topology-associated genes (TAG) corresponding to extracellular matrix (ECM) and Collagen Type I Trimer gene ontology terms are common to both prostate and breast cancer. In breast cancer specifically, we identify the ZAG-PIP Complex as a TAG. In prostate cancer, we identify distinct TAGs that are enriched for GI dysmotility and the IgA immunoglobulin complex. We identified TAGs in every ST slide regardless of cancer type. CONCLUSIONS These TAGs are enriched for ontology terms, illustrating the biological relevance to our image topology features and their potential utility in diagnostic and prognostic models.
Collapse
Affiliation(s)
- Lujain Alsaleh
- Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, IN 46202, USA
| | - Chen Li
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11794, USA
| | - Justin L. Couetil
- Department of Medical and Molecular Genetics, Indiana University, Indianapolis, IN 46202, USA
| | - Ze Ye
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11794, USA
| | - Kun Huang
- Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, IN 46202, USA
- Department of Medical and Molecular Genetics, Indiana University, Indianapolis, IN 46202, USA
- Regenstrief Institute, Indiana University, Indianapolis, IN 46202, USA
- Melvin and Bren Simon Comprehensive Cancer Center, Indiana University, Indianapolis, IN 46202, USA
| | - Jie Zhang
- Department of Medical and Molecular Genetics, Indiana University, Indianapolis, IN 46202, USA
- Melvin and Bren Simon Comprehensive Cancer Center, Indiana University, Indianapolis, IN 46202, USA
| | - Chao Chen
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY 11794, USA
| | - Travis S. Johnson
- Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, IN 46202, USA
- Melvin and Bren Simon Comprehensive Cancer Center, Indiana University, Indianapolis, IN 46202, USA
- Indiana Biosciences Research Institute, Indianapolis, IN 46202, USA
| |
Collapse
|
17
|
Gewirtz AD, Townes FW, Engelhardt BE. Telescoping bimodal latent Dirichlet allocation to identify expression QTLs across tissues. Life Sci Alliance 2022; 5:e202101297. [PMID: 35977827 PMCID: PMC9387650 DOI: 10.26508/lsa.202101297] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 07/15/2022] [Accepted: 07/18/2022] [Indexed: 11/24/2022] Open
Abstract
Expression quantitative trait loci (eQTLs), or single-nucleotide polymorphisms that affect average gene expression levels, provide important insights into context-specific gene regulation. Classic eQTL analyses use one-to-one association tests, which test gene-variant pairs individually and ignore correlations induced by gene regulatory networks and linkage disequilibrium. Probabilistic topic models, such as latent Dirichlet allocation, estimate latent topics for a collection of count observations. Prior multimodal frameworks that bridge genotype and expression data assume matched sample numbers between modalities. However, many data sets have a nested structure where one individual has several associated gene expression samples and a single germline genotype vector. Here, we build a telescoping bimodal latent Dirichlet allocation (TBLDA) framework to learn shared topics across gene expression and genotype data that allows multiple RNA sequencing samples to correspond to a single individual's genotype. By using raw count data, our model avoids possible adulteration via normalization procedures. Ancestral structure is captured in a genotype-specific latent space, effectively removing it from shared components. Using GTEx v8 expression data across 10 tissues and genotype data, we show that the estimated topics capture meaningful and robust biological signal in both modalities and identify associations within and across tissue types. We identify 4,645 cis-eQTLs and 995 trans-eQTLs by conducting eQTL mapping between the most informative features in each topic. Our TBLDA model is able to identify associations using raw sequencing count data when the samples in two separate data modalities are matched one-to-many, as is often the case in biological data. Our code is freely available at https://github.com/gewirtz/TBLDA.
Collapse
Affiliation(s)
- Ariel Dh Gewirtz
- Lewis-Sigler Institute of Integrative Genomics, Princeton University, Princeton, NJ, USA
| | - F William Townes
- Department of Computer Science, Princeton University, Princeton, NJ, USA
| | - Barbara E Engelhardt
- Department of Computer Science, Princeton University, Princeton, NJ, USA
- Gladstone Institutes, San Francisco, CA, USA
| |
Collapse
|
18
|
Abstract
The human liver is a complex organ made up of multiple specialized cell types that carry out key physiological functions. An incomplete understanding of liver biology limits our ability to develop therapeutics to prevent chronic liver diseases, liver cancers, and death as a result of organ failure. Recently, single-cell modalities have expanded our understanding of the cellular phenotypic heterogeneity and intercellular cross-talk in liver health and disease. This review summarizes these findings and looks forward to highlighting new avenues for the application of single-cell genomics to unravel unknown pathogenic pathways and disease mechanisms for the development of new therapeutics targeting liver pathology. As these technologies mature, their integration into clinical data analysis will aid in patient stratification and in developing treatment plans for patients suffering from liver disease.
Collapse
Affiliation(s)
- Jawairia Atif
- Ajmera Transplant Centre, Schwartz Reisman Liver Research Centre, Toronto General Research Institute, University Health Network, Toronto, Ontario, Canada
- Department of Immunology, University of Toronto, Medical Sciences Building, Toronto, Ontario, Canada
| | - Cornelia Thoeni
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
| | - Gary D. Bader
- Department of Molecular Genetics, University of Toronto, Toronto, Ontario, Canada
- The Donnelly Centre, University of Toronto, Toronto, Ontario, Canada
| | - Ian D. McGilvray
- Ajmera Transplant Centre, Schwartz Reisman Liver Research Centre, Toronto General Research Institute, University Health Network, Toronto, Ontario, Canada
| | - Sonya A. MacParland
- Ajmera Transplant Centre, Schwartz Reisman Liver Research Centre, Toronto General Research Institute, University Health Network, Toronto, Ontario, Canada
- Department of Immunology, University of Toronto, Medical Sciences Building, Toronto, Ontario, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
19
|
Huang H, Zhou G, Liu X, Deng L, Wu C, Zhang D, Liu H. Contrastive learning-based computational histopathology predict differential expression of cancer driver genes. Brief Bioinform 2022; 23:6651307. [PMID: 35901472 DOI: 10.1093/bib/bbac294] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2022] [Revised: 06/16/2022] [Accepted: 06/29/2022] [Indexed: 11/14/2022] Open
Abstract
MOTIVATION Digital pathological analysis is run as the main examination used for cancer diagnosis. Recently, deep learning-driven feature extraction from pathology images is able to detect genetic variations and tumor environment, but few studies focus on differential gene expression in tumor cells. RESULTS In this paper, we propose a self-supervised contrastive learning framework, HistCode, to infer differential gene expression from whole slide images (WSIs). We leveraged contrastive learning on large-scale unannotated WSIs to derive slide-level histopathological features in latent space, and then transfer it to tumor diagnosis and prediction of differentially expressed cancer driver genes. Our experiments showed that our method outperformed other state-of-the-art models in tumor diagnosis tasks, and also effectively predicted differential gene expression. Interestingly, we found the genes with higher fold change can be more precisely predicted. To intuitively illustrate the ability to extract informative features from pathological images, we spatially visualized the WSIs colored by the attention scores of image tiles. We found that the tumor and necrosis areas were highly consistent with the annotations of experienced pathologists. Moreover, the spatial heatmap generated by lymphocyte-specific gene expression patterns was also consistent with the manually labeled WSIs.
Collapse
Affiliation(s)
- Haojie Huang
- School of Computer Science and Engineering, Central South University, 410075, Changsha, China
| | - Gongming Zhou
- School of Computer Science and Engineering, Central South University, 410075, Changsha, China
| | - Xuejun Liu
- School of Computer Science and Technology, Nanjing Tech University, 211816, Nanjing, China
| | - Lei Deng
- School of Computer Science and Engineering, Central South University, 410075, Changsha, China
| | - Chen Wu
- The third affiliated hospital of Soochow University, 213100, Changzhou, China
| | - Dachuan Zhang
- The third affiliated hospital of Soochow University, 213100, Changzhou, China
| | - Hui Liu
- School of Computer Science and Technology, Nanjing Tech University, 211816, Nanjing, China
| |
Collapse
|
20
|
Kirchler M, Konigorski S, Norden M, Meltendorf C, Kloft M, Schurmann C, Lippert C. transferGWAS: GWAS of images using deep transfer learning. Bioinformatics 2022; 38:3621-3628. [PMID: 35640976 DOI: 10.1093/bioinformatics/btac369] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2021] [Revised: 05/05/2022] [Accepted: 05/26/2022] [Indexed: 11/13/2022] Open
Abstract
MOTIVATION Medical images can provide rich information about diseases and their biology. However, investigating their association with genetic variation requires non-standard methods. We propose transferGWAS, a novel approach to perform genome-wide association studies directly on full medical images. First, we learn semantically meaningful representations of the images based on a transfer learning task, during which a deep neural network is trained on independent but similar data. Then, we perform genetic association tests with these representations. RESULTS We validate the type I error rates and power of transferGWAS in simulation studies of synthetic images. Then we apply transferGWAS in a genome-wide association study of retinal fundus images from the UK Biobank. This first-of-a-kind GWAS of full imaging data yielded 60 genomic regions associated with retinal fundus images, of which 7 are novel candidate loci for eye-related traits and diseases. AVAILABILITY AND IMPLEMENTATION Our method is implemented in Python and available at https://github.com/mkirchler/transferGWAS/. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.
Collapse
Affiliation(s)
- Matthias Kirchler
- Digital Health-Machine Learning Research Group, Digital Health Center, Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany.,Department of Computer Science, TU Kaiserslautern, 67663 Kaiserslautern, Germany
| | - Stefan Konigorski
- Digital Health-Machine Learning Research Group, Digital Health Center, Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany.,Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Matthias Norden
- Digital Health & Personalized Medicine Research Group, Digital Health Center, Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany.,Department of Anesthesiology and Intensive Care Medicine, Charité-Universitätsmedizin Berlin, 10117 Berlin, Germany
| | - Christian Meltendorf
- Department of Electrical Engineering - Mechatronics - Optometry, Beuth University of Applied Sciences Berlin, 13353 Berlin, Germany
| | - Marius Kloft
- Department of Computer Science, TU Kaiserslautern, 67663 Kaiserslautern, Germany
| | - Claudia Schurmann
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA.,Digital Health & Personalized Medicine Research Group, Digital Health Center, Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany
| | - Christoph Lippert
- Digital Health-Machine Learning Research Group, Digital Health Center, Hasso Plattner Institute, University of Potsdam, 14482 Potsdam, Germany.,Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| |
Collapse
|
21
|
Murphy M, Jegelka S, Fraenkel E. Self-supervised learning of cell type specificity from immunohistochemical images. Bioinformatics 2022; 38:i395-i403. [PMID: 35758799 PMCID: PMC9235491 DOI: 10.1093/bioinformatics/btac263] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
Motivation Advances in bioimaging now permit in situ proteomic characterization of cell–cell interactions in complex tissues, with important applications across a spectrum of biological problems from development to disease. These methods depend on selection of antibodies targeting proteins that are expressed specifically in particular cell types. Candidate marker proteins are often identified from single-cell transcriptomic data, with variable rates of success, in part due to divergence between expression levels of proteins and the genes that encode them. In principle, marker identification could be improved by using existing databases of immunohistochemistry for thousands of antibodies in human tissue, such as the Human Protein Atlas. However, these data lack detailed annotations of the types of cells in each image. Results We develop a method to predict cell type specificity of protein markers from unlabeled images. We train a convolutional neural network with a self-supervised objective to generate embeddings of the images. Using non-linear dimensionality reduction, we observe that the model clusters images according to cell types and anatomical regions for which the stained proteins are specific. We then use estimates of cell type specificity derived from an independent single-cell transcriptomics dataset to train an image classifier, without requiring any human labelling of images. Our scheme demonstrates superior classification of known proteomic markers in kidney compared to selection via single-cell transcriptomics. Availability and implementation Code and trained model are available at www.github.com/murphy17/HPA-SimCLR. Supplementary information Supplementary data are available at Bioinformatics online.
Collapse
Affiliation(s)
- Michael Murphy
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA.,Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Stefanie Jegelka
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Ernest Fraenkel
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
22
|
Priya S, Burns MB, Ward T, Mars RAT, Adamowicz B, Lock EF, Kashyap PC, Knights D, Blekhman R. Identification of shared and disease-specific host gene-microbiome associations across human diseases using multi-omic integration. Nat Microbiol 2022; 7:780-795. [PMID: 35577971 PMCID: PMC9159953 DOI: 10.1038/s41564-022-01121-z] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2021] [Accepted: 04/06/2022] [Indexed: 12/19/2022]
Abstract
While gut microbiome and host gene regulation independently contribute to gastrointestinal disorders, it is unclear how the two may interact to influence host pathophysiology. Here we developed a machine learning-based framework to jointly analyse paired host transcriptomic (n = 208) and gut microbiome (n = 208) profiles from colonic mucosal samples of patients with colorectal cancer, inflammatory bowel disease and irritable bowel syndrome. We identified associations between gut microbes and host genes that depict shared as well as disease-specific patterns. We found that a common set of host genes and pathways implicated in gastrointestinal inflammation, gut barrier protection and energy metabolism are associated with disease-specific gut microbes. Additionally, we also found that mucosal gut microbes that have been implicated in all three diseases, such as Streptococcus, are associated with different host pathways in each disease, suggesting that similar microbes can affect host pathophysiology in a disease-specific manner through regulation of different host genes. Our framework can be applied to other diseases for the identification of host gene-microbiome associations that may influence disease outcomes.
Collapse
Affiliation(s)
- Sambhawa Priya
- Department of Genetics, Cell Biology and Development, University of Minnesota, Minneapolis, MN, USA
- Bioinformatics and Computational Biology, University of Minnesota, Minneapolis, MN, USA
| | - Michael B Burns
- Department of Biology, Loyola University Chicago, Chicago, IL, USA
| | - Tonya Ward
- BioTechnology Institute, College of Biological Sciences, University of Minnesota, Minneapolis, MN, USA
| | - Ruben A T Mars
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Mayo Clinic, Rochester, MN, USA
| | - Beth Adamowicz
- Department of Genetics, Cell Biology and Development, University of Minnesota, Minneapolis, MN, USA
| | - Eric F Lock
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Purna C Kashyap
- Division of Gastroenterology and Hepatology, Department of Internal Medicine, Mayo Clinic, Rochester, MN, USA
| | - Dan Knights
- BioTechnology Institute, College of Biological Sciences, University of Minnesota, Minneapolis, MN, USA
- Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN, USA
| | - Ran Blekhman
- Department of Genetics, Cell Biology and Development, University of Minnesota, Minneapolis, MN, USA.
- Department of Ecology, Evolution, and Behavior, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
23
|
Cui C, Asad Z, Dean WF, Smith IT, Madden C, Bao S, Landman BA, Roland JT, Coburn LA, Wilson KT, Zwerner JP, Zhao S, Wheless LE, Huo Y. Multi-modal Learning with Missing Data for Cancer Diagnosis Using Histopathological and Genomic Data. PROCEEDINGS OF SPIE--THE INTERNATIONAL SOCIETY FOR OPTICAL ENGINEERING 2022; 12033:120331D. [PMID: 36304178 PMCID: PMC9605813 DOI: 10.1117/12.2612318] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Multi-modal learning (e.g., integrating pathological images with genomic features) tends to improve the accuracy of cancer diagnosis and prognosis as compared to learning with a single modality. However, missing data is a common problem in clinical practice, i.e., not every patient has all modalities available. Most of the previous works directly discarded samples with missing modalities, which might lose information in these data and increase the likelihood of overfitting. In this work, we generalize the multi-modal learning in cancer diagnosis with the capacity of dealing with missing data using histological images and genomic data. Our integrated model can utilize all available data from patients with both complete and partial modalities. The experiments on the public TCGA-GBM and TCGA-LGG datasets show that the data with missing modalities can contribute to multi-modal learning, which improves the model performance in grade classification of glioma cancer.
Collapse
Affiliation(s)
- Can Cui
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Zuhayr Asad
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
| | - William F Dean
- College of Arts and Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Isabelle T Smith
- College of Arts and Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Christopher Madden
- College of Medicine, SUNY Downstate Health Science University, Brooklyn, NY 11203, USA
| | - Shunxing Bao
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| | - Bennett A Landman
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
| | - Joseph T Roland
- Department of Surgery, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Lori A Coburn
- Division of Gastroenterology Hepatology, and Nutrition, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Keith T Wilson
- Division of Gastroenterology Hepatology, and Nutrition, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Jeffrey P Zwerner
- Department of Dermatology, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Shilin Zhao
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Lee E Wheless
- Department of Dermatology, Vanderbilt University Medical Center, Nashville, TN 37215, USA
| | - Yuankai Huo
- Department of Computer Science, Vanderbilt University, Nashville, TN 37235, USA
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37235, USA
| |
Collapse
|
24
|
Carmichael I, Calhoun BC, Hoadley KA, Troester MA, Geradts J, Couture HD, Olsson L, Perou CM, Niethammer M, Hannig J, Marron JS. JOINT AND INDIVIDUAL ANALYSIS OF BREAST CANCER HISTOLOGIC IMAGES AND GENOMIC COVARIATES. Ann Appl Stat 2021; 15:1697-1722. [PMID: 35432688 PMCID: PMC9007558 DOI: 10.1214/20-aoas1433] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/30/2023]
Abstract
The two main approaches in the study of breast cancer are histopathology (analyzing visual characteristics of tumors) and genomics. While both histopathology and genomics are fundamental to cancer research, the connections between these fields have been relatively superficial. We bridge this gap by investigating the Carolina Breast Cancer Study through the development of an integrative, exploratory analysis framework. Our analysis gives insights - some known, some novel - that are engaging to both pathologists and geneticists. Our analysis framework is based on Angle-based Joint and Individual Variation Explained (AJIVE) for statistical data integration and exploits Convolutional Neural Networks (CNNs) as a powerful, automatic method for image feature extraction. CNNs raise interpretability issues that we address by developing novel methods to explore visual modes of variation captured by statistical algorithms (e.g. PCA or AJIVE) applied to CNN features.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | - Jan Hannig
- University of North Carolina at Chapel Hill
| | - J S Marron
- University of North Carolina at Chapel Hill
| |
Collapse
|
25
|
Lu C, Shiradkar R, Liu Z. Integrating pathomics with radiomics and genomics for cancer prognosis: A brief review. Chin J Cancer Res 2021; 33:563-573. [PMID: 34815630 PMCID: PMC8580801 DOI: 10.21147/j.issn.1000-9604.2021.05.03] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 10/22/2021] [Indexed: 11/18/2022] Open
Abstract
In the last decade, the focus of computational pathology research community has shifted from replicating the pathological examination for diagnosis done by pathologists to unlocking and discovering "sub-visual" prognostic image cues from the histopathological image. While we are getting more knowledge and experience in digital pathology, the emerging goal is to integrate other-omics or modalities that will contribute for building a better prognostic assay. In this paper, we provide a brief review of representative works that focus on integrating pathomics with radiomics and genomics for cancer prognosis. It includes: correlation of pathomics and genomics; fusion of pathomics and genomics; fusion of pathomics and radiomics. We also present challenges, potential opportunities, and avenues for future work.
Collapse
Affiliation(s)
- Cheng Lu
- Biomedical Engineering Department, Case Western Reserve University, Cleveland 44106, OH, USA
| | - Rakesh Shiradkar
- Biomedical Engineering Department, Case Western Reserve University, Cleveland 44106, OH, USA
| | - Zaiyi Liu
- Department of Radiology, Guangzhou First People's Hospital, School of Medicine, South China University of Technology, Guangzhou 510080, China
| |
Collapse
|
26
|
Hallou A, Yevick HG, Dumitrascu B, Uhlmann V. Deep learning for bioimage analysis in developmental biology. Development 2021; 148:dev199616. [PMID: 34490888 PMCID: PMC8451066 DOI: 10.1242/dev.199616] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
Deep learning has transformed the way large and complex image datasets can be processed, reshaping what is possible in bioimage analysis. As the complexity and size of bioimage data continues to grow, this new analysis paradigm is becoming increasingly ubiquitous. In this Review, we begin by introducing the concepts needed for beginners to understand deep learning. We then review how deep learning has impacted bioimage analysis and explore the open-source resources available to integrate it into a research project. Finally, we discuss the future of deep learning applied to cell and developmental biology. We analyze how state-of-the-art methodologies have the potential to transform our understanding of biological systems through new image-based analysis and modelling that integrate multimodal inputs in space and time.
Collapse
Affiliation(s)
- Adrien Hallou
- Cavendish Laboratory, Department of Physics, University of Cambridge, Cambridge, CB3 0HE, UK
- Wellcome Trust/Cancer Research UK Gurdon Institute, University of Cambridge, Cambridge, CB2 1QN, UK
- Wellcome Trust/Medical Research Council Stem Cell Institute, University of Cambridge, Cambridge, CB2 1QR, UK
| | - Hannah G. Yevick
- Department of Biology, Massachusetts Institute of Technology, Cambridge, MA, 02142, USA
| | - Bianca Dumitrascu
- Computer Laboratory, Cambridge, University of Cambridge, Cambridge, CB3 0FD, UK
| | - Virginie Uhlmann
- European Bioinformatics Institute, European Molecular Biology Laboratory, Cambridge, CB10 1SD, UK
| |
Collapse
|
27
|
Owens CE, Huffard HG, Nin-Velez AI, Duncan J, Teets CL, Daniels KM, Ealy AD, James RE, Knowlton KF, Cockrum RR. Microbiomes of Various Maternal Body Systems Are Predictive of Calf Digestive Bacterial Ecology. Animals (Basel) 2021; 11:ani11082210. [PMID: 34438668 PMCID: PMC8388428 DOI: 10.3390/ani11082210] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Revised: 07/16/2021] [Accepted: 07/19/2021] [Indexed: 12/29/2022] Open
Abstract
Body systems once thought sterile at birth instead have complex and sometimes abundant microbial ecosystems. However, relationships between dam and calf microbial ecosystems are still unclear. The objectives of this study were to (1) characterize the various maternal and calf microbiomes during peri-partum and post-partum periods and (2) examine the influence of the maternal microbiome on calf fecal microbiome composition during the pre-weaning phase. Multiparous Holstein cows were placed in individual, freshly bedded box stalls 14 d before expected calving. Caudal vaginal fluid samples were collected approximately 24 h before calving and dam fecal, oral, colostrum, and placenta samples were collected immediately after calving. Calf fecal samples were collected at birth (meconium) and 24 h, 7 d, 42 d, and 60 d of age. Amplicons covering V4 16S rDNA regions were generated using DNA extracted from all samples and were sequenced using 300 bp paired end Illumina MiSeq sequencing. Spearman rank correlations were performed between genera in maternal and calf fecal microbiomes. Negative binomial regression models were created for genera in calf fecal samples at each time point using genera in maternal microbiomes. We determined that Bacteroidetes dominated the calf fecal microbiome at all time points (relative abundance ≥42.55%) except for 24 h post-calving, whereas Proteobacteria were the dominant phylum (relative abundance = 85.10%). Maternal fecal, oral, placental, vaginal, and colostrum microbiomes were significant predictors of calf fecal microbiome throughout pre-weaning. Results indicate that calf fecal microbiome inoculation and development may be derived from various maternal sources. Maternal microbiomes could be used to predict calf microbiome development, but further research on the environmental and genetic influences is needed.
Collapse
Affiliation(s)
- Connor E. Owens
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Haley G. Huffard
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Alexandra I. Nin-Velez
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Jane Duncan
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Chrissy L. Teets
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Kristy M. Daniels
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Alan D. Ealy
- Department of Animal and Poultry Sciences, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA;
| | - Robert E. James
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Katharine F. Knowlton
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
| | - Rebecca R. Cockrum
- Department of Dairy Science, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA; (C.E.O.); (H.G.H.); (A.I.N.-V.); (J.D.); (C.L.T.); (K.M.D.); (R.E.J.); (K.F.K.)
- Correspondence: ; Tel.: +1-540-231-1568
| |
Collapse
|
28
|
Zaritsky A, Jamieson AR, Welf ES, Nevarez A, Cillay J, Eskiocak U, Cantarel BL, Danuser G. Interpretable deep learning uncovers cellular properties in label-free live cell images that are predictive of highly metastatic melanoma. Cell Syst 2021; 12:733-747.e6. [PMID: 34077708 PMCID: PMC8353662 DOI: 10.1016/j.cels.2021.05.003] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2020] [Revised: 01/22/2021] [Accepted: 05/07/2021] [Indexed: 12/22/2022]
Abstract
Deep learning has emerged as the technique of choice for identifying hidden patterns in cell imaging data but is often criticized as "black box." Here, we employ a generative neural network in combination with supervised machine learning to classify patient-derived melanoma xenografts as "efficient" or "inefficient" metastatic, validate predictions regarding melanoma cell lines with unknown metastatic efficiency in mouse xenografts, and use the network to generate in silico cell images that amplify the critical predictive cell properties. These exaggerated images unveiled pseudopodial extensions and increased light scattering as hallmark properties of metastatic cells. We validated this interpretation using live cells spontaneously transitioning between states indicative of low and high metastatic efficiency. This study illustrates how the application of artificial intelligence can support the identification of cellular properties that are predictive of complex phenotypes and integrated cell functions but are too subtle to be identified in the raw imagery by a human expert. A record of this paper's transparent peer review process is included in the supplemental information. VIDEO ABSTRACT.
Collapse
Affiliation(s)
- Assaf Zaritsky
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA; Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel.
| | - Andrew R Jamieson
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Erik S Welf
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Andres Nevarez
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA; Section of Molecular Biology, Division of Biological Sciences, University of California San Diego, 9500 Gilman Drive, San Diego, La Jolla, CA 92093, USA
| | - Justin Cillay
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Ugur Eskiocak
- Children's Research Institute and Department of Pediatrics, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Brandi L Cantarel
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA
| | - Gaudenz Danuser
- Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX 75390, USA.
| |
Collapse
|
29
|
Han X, Steven K, Qassim A, Marshall HN, Bean C, Tremeer M, An J, Siggs OM, Gharahkhani P, Craig JE, Hewitt AW, Trzaskowski M, MacGregor S. Automated AI labeling of optic nerve head enables insights into cross-ancestry glaucoma risk and genetic discovery in >280,000 images from UKB and CLSA. Am J Hum Genet 2021; 108:1204-1216. [PMID: 34077762 DOI: 10.1016/j.ajhg.2021.05.005] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Accepted: 05/10/2021] [Indexed: 02/06/2023] Open
Abstract
Cupping of the optic nerve head, a highly heritable trait, is a hallmark of glaucomatous optic neuropathy. Two key parameters are vertical cup-to-disc ratio (VCDR) and vertical disc diameter (VDD). However, manual assessment often suffers from poor accuracy and is time intensive. Here, we show convolutional neural network models can accurately estimate VCDR and VDD for 282,100 images from both UK Biobank and an independent study (Canadian Longitudinal Study on Aging), enabling cross-ancestry epidemiological studies and new genetic discovery for these optic nerve head parameters. Using the AI approach, we perform a systematic comparison of the distribution of VCDR and VDD and compare these with intraocular pressure and glaucoma diagnoses across various genetically determined ancestries, which provides an explanation for the high rates of normal tension glaucoma in East Asia. We then used the large number of AI gradings to conduct a more powerful genome-wide association study (GWAS) of optic nerve head parameters. Using the AI-based gradings increased estimates of heritability by ∼50% for VCDR and VDD. Our GWAS identified more than 200 loci associated with both VCDR and VDD (double the number of loci from previous studies) and uncovered dozens of biological pathways; many of the loci we discovered also confer risk for glaucoma.
Collapse
|