1
|
Stachura P, Lu Z, Kronberg RM, Xu HC, Liu W, Tu JW, Schaal K, Kameri E, Picard D, von Karstedt S, Fischer U, Bhatia S, Lang PA, Borkhardt A, Pandyra AA. Deep transfer learning approach for automated cell death classification reveals novel ferroptosis-inducing agents in subsets of B-ALL. Cell Death Dis 2025; 16:396. [PMID: 40382332 DOI: 10.1038/s41419-025-07704-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Revised: 04/23/2025] [Accepted: 04/29/2025] [Indexed: 05/20/2025]
Abstract
Ferroptosis is a recently described type of regulated necrotic cell death whose induction has anti-cancer therapeutic potential, especially in hematological malignancies. However, efforts to uncover novel ferroptosis-inducing therapeutics have been largely unsuccessful. In the current investigation, we classified brightfield microscopy images of tumor cells undergoing defined modes of cell death using deep transfer learning (DTL). The trained DTL network was subsequently combined with high-throughput pharmacological screening approaches using automated live cell imaging to identify novel ferroptosis-inducing functions of the polo-like kinase inhibitor volasertib. Secondary validation showed that subsets of B-cell acute lymphoblastic leukemia (B-ALL) cell lines, namely 697, NALM6, HAL01, REH and primary patient B-ALL samples were sensitive to ferroptosis induction by volasertib. This was accompanied by an upregulation of ferroptosis-related genes post-volasertib treatment in cell lines and patient samples. Importantly, using several leukemia models, we determined that volasertib delayed tumor growth and induced ferroptosis in vivo. Taken together, we have applied DTL to automated live-cell imaging in pharmacological screening to identify novel ferroptosis-inducing functions of a clinically relevant anti-cancer therapeutic.
Collapse
Affiliation(s)
- Paweł Stachura
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Department of Molecular Medicine II, Medical Faculty, Heinrich-Heine-University, Universitätsstraße 1, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Zhe Lu
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Raphael M Kronberg
- Department of Molecular Medicine II, Medical Faculty, Heinrich-Heine-University, Universitätsstraße 1, 40225, Düsseldorf, Germany
- Mathematical Modelling of Biological Systems, Heinrich Heine University, Düsseldorf, North Rhine-Westphalia, Germany
- Deep-Sea Ecology and Technology, Alfred Wegener Institute Helmholtz Centre for Polar and Marine Research, Bremerhaven, Germany
| | - Haifeng C Xu
- Department of Molecular Medicine II, Medical Faculty, Heinrich-Heine-University, Universitätsstraße 1, 40225, Düsseldorf, Germany
| | - Wei Liu
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- Institute of Clinical Chemistry and Clinical Pharmacology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany
- German Center for Infection Research (DZIF), Partner Site Bonn-Cologne, Bonn, Germany
| | - Jia-Wey Tu
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Katerina Schaal
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Ersen Kameri
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
- Cancer Prevention Graduate School (CPGS), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Daniel Picard
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
- Division of Pediatric Neuro-Oncogenomics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Silvia von Karstedt
- Department of Translational Genomics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Weyertal 115b, Cologne, 50931, Germany
- CECAD Cluster of Excellence, Faculty of Medicine and University Hospital Cologne, University of Cologne, Joseph-Stelzmann-Straße 26, Cologne, 50931, Germany
- Center for Molecular Medicine Cologne, Faculty of Medicine and University Hospital Cologne, University of Cologne, Robert-Koch-Straße 21, Cologne, 50931, Germany
| | - Ute Fischer
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
- Cancer Prevention Graduate School (CPGS), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Sanil Bhatia
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Philipp A Lang
- Department of Molecular Medicine II, Medical Faculty, Heinrich-Heine-University, Universitätsstraße 1, 40225, Düsseldorf, Germany
| | - Arndt Borkhardt
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany
| | - Aleksandra A Pandyra
- Department of Pediatric Oncology, Hematology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, Moorenstrasse 5, 40225, Düsseldorf, Germany.
- Center for Integrated Oncology Aachen Bonn Cologne Düsseldorf (CIO ABCD), Düsseldorf, Germany.
- German Consortium for Translational Cancer Research (DKTK), partner site Essen/Düsseldorf, Düsseldorf, Germany.
- Institute of Clinical Chemistry and Clinical Pharmacology, University Hospital Bonn, Venusberg-Campus 1, 53127, Bonn, Germany.
- German Center for Infection Research (DZIF), Partner Site Bonn-Cologne, Bonn, Germany.
| |
Collapse
|
2
|
Mali AK, Murugappan S, Prasad JR, Tofail SAM, Thorat ND. A deep learning pipeline for morphological and viability assessment of 3D cancer cell spheroids. Biol Methods Protoc 2025; 10:bpaf030. [PMID: 40352793 PMCID: PMC12064216 DOI: 10.1093/biomethods/bpaf030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2025] [Revised: 04/02/2025] [Accepted: 04/09/2025] [Indexed: 05/14/2025] Open
Abstract
Three-dimensional (3D) spheroid models have advanced cancer research by better mimicking the tumour microenvironment compared to traditional two-dimensional cell cultures. However, challenges persist in high-throughput analysis of morphological characteristics and cell viability, as traditional methods like manual fluorescence analysis are labour-intensive and inconsistent. Existing AI-based approaches often address segmentation or classification in isolation, lacking an integrated workflow. We propose a scalable, two-stage deep learning pipeline to address these gaps: (i) a U-Net model for precise detection and segmentation of 3D spheroids from microscopic images, achieving 95% prediction accuracy, and (ii) a CNN Regression Hybrid method for estimating live/dead cell percentages and classifying spheroids, with anR 2 value of 98%. This end-to-end pipeline automates cell viability quantification and generates key morphological parameters for spheroid growth kinetics. By integrating segmentation and analysis, our method addresses environmental variability and morphological characterization challenges, offering a robust tool for drug discovery, toxicity screening, and clinical research. This approach significantly improves efficiency and scalability of 3D spheroid evaluations, paving the way for advancements in cancer therapeutics.
Collapse
Affiliation(s)
- Ajay K Mali
- Department of Physics and Bernal Institute, University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
| | - Sivasubramanian Murugappan
- Department of Physics and Bernal Institute, University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
| | - Jayashree Rajesh Prasad
- Computer Science and Engineering, School of Computing, MIT Art Design and Technology University, Pune, Maharashtra, 412201, India
| | - Syed A M Tofail
- Department of Physics and Bernal Institute, University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
- Limerick Digital Cancer Research Centre (LDCRC), University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
| | - Nanasaheb D Thorat
- Department of Physics and Bernal Institute, University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
- Limerick Digital Cancer Research Centre (LDCRC), University of Limerick, Castletroy, Limerick, V94T9PX, Ireland
| |
Collapse
|
3
|
Woo E, Tasnim F, Kawamata H, Manfredi G, Konrad C. Investigation of mitochondrial phenotypes in motor neurons derived by direct conversion of fibroblasts from familial ALS subjects. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.02.13.637962. [PMID: 40027671 PMCID: PMC11870414 DOI: 10.1101/2025.02.13.637962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease of motor neurons, leading to fatal muscle paralysis. Familial forms of ALS (fALS) account for approximately 10% of cases and are associated with mutations in numerous genes. Alterations of mitochondrial functions have been proposed to contribute to disease pathogenesis. Here, we employed a direct conversion (DC) technique to generate induced motor neurons (iMN) from skin fibroblasts to investigate mitochondrial phenotypes in a patient-derived disease relevant cell culture system. We converted 7 control fibroblast lines and 17 lines harboring the following fALS mutations, SOD1 A4V , TDP-43 N352S , FUS R521G , CHCHD10 R15L , and C9orf72 repeat expansion. We developed new machine learning approaches to identify iMN, analyze their mitochondrial function, and follow their fate longitudinally. Mitochondrial and energetic abnormalities were observed, but not all fALS iMN lines exhibited the same alterations. SOD1 A4V , C9orf72, and TDP-43 N352S iMN had increased mitochondrial membrane potential, while in CHCHD10 R15L cells membrane potential was decreased. TDP-43 N352S iMN displayed changes in mitochondrial morphology and increased motility. SOD1 A4V , TDP-43 N352S , and CHCHD10 R15L iMN had increased oxygen consumption rates and altered extracellular acidification rates, reflecting a hypermetabolic state similar to the one described in sporadic ALS fibroblasts. FUS R521G mutants had decreased ATP/ADP ratio, suggesting impaired energy metabolism. We then tested the viability of iMN and found decreases in survival in SOD1 A4V , C9orf72, and FUS R521G , which were corrected by small molecules that target mitochondrial stress. Together, our findings reinforce the role of mitochondrial dysfunction in ALS and indicate that fibroblast-derived iMN may be useful to study fALS metabolic alterations. Strengths of the DC iMN approach include low cost, speed of transformation, and the preservation of epigenetic modifications. However, further refinement of the fibroblasts DC iMN technique is still needed to improve transformation efficiency, reproducibility, the relatively short lifespan of iMN, and the senescence of the parental fibroblasts.
Collapse
Affiliation(s)
- Evan Woo
- Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY, USA
| | - Faiza Tasnim
- Sophie Davis School of Biomedical Education, CUNY School of Medicine, New York, NY, USA
| | - Hibiki Kawamata
- Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY, USA
| | - Giovanni Manfredi
- Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY, USA
| | - Csaba Konrad
- Brain and Mind Research Institute, Weill Cornell Medicine, New York, NY, USA
| |
Collapse
|
4
|
Shahamatdar S, Saeed-Vafa D, Linsley D, Khalil F, Lovinger K, Li L, McLeod HT, Ramachandran S, Serre T. Deceptive learning in histopathology. Histopathology 2024; 85:116-132. [PMID: 38556922 PMCID: PMC11162337 DOI: 10.1111/his.15180] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 03/08/2024] [Accepted: 03/10/2024] [Indexed: 04/02/2024]
Abstract
AIMS Deep learning holds immense potential for histopathology, automating tasks that are simple for expert pathologists and revealing novel biology for tasks that were previously considered difficult or impossible to solve by eye alone. However, the extent to which the visual strategies learned by deep learning models in histopathological analysis are trustworthy or not has yet to be systematically analysed. Here, we systematically evaluate deep neural networks (DNNs) trained for histopathological analysis in order to understand if their learned strategies are trustworthy or deceptive. METHODS AND RESULTS We trained a variety of DNNs on a novel data set of 221 whole-slide images (WSIs) from lung adenocarcinoma patients, and evaluated their effectiveness at (1) molecular profiling of KRAS versus EGFR mutations, (2) determining the primary tissue of a tumour and (3) tumour detection. While DNNs achieved above-chance performance on molecular profiling, they did so by exploiting correlations between histological subtypes and mutations, and failed to generalise to a challenging test set obtained through laser capture microdissection (LCM). In contrast, DNNs learned robust and trustworthy strategies for determining the primary tissue of a tumour as well as detecting and localising tumours in tissue. CONCLUSIONS Our work demonstrates that DNNs hold immense promise for aiding pathologists in analysing tissue. However, they are also capable of achieving seemingly strong performance by learning deceptive strategies that leverage spurious correlations, and are ultimately unsuitable for research or clinical work. The framework we propose for model evaluation and interpretation is an important step towards developing reliable automated systems for histopathological analysis.
Collapse
Affiliation(s)
- Sahar Shahamatdar
- Center for Computational Molecular Biology, Brown University, Providence, RI, USA
- The Warren Alpert Medical School, Brown University, Providence, RI, USA
| | - Daryoush Saeed-Vafa
- Department of Anatomic Pathology, H. Lee Moffitt Cancer and Research Institute, Tampa, FL, USA
| | - Drew Linsley
- Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Department of Cognitive Linguistic & Psychological Sciences, Brown University, Providence, RI, USA
| | - Farah Khalil
- Department of Anatomic Pathology, H. Lee Moffitt Cancer and Research Institute, Tampa, FL, USA
| | - Katherine Lovinger
- Department of Molecular Biology, H. Lee Moffitt Cancer and Research Institute, Tampa, FL, USA
| | - Lester Li
- University of Rochester, Rochester, NY, USA
| | | | - Sohini Ramachandran
- Center for Computational Molecular Biology, Brown University, Providence, RI, USA
- Department of Ecology, Evolution and Organismal Biology, Brown University, Providence, RI, USA
- The Data Science Initiative, Brown University, Providence, RI, USA
| | - Thomas Serre
- Carney Institute for Brain Science, Brown University, Providence, RI, USA
- Department of Cognitive Linguistic & Psychological Sciences, Brown University, Providence, RI, USA
| |
Collapse
|
5
|
Shkarina K, Broz P. Selective induction of programmed cell death using synthetic biology tools. Semin Cell Dev Biol 2024; 156:74-92. [PMID: 37598045 DOI: 10.1016/j.semcdb.2023.07.012] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Revised: 07/21/2023] [Accepted: 07/21/2023] [Indexed: 08/21/2023]
Abstract
Regulated cell death (RCD) controls the removal of dispensable, infected or malignant cells, and is thus essential for development, homeostasis and immunity of multicellular organisms. Over the last years different forms of RCD have been described (among them apoptosis, necroptosis, pyroptosis and ferroptosis), and the cellular signaling pathways that control their induction and execution have been characterized at the molecular level. It has also become apparent that different forms of RCD differ in their capacity to elicit inflammation or an immune response, and that RCD pathways show a remarkable plasticity. Biochemical and genetic studies revealed that inhibition of a given pathway often results in the activation of back-up cell death mechanisms, highlighting close interconnectivity based on shared signaling components and the assembly of multivalent signaling platforms that can initiate different forms of RCD. Due to this interconnectivity and the pleiotropic effects of 'classical' cell death inducers, it is challenging to study RCD pathways in isolation. This has led to the development of tools based on synthetic biology that allow the targeted induction of RCD using chemogenetic or optogenetic methods. Here we discuss recent advances in the development of such toolset, highlighting their advantages and limitations, and their application for the study of RCD in cells and animals.
Collapse
Affiliation(s)
- Kateryna Shkarina
- Institute of Innate Immunity, University Hospital Bonn, Germany; German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.
| | - Petr Broz
- Department of Immunobiology, University of Lausanne, Switzerland.
| |
Collapse
|
6
|
Tanti GK, Pandey P, Shreya S, Jain BP. Striatin family proteins: The neglected scaffolds. BIOCHIMICA ET BIOPHYSICA ACTA. MOLECULAR CELL RESEARCH 2023; 1870:119430. [PMID: 36638846 DOI: 10.1016/j.bbamcr.2023.119430] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Revised: 12/19/2022] [Accepted: 12/31/2022] [Indexed: 01/12/2023]
Abstract
The Striatin family of proteins constitutes Striatin, SG2NA, and Zinedin. Members of this family of proteins act as a signaling scaffold due to the presence of multiple protein-protein interaction domains. At least two members of this family, namely Zinedin and SG2NA, have a proven role in cancer cell proliferation. SG2NA, the second member of this family, undergoes alternative splicing and gives rise to several isoforms which are differentially regulated in a tissue-dependent manner. SG2NA evolved earlier than the other two members of the family, and SG2NA undergoes not only alternative splicing but also other posttranscriptional gene regulation. Striatin also undergoes alternative splicing, and as a result, it gives rise to multiple isoforms. It has been shown that this family of proteins plays a significant role in estrogen signaling, neuroprotection, cancer as well as in cell cycle regulation. Members of the striatin family form a complex network of signaling hubs with different kinases and phosphatases, and other signaling proteins named STRIPAK. Here, in the present manuscript, we thoroughly reviewed the findings on striatin family members to elaborate on the overall structural and functional idea of this family of proteins. We also commented on the involvement of these proteins in STRIPAK complexes and their functional relevance.
Collapse
Affiliation(s)
- Goutam Kumar Tanti
- Department of Neurology, School of Medicine, Technical University of Munich, Germany.
| | - Prachi Pandey
- National Institute of Plant Genome Research, New Delhi, India
| | - Smriti Shreya
- Department of Zoology, Mahatma Gandhi Central University, Motihari, Bihar, India
| | - Buddhi Prakash Jain
- Department of Zoology, Mahatma Gandhi Central University, Motihari, Bihar, India.
| |
Collapse
|
7
|
Wang S, Linsley JW, Linsley DA, Lamstein J, Finkbeiner S. Fluorescently labeled nuclear morphology is highly informative of neurotoxicity. FRONTIERS IN TOXICOLOGY 2022; 4:935438. [PMID: 36093369 PMCID: PMC9449453 DOI: 10.3389/ftox.2022.935438] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 07/27/2022] [Indexed: 11/16/2022] Open
Abstract
Neurotoxicity can be detected in live microscopy by morphological changes such as retraction of neurites, fragmentation, blebbing of the neuronal soma and ultimately the disappearance of fluorescently labeled neurons. However, quantification of these features is often difficult, low-throughput, and imprecise due to the overreliance on human curation. Recently, we showed that convolutional neural network (CNN) models can outperform human curators in the assessment of neuronal death from images of fluorescently labeled neurons, suggesting that there is information within the images that indicates toxicity but that is not apparent to the human eye. In particular, the CNN's decision strategy indicated that information within the nuclear region was essential for its superhuman performance. Here, we systematically tested this prediction by comparing images of fluorescent neuronal morphology from nuclear-localized fluorescent protein to those from freely diffused fluorescent protein for classifying neuronal death. We found that biomarker-optimized (BO-) CNNs could learn to classify neuronal death from fluorescent protein-localized nuclear morphology (mApple-NLS-CNN) alone, with super-human accuracy. Furthermore, leveraging methods from explainable artificial intelligence, we identified novel features within the nuclear-localized fluorescent protein signal that were indicative of neuronal death. Our findings suggest that the use of a nuclear morphology marker in live imaging combined with computational models such mApple-NLS-CNN can provide an optimal readout of neuronal death, a common result of neurotoxicity.
Collapse
Affiliation(s)
- Shijie Wang
- Center for Systems and Therapeutics, Gladstone Institutes, San Francisco, CA, United States
| | - Jeremy W. Linsley
- Center for Systems and Therapeutics, Gladstone Institutes, San Francisco, CA, United States
| | - Drew A. Linsley
- Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, United States
- Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, RI, United States
| | - Josh Lamstein
- Center for Systems and Therapeutics, Gladstone Institutes, San Francisco, CA, United States
| | - Steven Finkbeiner
- Center for Systems and Therapeutics, Gladstone Institutes, San Francisco, CA, United States
- Taube/Koret Center for Neurodegenerative Disease, Gladstone Institutes, San Francisco, CA, United States
- Departments of Neurology and Physiology, University of California, San Francisco, San Francisco, CA, United States
- Neuroscience Graduate Program, University of California, San Francisco, San Francisco, CA, United States
- Biomedical Sciences Graduate Program, University of California, San Francisco, San Francisco, CA, United States
| |
Collapse
|
8
|
Spagnuolo EJ, Wilf P, Serre T. Decoding family-level features for modern and fossil leaves from computer-vision heat maps. AMERICAN JOURNAL OF BOTANY 2022; 109:768-788. [PMID: 35319778 DOI: 10.1002/ajb2.1842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/18/2021] [Revised: 03/07/2022] [Accepted: 03/08/2022] [Indexed: 06/14/2023]
Abstract
PREMISE Angiosperm leaves present a classic identification problem due to their morphological complexity. Computer-vision algorithms can identify diagnostic regions in images, and heat map outputs illustrate those regions for identification, providing novel insights through visual feedback. We investigate the potential of analyzing leaf heat maps to reveal novel, human-friendly botanical information with applications for extant- and fossil-leaf identification. METHODS We developed a manual scoring system for hotspot locations on published computer-vision heat maps of cleared leaves that showed diagnostic regions for family identification. Heat maps of 3114 cleared leaves of 930 genera in 14 angiosperm families were analyzed. The top-5 and top-1 hotspot regions of highest diagnostic value were scored for 21 leaf locations. The resulting data were viewed using box plots and analyzed using cluster and principal component analyses. We manually identified similar features in fossil leaves to informally demonstrate potential fossil applications. RESULTS The method successfully mapped machine strategy using standard botanical language, and distinctive patterns emerged for each family. Hotspots were concentrated on secondary veins (Salicaceae, Myrtaceae, Anacardiaceae), tooth apices (Betulaceae, Rosaceae), and on the little-studied margins of untoothed leaves (Rubiaceae, Annonaceae, Ericaceae). Similar features drove the results from multivariate analyses. The results echo many traditional observations, while also showing that most diagnostic leaf features remain undescribed. CONCLUSIONS Machine-derived heat maps that initially appear to be dominated by noise can be translated into human-interpretable knowledge, highlighting paths forward for botanists and paleobotanists to discover new diagnostic botanical characters.
Collapse
Affiliation(s)
- Edward J Spagnuolo
- Department of Geosciences and Earth and Environmental Systems Institute, Pennsylvania State University, University Park, Pennsylvania, 16802, USA
- Millennium Scholars Program, Pennsylvania State University, University Park, Pennsylvania, 16802, USA
- Schreyer Honors College, Pennsylvania State University, University Park, Pennsylvania, 16802, USA
| | - Peter Wilf
- Department of Geosciences and Earth and Environmental Systems Institute, Pennsylvania State University, University Park, Pennsylvania, 16802, USA
| | - Thomas Serre
- Department of Cognitive, Linguistic and Psychological Sciences, Carney Institute for Brain Science, Brown University, Providence, Rhode Island, 02912, USA
| |
Collapse
|