1
|
Bengs BD, Nde J, Dutta S, Li Y, Sardiu ME. Integrative approaches for predicting protein network perturbations through machine learning and structural characterization. J Proteomics 2025; 316:105439. [PMID: 40228603 DOI: 10.1016/j.jprot.2025.105439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2025] [Revised: 03/14/2025] [Accepted: 04/08/2025] [Indexed: 04/16/2025]
Abstract
Chromatin remodeling complexes, such as the Saccharomyces cerevisiae INO80 complex, exemplify how dynamic protein interaction networks govern cellular function through a balance of conserved structural modules and context-dependent functional partnerships, as revealed by integrative machine learning and structural mapping approaches. In this study, we explored the INO80 complex using machine learning to predict network changes caused by genetic deletions. Tree-based models outperformed linear approaches, highlighting non-linear relationships within the interaction network. Feature selection identified key INO80 components (e.g., Arp5, Arp8) and cross-compartment features from other remodeling complexes like SWR1 and NuA4, emphasizing shared functional pathways. Perturbation patterns aligned with biological modules, particularly those linked to telomere maintenance and aging, underscoring the functional coherence of these networks. Structural mapping revealed that not all interactions are predictable through proximity alone, particularly with Arp5 and Yta7. By combining structural insights with machine learning, we enhanced predictions of genetic perturbation effects, providing a template for analyzing cross-species homologs (e.g., human INO80) and their disease-associated variants. This integrative approach bridges the gap between static structural data and dynamic functional networks, offering a pathway to disentangle conserved mechanisms from context-dependent adaptations in chromatin biology. SIGNIFICANCE: By leveraging an innovative, integrative machine learning approach, we have successfully predicted and analyzed perturbations in the INO80 network with good accuracy and depth. Our novel combination of machine learning, perturbation analysis, and structural investigation approach has provided crucial insights into the complex's structure-function relationships, shedding new light on its pivotal roles in affected pathways such as telomere maintenance. Our findings not only enhance our understanding of the INO80 complex but also establish a powerful framework for future studies in chromatin biology and beyond. This work represents a step forward in our understanding of chromatin remodeling complexes and their diverse cellular functions, laying the groundwork for future studies that can further refine our computational approaches and experimental techniques in this field.
Collapse
Affiliation(s)
- Bethany D Bengs
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas, USA
| | - Jules Nde
- Department of Cancer Biology, University of Kansas Medical Center, Kansas, USA
| | - Sreejata Dutta
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas, USA
| | - Yanming Li
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas, USA
| | - Mihaela E Sardiu
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas, USA; University of Kansas Cancer Center, Kansas City, USA; Kansas Institute for Precision Medicine, University of Kansas Medical Center, Kansas, USA.
| |
Collapse
|
2
|
Irvine A, Moustafa MM, Patel S, Patel A, Hardardottir L, Delvecchio F, Foreman T, Oak J, Bornheimer SJ, Cimbro R. Automation of flow cytometry data analysis with elastic image registration. Sci Rep 2025; 15:16949. [PMID: 40374736 DOI: 10.1038/s41598-025-99118-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2024] [Accepted: 04/17/2025] [Indexed: 05/18/2025] Open
Abstract
Cell populations in flow cytometry are typically identified via visual manual gating, a time-consuming and error-prone approach to select subpopulations based on expression of cellular markers. Batch processing can be used to automate the analysis of bimodally distributed data but underperforms with highly-variable or continuously-expressed markers. We developed a visual pattern recognition automated gating tool, BD ElastiGate Software (hereafter ElastiGate), to recapitulate the visual process of manual gating by automatically adjusting gates to capture local variability. ElastiGate converts histograms and two-dimensional plots into images, then uses elastic B-spline image registration to transform pre-gated training plot images and their gates to corresponding ungated target plot images, thereby adjusting for local variations. ElastiGate was validated with biologically relevant datasets in CAR-T cell manufacturing, tumor-infiltrating immunophenotyping, cytotoxicity assays (> 500 data files), and a high-dimensional dataset. Accuracy was evaluated against corresponding manually gated analysis using F1 score statistics. ElastiGate performed similarly to manual gating, with average F1 scores of > 0.9 across all gates. ElastiGate, accessible as a FlowJo Software plugin or in BD FACSuite Software, uses minimal training samples to automate gating while substantially reducing analysis time and outperforming existing 2D plot autogating solutions in F1 scores and ease of implementation.
Collapse
Affiliation(s)
| | - Mohamed Mahmoud Moustafa
- Dynamic Omics, Centre for Genomics Research, Discovery Sciences, R&D, AstraZeneca, Gaithersburg, MD, USA
| | - Sahul Patel
- Cell and Gene Therapy Manufacturing, Novartis, Morris Plains, NJ, USA
| | - Aniket Patel
- Cell and Gene Therapy Manufacturing, Novartis, Morris Plains, NJ, USA
| | - Lilja Hardardottir
- Dynamic Omics, Center for Genomics Research, Discovery Sciences, R&D, AstraZeneca, Cambridge, UK
| | | | - Taylor Foreman
- Oncology, Immune Engagers, R&D, AstraZeneca, Gaithersburg, MD, USA
| | - Jean Oak
- Department of Clinical Pathology, Stanford University School of Medicine, Stanford, CA, USA
| | | | - Raffaello Cimbro
- Dynamic Omics, Center for Genomics Research, Discovery Sciences, R&D, AstraZeneca, Cambridge, UK
| |
Collapse
|
3
|
Sadighi Akha AA, Csomós K, Ujházi B, Walter JE, Kumánovics A. Evolving Approach to Clinical Cytometry for Immunodeficiencies and Other Immune Disorders. Immunol Allergy Clin North Am 2025; 45:205-221. [PMID: 40287169 DOI: 10.1016/j.iac.2025.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/29/2025]
Abstract
Primary immunodeficiencies were initially identified on the basis of recurrent, severe or unusual infections. Subsequently, it was noted that these diseases can also manifest with autoimmunity, autoinflammation, allergy, lymphoproliferation and malignancy, hence a conceptual change and their renaming as inborn errors of immunity. Ongoing advances in flow cytometry provide the opportunity to expand or modify the utility and scope of existing laboratory tests in this field to mirror this conceptual change. Here we have used the B cell subset, variably known as CD21low B cells, age-associated B cells and T-bet+ B cells, as an example to demonstrate this possibility.
Collapse
Affiliation(s)
- Amir A Sadighi Akha
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Krisztián Csomós
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Boglárka Ujházi
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Jolán E Walter
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Attila Kumánovics
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
4
|
Tursi AR, Lages CS, Quayle K, Koenig ZT, Loni R, Eswar S, Cobeña-Reyes J, Thornton S, Tilburgs T, Andorf S. CytoPheno: Automated descriptive cell type naming in flow and mass cytometry. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2025:2025.03.11.639902. [PMID: 40161808 PMCID: PMC11952469 DOI: 10.1101/2025.03.11.639902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 04/02/2025]
Abstract
Advances in cytometry have led to increases in the number of cellular markers that are routinely measured. The resulting complexity of the data has prompted a shift from manual to automated analysis methods. Currently, numerous unsupervised methods are available to cluster cells based on marker expression values. However, phenotyping the resulting clusters is typically not part of the automated process. Manually identifying both marker definitions (e.g. CD4+, CCR7+, CD45RA+, CD19-) and descriptive cell type names (e.g. naïve CD4+ T cells) based on marker expression values can be time-consuming, subjective, and error-prone. In this work we propose an algorithm that addresses these problems through the creation of an automated tool, CytoPheno, that assigns marker definitions and cell type names to unidentified clusters. First, post-clustered expression data undergoes per-marker calculations to assign markers as positive or negative. Next, marker names undergo a standardization process to match to Protein Ontology identifier terms. Finally, marker descriptions are matched to cell type names within the Cell Ontology. Each part of the tool was tested with benchmark data to demonstrate performance. Additionally, the tool is encompassed in a graphical user interface (R Shiny) to increase user accessibility and interpretability. Overall, CytoPheno can aid researchers in timely and unbiased phenotyping of post-clustered cytometry data.
Collapse
Affiliation(s)
- Amanda R Tursi
- Department of Biomedical Informatics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
- Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Celine S Lages
- Division of Rheumatology, Research Flow Cytometry Core, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Kenneth Quayle
- Division of Rheumatology, Research Flow Cytometry Core, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Zachary T Koenig
- Division of Immunobiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Rashi Loni
- Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Shruti Eswar
- Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
- Department of Pharmacology, Physiology & Neurobiology, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - José Cobeña-Reyes
- Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Sherry Thornton
- Division of Rheumatology, Research Flow Cytometry Core, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Tamara Tilburgs
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
- Division of Immunobiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| | - Sandra Andorf
- Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
- Division of Allergy and Immunology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
- Division of Biostatistics and Epidemiology, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA
| |
Collapse
|
5
|
Śliwa-Dominiak J, Czechowska K, Blanco A, Sielatycka K, Radaczyńska M, Skonieczna-Żydecka K, Marlicz W, Łoniewski I. Flow Cytometry in Microbiology: A Review of the Current State in Microbiome Research, Probiotics, and Industrial Manufacturing. Cytometry A 2025; 107:145-164. [PMID: 40028773 DOI: 10.1002/cyto.a.24920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2024] [Revised: 12/22/2024] [Accepted: 01/25/2025] [Indexed: 03/05/2025]
Abstract
Flow cytometry (FC) is a versatile and powerful tool in microbiology, enabling precise analysis of single cells for a variety of applications, including the detection and quantification of bacteria, viruses, fungi, as well as algae, phytoplankton, and parasites. Its utility in assessing cell viability, metabolic activity, immune responses, and pathogen-host interactions makes it indispensable in both research and diagnostics. The analysis of microbiota (community of microorganisms) and microbiome (collective genomes of the microorganisms) has become essential for understanding the intricate role of microbial communities in health, disease, and physiological functions. FC offers a promising complement, providing rapid, cost-effective, and dynamic profiling of microbial communities, with the added ability to isolate and sort bacterial populations for further analysis. In the probiotic industry, FC facilitates fast, affordable, and versatile analyses, helping assess both probiotics and postbiotics. It also supports the study of bacterial viability under stress conditions, including gastric acid and bile, improving insight into probiotic survival and adhesion to the intestinal mucosa. Additionally, the integration of Machine Learning in microbiology research has transformative potential, improving data analysis and supporting advances in personalized medicine and probiotic formulations. Despite the need for further standardization, FC continues to evolve as a key tool in modern microbiology and clinical diagnostics.
Collapse
Affiliation(s)
- Joanna Śliwa-Dominiak
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Department of Biochemical Science, Faculty of Health Sciences, Pomeranian Medical University, Szczecin, Poland
| | | | - Alfonso Blanco
- Flow Cytometry Core Technology, UCD Conway Institute of Biomolecular and Biomedical Research, University College Dublin, Dublin 4, Ireland
| | - Katarzyna Sielatycka
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Institute of Biology, Faculty of Exact and Natural Sciences, University of Szczecin, Szczecin, Poland
| | - Martyna Radaczyńska
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Department of Biochemical Science, Faculty of Health Sciences, Pomeranian Medical University, Szczecin, Poland
| | - Karolina Skonieczna-Żydecka
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Department of Biochemical Science, Faculty of Health Sciences, Pomeranian Medical University, Szczecin, Poland
| | - Wojciech Marlicz
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Department of Gastroenterology, Faculty of Medicine, Pomeranian Medical University in Szczecin, Szczecin, Poland
| | - Igor Łoniewski
- Research and Development Centre, Sanprobi, Szczecin, Poland
- Department of Biochemical Science, Faculty of Health Sciences, Pomeranian Medical University, Szczecin, Poland
| |
Collapse
|
6
|
Chen YM, Hsiao TH, Lin CH, Fann YC. Unlocking precision medicine: clinical applications of integrating health records, genetics, and immunology through artificial intelligence. J Biomed Sci 2025; 32:16. [PMID: 39915780 PMCID: PMC11804102 DOI: 10.1186/s12929-024-01110-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2024] [Accepted: 12/02/2024] [Indexed: 02/09/2025] Open
Abstract
Artificial intelligence (AI) has emerged as a transformative force in precision medicine, revolutionizing the integration and analysis of health records, genetics, and immunology data. This comprehensive review explores the clinical applications of AI-driven analytics in unlocking personalized insights for patients with autoimmune rheumatic diseases. Through the synergistic approach of integrating AI across diverse data sets, clinicians gain a holistic view of patient health and potential risks. Machine learning models excel at identifying high-risk patients, predicting disease activity, and optimizing therapeutic strategies based on clinical, genomic, and immunological profiles. Deep learning techniques have significantly advanced variant calling, pathogenicity prediction, splicing analysis, and MHC-peptide binding predictions in genetics. AI-enabled immunology data analysis, including dimensionality reduction, cell population identification, and sample classification, provides unprecedented insights into complex immune responses. The review highlights real-world examples of AI-driven precision medicine platforms and clinical decision support tools in rheumatology. Evaluation of outcomes demonstrates the clinical benefits and impact of these approaches in revolutionizing patient care. However, challenges such as data quality, privacy, and clinician trust must be navigated for successful implementation. The future of precision medicine lies in the continued research, development, and clinical integration of AI-driven strategies to unlock personalized patient care and drive innovation in rheumatology.
Collapse
Affiliation(s)
- Yi-Ming Chen
- Division of Allergy, Immunology and Rheumatology, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung, 40705, Taiwan
- School of Medicine, National Yang Ming Chiao Tung University, Taipei, 11221, Taiwan
- Department of Medical Research, Taichung Veterans General Hospital, Taichung, 40705, Taiwan
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taipei, 112304, Taiwan
- Graduate Institute of Clinical Medicine, College of Medicine, National Chung Hsing University, Taichung, 402202, Taiwan
- Precision Medicine Research Center, College of Medicine, National Chung Hsing University, Taichung, 402202, Taiwan
| | - Tzu-Hung Hsiao
- Department of Medical Research, Taichung Veterans General Hospital, Taichung, 40705, Taiwan
- Department of Public Health, College of Medicine, Fu Jen Catholic University, New Taipei City, 242062, Taiwan
- Institute of Genomics and Bioinformatics, National Chung Hsing University, Taichung, 402202, Taiwan
| | - Ching-Heng Lin
- Department of Medical Research, Taichung Veterans General Hospital, Taichung, 40705, Taiwan.
- Department of Public Health, College of Medicine, Fu Jen Catholic University, New Taipei City, 242062, Taiwan.
- Department of Industrial Engineering and Enterprise Information, Tunghai University, Taichung, 407224, Taiwan.
- Institute of Public Health and Community Medicine Research Center, National Yang Ming Chiao Tung University, Taipei, 11221, Taiwan.
| | - Yang C Fann
- Division of Intramural Research, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, MD, 20892, USA.
| |
Collapse
|
7
|
Bongrand P. Should Artificial Intelligence Play a Durable Role in Biomedical Research and Practice? Int J Mol Sci 2024; 25:13371. [PMID: 39769135 PMCID: PMC11676049 DOI: 10.3390/ijms252413371] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2024] [Revised: 11/26/2024] [Accepted: 12/09/2024] [Indexed: 01/11/2025] Open
Abstract
During the last decade, artificial intelligence (AI) was applied to nearly all domains of human activity, including scientific research. It is thus warranted to ask whether AI thinking should be durably involved in biomedical research. This problem was addressed by examining three complementary questions (i) What are the major barriers currently met by biomedical investigators? It is suggested that during the last 2 decades there was a shift towards a growing need to elucidate complex systems, and that this was not sufficiently fulfilled by previously successful methods such as theoretical modeling or computer simulation (ii) What is the potential of AI to meet the aforementioned need? it is suggested that recent AI methods are well-suited to perform classification and prediction tasks on multivariate systems, and possibly help in data interpretation, provided their efficiency is properly validated. (iii) Recent representative results obtained with machine learning suggest that AI efficiency may be comparable to that displayed by human operators. It is concluded that AI should durably play an important role in biomedical practice. Also, as already suggested in other scientific domains such as physics, combining AI with conventional methods might generate further progress and new applications, involving heuristic and data interpretation.
Collapse
Affiliation(s)
- Pierre Bongrand
- Laboratory Adhesion and Inflammation (LAI), Inserm UMR 1067, Cnrs Umr 7333, Aix-Marseille Université UM 61, 13009 Marseille, France
| |
Collapse
|
8
|
Villegas-Valverde CA, Bencomo-Hernandez AA, Castillo-Aleman YM, Ventura-Carmenate Y, Casado-Hernandez I, Rivero-Jimenez RA. Application of mass cytometry to characterize hematopoietic stem cells in apheresis products of patients with hematological malignancies. Hematol Transfus Cell Ther 2024; 46 Suppl 6:S59-S70. [PMID: 38177056 PMCID: PMC11726105 DOI: 10.1016/j.htct.2023.10.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2023] [Revised: 07/25/2023] [Accepted: 10/20/2023] [Indexed: 01/06/2024] Open
Abstract
INTRODUCTION Hematopoietic stem cell transplantation (HSCT) is a widely used therapy, but its success largely depends on the number and quality of stem cells collected. Current evidence shows the complexity of the hematopoietic system, which implies that, in the quality assurance of the apheresis product, the hematopoietic stem cells are adequately characterized and quantified, in which mass cytometry (MC) can provide its advantages in high-dimensional analysis. OBJECTIVE This research aimed to characterize and enumerate CD45dim/CD34+ stem cells using the MC in apheresis product yields from patients with chronic lymphoid malignant diseases undergoing autologous transplantation at the Abu Dhabi Stem Cells Center. METHODS An analytical and cross-sectional study was performed on 31 apheresis products from 15 patients diagnosed with multiple myeloma (n = 9) and non-Hodgkin lymphomas (n = 6) eligible for HSCT. The MC was employed using the MaxPar Kit for stem cell immunophenotyping. The analysis was performed manually in the Kaluza and unsupervised by machine learning in Cytobank Premium. RESULTS An excellent agreement was found between mass and flow cytometry for the relative and absolute counts of CD45dim/CD34+ cells (Bland-Altman bias: -0.029 and -64, respectively), seven subpopulations were phenotyped and no lineage bias was detected for any of the methods used in the pool of collected cells. A CD34+/CD38+/CD138+ population was seen in the analyses performed on four patients with multiple myeloma. CONCLUSIONS The MC helps to characterize subpopulations of stem cells in apheresis products. It also allows cell quantification by double platform. Unsupervised analysis allows results completion and validation of the manual strategy. The proposed methodology can be extended to apheresis products for purposes other than HSCT.
Collapse
|
9
|
Wlosik J, Granjeaud S, Gorvel L, Olive D, Chretien AS. A beginner's guide to supervised analysis for mass cytometry data in cancer biology. Cytometry A 2024; 105:853-869. [PMID: 39486897 DOI: 10.1002/cyto.a.24901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2024] [Revised: 09/16/2024] [Accepted: 10/01/2024] [Indexed: 11/04/2024]
Abstract
Mass cytometry enables deep profiling of biological samples at single-cell resolution. This technology is more than relevant in cancer research due to high cellular heterogeneity and complexity. Downstream analysis of high-dimensional datasets increasingly relies on machine learning (ML) to extract clinically relevant information, including supervised algorithms for classification and regression purposes. In cancer research, they are used to develop predictive models that will guide clinical decision making. However, the development of supervised algorithms faces major challenges, such as sufficient validation, before being translated into the clinics. In this work, we provide a framework for the analysis of mass cytometry data with a specific focus on supervised algorithms and practical examples of their applications. We also raise awareness on key issues regarding good practices for researchers curious to implement supervised ML on their mass cytometry data. Finally, we discuss the challenges of supervised ML application to cancer research.
Collapse
Affiliation(s)
- Julia Wlosik
- Team 'Immunity and Cancer', Marseille Cancer Research Center, Inserm U1068, CNRS UMR7258, Paoli-Calmettes Institute, Aix-Marseille University UM105, Marseille, France
- Immunomonitoring Department, Paoli-Calmettes Institute, Marseille, France
| | - Samuel Granjeaud
- Systems Biology Platform, Marseille Cancer Research Center, Inserm U1068, CNRS UMR7258, Paoli-Calmettes Institute, Aix-Marseille University UM105, Marseille, France
| | - Laurent Gorvel
- Team 'Immunity and Cancer', Marseille Cancer Research Center, Inserm U1068, CNRS UMR7258, Paoli-Calmettes Institute, Aix-Marseille University UM105, Marseille, France
- Immunomonitoring Department, Paoli-Calmettes Institute, Marseille, France
| | - Daniel Olive
- Team 'Immunity and Cancer', Marseille Cancer Research Center, Inserm U1068, CNRS UMR7258, Paoli-Calmettes Institute, Aix-Marseille University UM105, Marseille, France
- Immunomonitoring Department, Paoli-Calmettes Institute, Marseille, France
| | - Anne-Sophie Chretien
- Team 'Immunity and Cancer', Marseille Cancer Research Center, Inserm U1068, CNRS UMR7258, Paoli-Calmettes Institute, Aix-Marseille University UM105, Marseille, France
- Immunomonitoring Department, Paoli-Calmettes Institute, Marseille, France
| |
Collapse
|
10
|
Irvine A, Tahir S, Tripathi V, Oreizy F, Sen M, Giuliano A, Lin A, Chen A, Lai CH, Omana-Zapata I, Zeng Y, Jain P, Bornheimer SJ. Automated analysis of flow cytometry data with minimal training files: Research evaluation of an elastic image registration algorithm for TBNK, stem cell enumeration, and lymphoid screening tube assays. CYTOMETRY. PART B, CLINICAL CYTOMETRY 2024. [PMID: 39417216 DOI: 10.1002/cyto.b.22210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 09/15/2024] [Accepted: 09/25/2024] [Indexed: 10/19/2024]
Abstract
Automated analysis of flow cytometry data can improve objectivity and reduce analysis time but has generally required work by software and algorithm experts. Here, we investigated the performance of BD ElastiGate™ Software (hereafter ElastiGate), which allows users to automate gating by selecting gated training files, then uses elastic image registration to gate new files. Three assays of increasing complexity were examined: TBNK, stem cell enumeration (SCE), and lymphoid screening tube (LST). For TBNK analysis, 60 peripheral blood (PB) samples from normal, HIV+, and controls were tested with ground truth analysis by an existing automated method. For SCE, 128 samples including bone marrow (BM), cord blood (CB), and apheresis were tested with analysis by multiple manual analysts. For LST, 80 PB and 28 BM samples were tested with manual analysis. For ElastiGate, a minimal number of training files was selected. Results were compared by Bland-Altman or F1 score analysis. For TBNK, ElastiGate using three training files (1 control, 1 normal, 1 HIV+) showed mean %bias across all reported populations between -1.48% and 7.13% (average 2.08%). For SCE, ElastiGate using three BM and two CB training files showed median F1 scores >0.93 in comparison to >0.94 and >0.92 for two other manual analysts. For LST, ElastiGate using four training files for each of PB and BM showed median F1 scores >0.945 for 13 of 14 PB populations and 10 of 14 BM populations, with generally similar or better performance for normal samples compared to abnormal; populations with lower scores were often associated with lower agreement between manual analysts. Based on analysis of three assays with four sample types of increasing complexity, ElastiGate with minimal training files may perform as an automated gating assistant. The results reported here are for research use only, not for use in diagnostic or therapeutic procedures.
Collapse
Affiliation(s)
| | | | | | | | - Moen Sen
- BD Biosciences, San Jose, California, USA
| | | | - Anna Lin
- BD Biosciences, San Jose, California, USA
| | | | | | | | - Yang Zeng
- BD Biosciences, San Jose, California, USA
| | | | | |
Collapse
|
11
|
Eslami M, Moseley RC, Eramian H, Bryce D, Haase SB. AutoGater: a weakly supervised neural network model to gate cells in flow cytometric analyses. Sci Rep 2024; 14:23581. [PMID: 39384769 PMCID: PMC11479614 DOI: 10.1038/s41598-024-66936-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 07/05/2024] [Indexed: 10/11/2024] Open
Abstract
Flow cytometry is a useful and efficient method for the rapid characterization of a cell population based on the optical and fluorescence properties of individual cells. Ideally, the cell population would consist of only healthy viable cells as dead cells can confound the analysis. Thus, separating out healthy cells from dying and dead cells, and any potential debris, is an important first step in analysis of flow cytometry data. While gating of debris can be conducted using measured optical properties, identifying dead and dying cells often requires utilizing fluorescent stains (e.g. Sytox, a nucleic acid stain that stains cells with compromised cell membranes) to identify cells that should be excluded from downstream analyses. These stains prolong the experimental preparation process and use a flow cytometer's fluorescence channels that could otherwise be used to measure additional fluorescent markers within the cells (e.g. reporter proteins). Here we outline a stain-free method for identifying viable cells for downstream processing by gating cells that are dying or dead. AutoGater is a weakly supervised deep learning model that can separate healthy populations from unhealthy and dead populations using only light-scatter channels. In addition, AutoGater harmonizes different measurements of dead cells such as Sytox and CFUs.
Collapse
Affiliation(s)
| | - Robert C Moseley
- Department of Biology, Duke University, Durham, NC, USA
- Cymantix, LLC, Chapel Hill, NC, USA
| | | | - Daniel Bryce
- Smart Information Flow Technologies, LLC, St. Paul, USA
| | - Steven B Haase
- Departments of Biology and Medicine, Duke University, Durham, NC, USA.
| |
Collapse
|
12
|
Zhang W, Sen A, Pena JK, Reitsma A, Alexander OC, Tajima T, Martinez OM, Krams SM. Application of Mass Cytometry Platforms to Solid Organ Transplantation. Transplantation 2024; 108:2034-2044. [PMID: 38467594 PMCID: PMC11390974 DOI: 10.1097/tp.0000000000004925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2024]
Abstract
Transplantation serves as the cornerstone of treatment for patients with end-stage organ disease. The prevalence of complications, such as allograft rejection, infection, and malignancies, underscores the need to dissect the complex interactions of the immune system at the single-cell level. In this review, we discuss studies using mass cytometry or cytometry by time-of-flight, a cutting-edge technology enabling the characterization of immune populations and cell-to-cell interactions in granular detail. We review the application of mass cytometry in human and experimental animal studies in the context of transplantation, uncovering invaluable contributions of the tool to understanding rejection and other transplant-related complications. We discuss recent innovations that have the potential to streamline and standardize mass cytometry workflows for application to multisite clinical trials. Additionally, we introduce imaging mass cytometry, a technique that couples the power of mass cytometry with spatial context, thereby mapping cellular interactions within tissue microenvironments. The synergistic integration of mass cytometry and imaging mass cytometry data with other omics data sets and high-dimensional data platforms to further define immune dynamics is discussed. In conclusion, mass cytometry technologies, when integrated with other tools and data, shed light on the intricate landscape of the immune response in transplantation. This approach holds significant potential for enhancing patient outcomes by advancing our understanding and facilitating the development of new diagnostics and therapeutics.
Collapse
Affiliation(s)
- Wenming Zhang
- Department of Surgery, Stanford University, Stanford, CA, United States
| | - Ayantika Sen
- Department of Surgery, Stanford University, Stanford, CA, United States
| | - Josselyn K. Pena
- Department of Surgery, Stanford University, Stanford, CA, United States
| | - Andrea Reitsma
- Department of Surgery, Stanford University, Stanford, CA, United States
| | - Oliver C. Alexander
- Department of Surgery, Stanford University, Stanford, CA, United States
- Meharry Medical College, School of Medicine, Nashville, TN, United States
| | - Tetsuya Tajima
- Department of Surgery, Stanford University, Stanford, CA, United States
| | | | - Sheri M. Krams
- Department of Surgery, Stanford University, Stanford, CA, United States
| |
Collapse
|
13
|
Wasilewski T, Kamysz W, Gębicki J. AI-Assisted Detection of Biomarkers by Sensors and Biosensors for Early Diagnosis and Monitoring. BIOSENSORS 2024; 14:356. [PMID: 39056632 PMCID: PMC11274923 DOI: 10.3390/bios14070356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 06/25/2024] [Accepted: 06/28/2024] [Indexed: 07/28/2024]
Abstract
The steady progress in consumer electronics, together with improvement in microflow techniques, nanotechnology, and data processing, has led to implementation of cost-effective, user-friendly portable devices, which play the role of not only gadgets but also diagnostic tools. Moreover, numerous smart devices monitor patients' health, and some of them are applied in point-of-care (PoC) tests as a reliable source of evaluation of a patient's condition. Current diagnostic practices are still based on laboratory tests, preceded by the collection of biological samples, which are then tested in clinical conditions by trained personnel with specialistic equipment. In practice, collecting passive/active physiological and behavioral data from patients in real time and feeding them to artificial intelligence (AI) models can significantly improve the decision process regarding diagnosis and treatment procedures via the omission of conventional sampling and diagnostic procedures while also excluding the role of pathologists. A combination of conventional and novel methods of digital and traditional biomarker detection with portable, autonomous, and miniaturized devices can revolutionize medical diagnostics in the coming years. This article focuses on a comparison of traditional clinical practices with modern diagnostic techniques based on AI and machine learning (ML). The presented technologies will bypass laboratories and start being commercialized, which should lead to improvement or substitution of current diagnostic tools. Their application in PoC settings or as a consumer technology accessible to every patient appears to be a real possibility. Research in this field is expected to intensify in the coming years. Technological advancements in sensors and biosensors are anticipated to enable the continuous real-time analysis of various omics fields, fostering early disease detection and intervention strategies. The integration of AI with digital health platforms would enable predictive analysis and personalized healthcare, emphasizing the importance of interdisciplinary collaboration in related scientific fields.
Collapse
Affiliation(s)
- Tomasz Wasilewski
- Department of Inorganic Chemistry, Faculty of Pharmacy, Medical University of Gdansk, Hallera 107, 80-416 Gdansk, Poland
| | - Wojciech Kamysz
- Department of Inorganic Chemistry, Faculty of Pharmacy, Medical University of Gdansk, Hallera 107, 80-416 Gdansk, Poland
| | - Jacek Gębicki
- Department of Process Engineering and Chemical Technology, Faculty of Chemistry, Gdansk University of Technology, Narutowicza 11/12, 80-233 Gdansk, Poland;
| |
Collapse
|
14
|
Ng DP, Simonson PD, Tarnok A, Lucas F, Kern W, Rolf N, Bogdanoski G, Green C, Brinkman RR, Czechowska K. Recommendations for using artificial intelligence in clinical flow cytometry. CYTOMETRY. PART B, CLINICAL CYTOMETRY 2024; 106:228-238. [PMID: 38407537 DOI: 10.1002/cyto.b.22166] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Revised: 01/16/2024] [Accepted: 02/06/2024] [Indexed: 02/27/2024]
Abstract
Flow cytometry is a key clinical tool in the diagnosis of many hematologic malignancies and traditionally requires close inspection of digital data by hematopathologists with expert domain knowledge. Advances in artificial intelligence (AI) are transferable to flow cytometry and have the potential to improve efficiency and prioritization of cases, reduce errors, and highlight fundamental, previously unrecognized associations with underlying biological processes. As a multidisciplinary group of stakeholders, we review a range of critical considerations for appropriately applying AI to clinical flow cytometry, including use case identification, low and high risk use cases, validation, revalidation, computational considerations, and the present regulatory frameworks surrounding AI in clinical medicine. In particular, we provide practical guidance for the development, implementation, and suggestions for potential regulation of AI-based methods in the clinical flow cytometry laboratory. We expect these recommendations to be a helpful initial framework of reference, which will also require additional updates as the field matures.
Collapse
Affiliation(s)
- David P Ng
- Department of Pathology, University of Utah, Salt Lake City, Utah, USA
| | - Paul D Simonson
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York, USA
| | - Attila Tarnok
- Department of Preclinical Development and Validation, Fraunhofer Institute for Cell Therapy and Immunology, IZI, Leipzig, Germany
| | - Fabienne Lucas
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| | - Wolfgang Kern
- MLL Munich Leukemia Laboratory GmbH, Munich, Germany
| | - Nina Rolf
- BC Children's Hospital Research Institute, University of British Columbia, Vancouver, British Columbia, Canada
| | - Goce Bogdanoski
- Clinical Development & Operations Quality, R&D Quality, Bristol Myers Squibb, Princeton, New Jersey, USA
| | - Cherie Green
- Translational Science, Ozette Technologies, Seattle, Washington, USA
| | | | | |
Collapse
|
15
|
Bogdanoski G, Lucas F, Kern W, Czechowska K. Translating the regulatory landscape of medical devices to create fit-for-purpose artificial intelligence (AI) cytometry solutions. CYTOMETRY. PART B, CLINICAL CYTOMETRY 2024; 106:294-307. [PMID: 38396223 DOI: 10.1002/cyto.b.22167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 01/23/2024] [Accepted: 02/08/2024] [Indexed: 02/25/2024]
Abstract
The implementation of medical software and artificial intelligence (AI) algorithms into routine clinical cytometry diagnostic practice requires a thorough understanding of regulatory requirements and challenges throughout the cytometry software product lifecycle. To provide cytometry software developers, computational scientists, researchers, industry professionals, and diagnostic physicians/pathologists with an introduction to European Union (EU) and United States (US) regulatory frameworks. Informed by community feedback and needs assessment established during two international cytometry workshops, this article provides an overview of regulatory landscapes as they pertain to the application of AI, AI-enabled medical devices, and Software as a Medical Device in diagnostic flow cytometry. Evolving regulatory frameworks are discussed, and specific examples regarding cytometry instruments, analysis software and clinical flow cytometry in-vitro diagnostic assays are provided. An important consideration for cytometry software development is the modular approach. As such, modules can be segregated and treated as independent components based on the medical purpose and risk and become subjected to a range of context-dependent compliance and regulatory requirements throughout their life cycle. Knowledge of regulatory and compliance requirements enhances the communication and collaboration between developers, researchers, end-users and regulators. This connection is essential to translate scientific innovation into diagnostic practice and to continue to shape the development and revision of new policies, standards, and approaches.
Collapse
Affiliation(s)
- Goce Bogdanoski
- Clinical Development & Operations Quality, R&D Quality, Bristol Myers Squibb, Princeton, New Jersey, USA
| | - Fabienne Lucas
- Department of Laboratory Medicine and Pathology, University of Washington, Seattle, Washington, USA
| | | | | |
Collapse
|
16
|
Dutta S, Mudaranthakam DP, Li Y, Sardiu ME. PerSEveML: a web-based tool to identify persistent biomarker structure for rare events using an integrative machine learning approach. Mol Omics 2024; 20:348-358. [PMID: 38690925 DOI: 10.1039/d4mo00008k] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/03/2024]
Abstract
Omics data sets often pose a computational challenge due to their high dimensionality, large size, and non-linear structures. Analyzing these data sets becomes especially daunting in the presence of rare events. Machine learning (ML) methods have gained traction for analyzing rare events, yet there has been limited exploration of bioinformatics tools that integrate ML techniques to comprehend the underlying biology. Expanding upon our previously developed computational framework of an integrative machine learning approach, we introduce PerSEveML, an interactive web-based tool that uses crowd-sourced intelligence to predict rare events and determine feature selection structures. PerSEveML provides a comprehensive overview of the integrative approach through evaluation metrics that help users understand the contribution of individual ML methods to the prediction process. Additionally, PerSEveML calculates entropy and rank scores, which visually organize input features into a persistent structure of selected, unselected, and fluctuating categories that help researchers uncover meaningful hypotheses regarding the underlying biology. We have evaluated PerSEveML on three diverse biologically complex data sets with extremely rare events from small to large scale and have demonstrated its ability to generate valid hypotheses. PerSEveML is available at https://biostats-shinyr.kumc.edu/PerSEveML/ and https://github.com/sreejatadutta/PerSEveML.
Collapse
Affiliation(s)
- Sreejata Dutta
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA.
| | - Dinesh Pal Mudaranthakam
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA.
- University of Kansas Cancer Center, Kansas City, USA
| | - Yanming Li
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA.
- University of Kansas Cancer Center, Kansas City, USA
| | - Mihaela E Sardiu
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA.
- University of Kansas Cancer Center, Kansas City, USA
- Kansas Institute for Precision Medicine, University of Kansas Medical Center, Kansas City, Kansas, USA
| |
Collapse
|
17
|
Cheng FM, Lo SC, Lin CC, Lo WJ, Chien SY, Sun TH, Hsu KC. Deep learning assists in acute leukemia detection and cell classification via flow cytometry using the acute leukemia orientation tube. Sci Rep 2024; 14:8350. [PMID: 38594383 PMCID: PMC11004172 DOI: 10.1038/s41598-024-58580-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 04/01/2024] [Indexed: 04/11/2024] Open
Abstract
This study aimed to evaluate the sensitivity of AI in screening acute leukemia and its capability to classify either physiological or pathological cells. Utilizing an acute leukemia orientation tube (ALOT), one of the protocols of Euroflow, flow cytometry efficiently identifies various forms of acute leukemia. However, the analysis of flow cytometry can be time-consuming work. This retrospective study included 241 patients who underwent flow cytometry examination using ALOT between 2017 and 2022. The collected flow cytometry data were used to train an artificial intelligence using deep learning. The trained AI demonstrated a 94.6% sensitivity in detecting acute myeloid leukemia (AML) patients and a 98.2% sensitivity for B-lymphoblastic leukemia (B-ALL) patients. The sensitivities of physiological cells were at least 80%, with variable performance for pathological cells. In conclusion, the AI, trained with ResNet-50 and EverFlow, shows promising results in identifying patients with AML and B-ALL, as well as classifying physiological cells.
Collapse
Affiliation(s)
- Fu-Ming Cheng
- Division of Hematology and Oncology, Department of Internal Medicine, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Shih-Chang Lo
- Artificial Intelligence Center, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Ching-Chan Lin
- Division of Hematology and Oncology, Department of Internal Medicine, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Wen-Jyi Lo
- Division of Hematology and Oncology, Department of Internal Medicine, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Shang-Yu Chien
- Artificial Intelligence Center, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Ting-Hsuan Sun
- Artificial Intelligence Center, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan
| | - Kai-Cheng Hsu
- Artificial Intelligence Center, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan.
- School of Medicine, China Medical University, Taichung, 404, Taiwan.
- Neuroscience and Brain Disease Center, China Medical University, Taichung, 404, Taiwan.
- Department of Neurology, China Medical University Hospital, China Medical University, Taichung, 404, Taiwan.
| |
Collapse
|
18
|
Nollmann C, Moskorz W, Wimmenauer C, Jäger PS, Cadeddu RP, Timm J, Heinzel T, Haas R. Characterization of CD34 + Cells from Patients with Acute Myeloid Leukemia (AML) and Myelodysplastic Syndromes (MDS) Using a t-Distributed Stochastic Neighbor Embedding (t-SNE) Protocol. Cancers (Basel) 2024; 16:1320. [PMID: 38610998 PMCID: PMC11010974 DOI: 10.3390/cancers16071320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 03/26/2024] [Accepted: 03/26/2024] [Indexed: 04/14/2024] Open
Abstract
Using multi-color flow cytometry analysis, we studied the immunophenotypical differences between leukemic cells from patients with AML/MDS and hematopoietic stem and progenitor cells (HSPCs) from patients in complete remission (CR) following their successful treatment. The panel of markers included CD34, CD38, CD45RA, CD123 as representatives for a hierarchical hematopoietic stem and progenitor cell (HSPC) classification as well as programmed death ligand 1 (PD-L1). Rather than restricting the evaluation on a 2- or 3-dimensional analysis, we applied a t-distributed stochastic neighbor embedding (t-SNE) approach to obtain deeper insight and segregation between leukemic cells and normal HPSCs. For that purpose, we created a t-SNE map, which resulted in the visualization of 27 cell clusters based on their similarity concerning the composition and intensity of antigen expression. Two of these clusters were "leukemia-related" containing a great proportion of CD34+/CD38- hematopoietic stem cells (HSCs) or CD34+ cells with a strong co-expression of CD45RA/CD123, respectively. CD34+ cells within the latter cluster were also highly positive for PD-L1 reflecting their immunosuppressive capacity. Beyond this proof of principle study, the inclusion of additional markers will be helpful to refine the differentiation between normal HSPCs and leukemic cells, particularly in the context of minimal disease detection and antigen-targeted therapeutic interventions. Furthermore, we suggest a protocol for the assignment of new cell ensembles in quantitative terms, via a numerical value, the Pearson coefficient, based on a similarity comparison of the t-SNE pattern with a reference.
Collapse
Affiliation(s)
- Cathrin Nollmann
- Condensed Matter Physics Laboratory, Heinrich-Heine-University, 40204 Düsseldorf, Germany; (C.N.)
| | - Wiebke Moskorz
- Institute of Virology, Heinrich-Heine-University, 40204 Düsseldorf, Germany (J.T.)
| | - Christian Wimmenauer
- Condensed Matter Physics Laboratory, Heinrich-Heine-University, 40204 Düsseldorf, Germany; (C.N.)
| | - Paul S. Jäger
- Department of Hematology, Oncology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, 40225 Düsseldorf, Germany; (P.S.J.)
| | - Ron P. Cadeddu
- Department of Hematology, Oncology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, 40225 Düsseldorf, Germany; (P.S.J.)
| | - Jörg Timm
- Institute of Virology, Heinrich-Heine-University, 40204 Düsseldorf, Germany (J.T.)
| | - Thomas Heinzel
- Condensed Matter Physics Laboratory, Heinrich-Heine-University, 40204 Düsseldorf, Germany; (C.N.)
| | - Rainer Haas
- Department of Hematology, Oncology and Clinical Immunology, Medical Faculty, Heinrich-Heine-University, 40225 Düsseldorf, Germany; (P.S.J.)
| |
Collapse
|
19
|
Lewis JE, Cooper LAD, Jaye DL, Pozdnyakova O. Automated Deep Learning-Based Diagnosis and Molecular Characterization of Acute Myeloid Leukemia Using Flow Cytometry. Mod Pathol 2024; 37:100373. [PMID: 37925056 DOI: 10.1016/j.modpat.2023.100373] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 10/23/2023] [Accepted: 10/28/2023] [Indexed: 11/06/2023]
Abstract
The current flow cytometric analysis of blood and bone marrow samples for diagnosis of acute myeloid leukemia (AML) relies heavily on manual intervention in the processing and analysis steps, introducing significant subjectivity into resulting diagnoses and necessitating highly trained personnel. Furthermore, concurrent molecular characterization via cytogenetics and targeted sequencing can take multiple days, delaying patient diagnosis and treatment. Attention-based multi-instance learning models (ABMILMs) are deep learning models that make accurate predictions and generate interpretable insights regarding the classification of a sample from individual events/cells; nonetheless, these models have yet to be applied to flow cytometry data. In this study, we developed a computational pipeline using ABMILMs for the automated diagnosis of AML cases based exclusively on flow cytometric data. Analysis of 1820 flow cytometry samples shows that this pipeline provides accurate diagnoses of acute leukemia (area under the receiver operating characteristic curve [AUROC] 0.961) and accurately differentiates AML vs B- and T-lymphoblastic leukemia (AUROC 0.965). Models for prediction of 9 cytogenetic aberrancies and 32 pathogenic variants in AML provide accurate predictions, particularly for t(15;17)(PML::RARA) [AUROC 0.929], t(8;21)(RUNX1::RUNX1T1) (AUROC 0.814), and NPM1 variants (AUROC 0.807). Finally, we demonstrate how these models generate interpretable insights into which individual flow cytometric events and markers deliver optimal diagnostic utility, providing hematopathologists with a data visualization tool for improved data interpretation, as well as novel biological associations between flow cytometric marker expression and cytogenetic/molecular variants in AML. Our study is the first to illustrate the feasibility of using deep learning-based analysis of flow cytometric data for automated AML diagnosis and molecular characterization.
Collapse
Affiliation(s)
- Joshua E Lewis
- Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts
| | - Lee A D Cooper
- Department of Pathology, Northwestern University, Chicago, Illinois
| | - David L Jaye
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Olga Pozdnyakova
- Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts.
| |
Collapse
|
20
|
Shetab Boushehri S, Essig K, Chlis NK, Herter S, Bacac M, Theis FJ, Glasmacher E, Marr C, Schmich F. Explainable machine learning for profiling the immunological synapse and functional characterization of therapeutic antibodies. Nat Commun 2023; 14:7888. [PMID: 38036503 PMCID: PMC10689847 DOI: 10.1038/s41467-023-43429-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Accepted: 11/09/2023] [Indexed: 12/02/2023] Open
Abstract
Therapeutic antibodies are widely used to treat severe diseases. Most of them alter immune cells and act within the immunological synapse; an essential cell-to-cell interaction to direct the humoral immune response. Although many antibody designs are generated and evaluated, a high-throughput tool for systematic antibody characterization and prediction of function is lacking. Here, we introduce the first comprehensive open-source framework, scifAI (single-cell imaging flow cytometry AI), for preprocessing, feature engineering, and explainable, predictive machine learning on imaging flow cytometry (IFC) data. Additionally, we generate the largest publicly available IFC dataset of the human immunological synapse containing over 2.8 million images. Using scifAI, we analyze class frequency and morphological changes under different immune stimulation. T cell cytokine production across multiple donors and therapeutic antibodies is quantitatively predicted in vitro, linking morphological features with function and demonstrating the potential to significantly impact antibody design. scifAI is universally applicable to IFC data. Given its modular architecture, it is straightforward to incorporate into existing workflows and analysis pipelines, e.g., for rapid antibody screening and functional characterization.
Collapse
Affiliation(s)
- Sayedali Shetab Boushehri
- Institute of AI for Health, Helmholtz Zentrum München, German Research Center for Environmental Health, Neuherberg, Germany
- Institute of Computational Biology, Helmholtz Zentrum München, German Research Center for Environmental Health, Neuherberg, Germany
- Technical University of Munich, Department of Mathematics, Munich, Germany
- Data & Analytics (D&A), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Munich, Germany
| | - Katharina Essig
- Large Molecule Research (LMR), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Munich, Germany
| | - Nikolaos-Kosmas Chlis
- Large Molecule Research (LMR), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Munich, Germany
| | - Sylvia Herter
- Roche Innovation Center Zurich, Roche Pharma Research and Early Development (pRED), Zurich, Switzerland
| | - Marina Bacac
- Roche Innovation Center Zurich, Roche Pharma Research and Early Development (pRED), Zurich, Switzerland
| | - Fabian J Theis
- Institute of Computational Biology, Helmholtz Zentrum München, German Research Center for Environmental Health, Neuherberg, Germany
- Technical University of Munich, Department of Mathematics, Munich, Germany
| | - Elke Glasmacher
- Research and Early Development (RED), Roche Diagnostics Solutions, Roche Innovation Center Munich, Munich, Germany.
| | - Carsten Marr
- Institute of AI for Health, Helmholtz Zentrum München, German Research Center for Environmental Health, Neuherberg, Germany.
- Institute of Computational Biology, Helmholtz Zentrum München, German Research Center for Environmental Health, Neuherberg, Germany.
| | - Fabian Schmich
- Data & Analytics (D&A), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Munich, Germany.
| |
Collapse
|
21
|
Na S, Choo Y, Yoon TH, Paek E. CyGate Provides a Robust Solution for Automatic Gating of Single Cell Cytometry Data. Anal Chem 2023; 95:16918-16926. [PMID: 37946317 PMCID: PMC10666088 DOI: 10.1021/acs.analchem.3c03006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 10/12/2023] [Accepted: 10/23/2023] [Indexed: 11/12/2023]
Abstract
To gain a better understanding of the complex human immune system, it is necessary to measure and interpret numerous cellular protein expressions at the single cell level. Mass cytometry is a relatively new technology that offers unprecedented information about the protein expression of a single cell. Conversely, the analysis of high-dimensional and multiparametric mass cytometric data sets presents a new computational challenge. For instance, conventional "manual gating" analysis was inefficient and unreliable for multiparametric phenotyping of the heterogeneous immune cellular system; consequently, automated methods have been developed to address the high dimensionality of mass cytometry data and enhance the reproducibility of the analysis. Here, we present CyGate, a semiautomated method for classifying single cells into their respective cell types. CyGate learns a gating strategy from a reference data set, trains a model for cell classification, and then automatically analyzes additional data sets using the trained model. CyGate also supports the machine learning framework for the classification of "ungated" cells, which are typically disregarded by automated methods. CyGate's utility was demonstrated by its high performance in cell type classification and the lowest generalization error on various public data sets when compared to the state-of-the-art semiautomated methods. Notably, CyGate had the shortest execution time, allowing it to scale with a growing number of samples. CyGate is available at https://github.com/seungjinna/cygate.
Collapse
Affiliation(s)
- Seungjin Na
- Institute
for Artificial Intelligence Research, Hanyang
University, Seoul 04763, Republic
of Korea
- Department
of Computer Science, Hanyang University, Seoul 04763, Republic of Korea
| | - Yujin Choo
- Department
of Artificial Intelligence, Hanyang University, Seoul 04763, Republic of Korea
| | - Tae Hyun Yoon
- Department
of Chemistry, College of Natural Sciences, Hanyang University, Seoul 04763, Republic
of Korea
- Institute
of Next Generation Material Design, Hanyang
University, Seoul 04763, Republic of Korea
- Yoon
Idea
Lab Co., Ltd., Seoul 04763, Republic of Korea
| | - Eunok Paek
- Institute
for Artificial Intelligence Research, Hanyang
University, Seoul 04763, Republic
of Korea
- Department
of Computer Science, Hanyang University, Seoul 04763, Republic of Korea
- Department
of Artificial Intelligence, Hanyang University, Seoul 04763, Republic of Korea
| |
Collapse
|
22
|
Montante S, Chen Y, Brinkman RR. flowSim: Near duplicate detection for flow cytometry data. Cytometry A 2023; 103:889-901. [PMID: 37530476 PMCID: PMC10834853 DOI: 10.1002/cyto.a.24776] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 06/22/2023] [Accepted: 07/11/2023] [Indexed: 08/03/2023]
Abstract
The analysis of large amounts of data is important for the development of machine learning (ML) models. flowSim is the first algorithm designed to visualize, detect and remove highly redundant information in flow cytometry (FCM) training sets to decrease the computational time for training and increase the performance of ML algorithms by reducing overfitting. flowSim performs near duplicate image detection by combining community detection algorithms with the density analysis of the marker expression values. flowSim clustering compared to consensus manual clustering on a dataset composed of 160 images of bivariate FCM data had a mean Adjusted Rand Index of 0.90, demonstrating its efficiency in identifying similar patterns. flowSim selectively discarded near duplicate files in datasets constructed with known redundancy, and removed 92.6% of FCM images in a dataset of over 500,000 drawn from public repositories.
Collapse
Affiliation(s)
- Sebastiano Montante
- Terry Fox Laboratory, BC Cancer Research, Vancouver, British Columbia, Canada
| | - Yixuan Chen
- Terry Fox Laboratory, BC Cancer Research, Vancouver, British Columbia, Canada
| | - Ryan R. Brinkman
- Terry Fox Laboratory, BC Cancer Research, Vancouver, British Columbia, Canada
- Department of Medical Genetics, University of British Columbia, Vancouver, British Columbia, Canada, 675 West 10th Avenue
| |
Collapse
|
23
|
Dutta S, Box AC, Li Y, Sardiu ME. Identifying dynamical persistent biomarker structures for rare events using modern integrative machine learning approach. Proteomics 2023; 23:e2200290. [PMID: 36852539 PMCID: PMC11503472 DOI: 10.1002/pmic.202200290] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 01/30/2023] [Accepted: 02/17/2023] [Indexed: 03/01/2023]
Abstract
The evolution of omics and computational competency has accelerated discoveries of the underlying biological processes in an unprecedented way. High throughput methodologies, such as flow cytometry, can reveal deeper insights into cell processes, thereby allowing opportunities for scientific discoveries related to health and diseases. However, working with cytometry data often imposes complex computational challenges due to high-dimensionality, large size, and nonlinearity of the data structure. In addition, cytometry data frequently exhibit diverse patterns across biomarkers and suffer from substantial class imbalances which can further complicate the problem. The existing methods of cytometry data analysis either predict cell population or perform feature selection. Through this study, we propose a "wisdom of the crowd" approach to simultaneously predict rare cell populations and perform feature selection by integrating a pool of modern machine learning (ML) algorithms. Given that our approach integrates superior performing ML models across different normalization techniques based on entropy and rank, our method can detect diverse patterns existing across the model features. Furthermore, the method identifies a dynamic biomarker structure that divides the features into persistently selected, unselected, and fluctuating assemblies indicating the role of each biomarker in rare cell prediction, which can subsequently aid in studies of disease progression.
Collapse
Affiliation(s)
- Sreejata Dutta
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
| | - Andrew C. Box
- Stowers Institute for Medical Research, Kansas City, Missouri, USA
| | - Yanming Li
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
- University of Kansas Cancer Center, Kansas City, Kansas, USA
| | - Mihaela E. Sardiu
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
- University of Kansas Cancer Center, Kansas City, Kansas, USA
- Kansas Institute for Precision Medicine, University of Kansas Medical Center, Kansas City, Kansas, USA
| |
Collapse
|
24
|
Dutta S, Mudaranthakam DP, Li Y, Sardiu ME. PerSEveML: A Web-Based Tool to Identify Persistent Biomarker Structure for Rare Events Using Integrative Machine Learning Approach. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.25.564000. [PMID: 38196661 PMCID: PMC10775315 DOI: 10.1101/2023.10.25.564000] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/11/2024]
Abstract
Omics datasets often pose a computational challenge due to their high dimensionality, large size, and non-linear structures. Analyzing these datasets becomes especially daunting in the presence of rare events. Machine learning (ML) methods have gained traction for analyzing rare events, yet there remains a limited exploration of bioinformatics tools that integrate ML techniques to comprehend the underlying biology. Expanding upon our previously developed computational framework of an integrative machine learning approach1, we introduce PerSEveML, an interactive web-based that uses crowd-sourced intelligence to predict rare events and determine feature selection structures. PerSEveML provides a comprehensive overview of the integrative approach through evaluation metrics that help users understand the contribution of individual ML methods to the prediction process. Additionally, PerSEveML calculates entropy and rank scores, which visually organize input features into a persistent structure of selected, unselected, and fluctuating categories that help researchers uncover meaningful hypotheses regarding the underlying biology. We have evaluated PerSEveML on three diverse biologically complex data sets with extremely rare events from small to large scale and have demonstrated its ability to generate valid hypotheses. PerSEveML is available at https://biostats-shinyr.kumc.edu/PerSEveML/ and https://github.com/sreejatadutta/PerSEveML.
Collapse
Affiliation(s)
- Sreejata Dutta
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
| | - Dinesh Pal Mudaranthakam
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
- University of Kansas Cancer Center, Kansas City, USA
| | - Yanming Li
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
- University of Kansas Cancer Center, Kansas City, USA
| | - Mihaela E Sardiu
- Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
- University of Kansas Cancer Center, Kansas City, USA
- Kansas Institute for Precision Medicine, University of Kansas Medical Center, Kansas City, Kansas, USA
| |
Collapse
|
25
|
Robles EE, Jin Y, Smyth P, Scheuermann RH, Bui JD, Wang HY, Oak J, Qian Y. A cell-level discriminative neural network model for diagnosis of blood cancers. Bioinformatics 2023; 39:btad585. [PMID: 37756695 PMCID: PMC10563151 DOI: 10.1093/bioinformatics/btad585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 09/12/2023] [Accepted: 09/22/2023] [Indexed: 09/29/2023] Open
Abstract
MOTIVATION Precise identification of cancer cells in patient samples is essential for accurate diagnosis and clinical monitoring but has been a significant challenge in machine learning approaches for cancer precision medicine. In most scenarios, training data are only available with disease annotation at the subject or sample level. Traditional approaches separate the classification process into multiple steps that are optimized independently. Recent methods either focus on predicting sample-level diagnosis without identifying individual pathologic cells or are less effective for identifying heterogeneous cancer cell phenotypes. RESULTS We developed a generalized end-to-end differentiable model, the Cell Scoring Neural Network (CSNN), which takes sample-level training data and predicts the diagnosis of the testing samples and the identity of the diagnostic cells in the sample, simultaneously. The cell-level density differences between samples are linked to the sample diagnosis, which allows the probabilities of individual cells being diagnostic to be calculated using backpropagation. We applied CSNN to two independent clinical flow cytometry datasets for leukemia diagnosis. In both qualitative and quantitative assessments, CSNN outperformed preexisting neural network modeling approaches for both cancer diagnosis and cell-level classification. Post hoc decision trees and 2D dot plots were generated for interpretation of the identified cancer cells, showing that the identified cell phenotypes match the cancer endotypes observed clinically in patient cohorts. Independent data clustering analysis confirmed the identified cancer cell populations. AVAILABILITY AND IMPLEMENTATION The source code of CSNN and datasets used in the experiments are publicly available on GitHub (http://github.com/erobl/csnn). Raw FCS files can be downloaded from FlowRepository (ID: FR-FCM-Z6YK).
Collapse
Affiliation(s)
- Edgar E Robles
- Department of Computer Science, University of California, Irvine, CA 92697, United States
| | - Ye Jin
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA 02115, United States
| | - Padhraic Smyth
- Department of Computer Science, University of California, Irvine, CA 92697, United States
| | - Richard H Scheuermann
- Department of Informatics, J. Craig Venter Institute, La Jolla, CA 92037, United States
- Department of Pathology, University of California, San Diego, CA 92093, United States
- Center for Infectious Disease and Vaccine Research, La Jolla Institute for Immunology, La Jolla, CA 92037, United States
| | - Jack D Bui
- Department of Pathology, University of California, San Diego, CA 92093, United States
| | - Huan-You Wang
- Department of Pathology, University of California, San Diego, CA 92093, United States
| | - Jean Oak
- Department of Pathology, Stanford University, Stanford, CA 94305, United States
| | - Yu Qian
- Department of Informatics, J. Craig Venter Institute, La Jolla, CA 92037, United States
| |
Collapse
|
26
|
Lewis JE, Cooper LA, Jaye DL, Pozdnyakova O. Automated Deep Learning-Based Diagnosis and Molecular Characterization of Acute Myeloid Leukemia using Flow Cytometry. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.09.18.558289. [PMID: 37808719 PMCID: PMC10557578 DOI: 10.1101/2023.09.18.558289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/10/2023]
Abstract
Current flow cytometric analysis of blood and bone marrow samples for diagnosis of acute myeloid leukemia (AML) relies heavily on manual intervention in both the processing and analysis steps, introducing significant subjectivity into resulting diagnoses and necessitating highly trained personnel. Furthermore, concurrent molecular characterization via cytogenetics and targeted sequencing can take multiple days, delaying patient diagnosis and treatment. Attention-based multi-instance learning models (ABMILMs) are deep learning models which make accurate predictions and generate interpretable insights regarding the classification of a sample from individual events/cells; nonetheless, these models have yet to be applied to flow cytometry data. In this study, we developed a computational pipeline using ABMILMs for the automated diagnosis of AML cases based exclusively on flow cytometric data. Analysis of 1,820 flow cytometry samples shows that this pipeline provides accurate diagnoses of acute leukemia [AUROC 0.961] and accurately differentiates AML versus B- and T-lymphoblastic leukemia [AUROC 0.965]. Models for prediction of 9 cytogenetic aberrancies and 32 pathogenic variants in AML provide accurate predictions, particularly for t(15;17)(PML::RARA) [AUROC 0.929], t(8;21)(RUNX1::RUNX1T1) [AUROC 0.814], and NPM1 variants [AUROC 0.807]. Finally, we demonstrate how these models generate interpretable insights into which individual flow cytometric events and markers deliver optimal diagnostic utility, providing hematopathologists with a data visualization tool for improved data interpretation, as well as novel biological associations between flow cytometric marker expression and cytogenetic/molecular variants in AML. Our study is the first to illustrate the feasibility of using deep learning-based analysis of flow cytometric data for automated AML diagnosis and molecular characterization.
Collapse
Affiliation(s)
- Joshua E. Lewis
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA, USA
| | - Lee A.D. Cooper
- Department of Pathology, Northwestern University, Chicago, IL, USA
| | - David L. Jaye
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
| | - Olga Pozdnyakova
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA, USA
| |
Collapse
|
27
|
Saihi H, Bessant C, Alazawi W. Automated and reproducible cell identification in mass cytometry using neural networks. Brief Bioinform 2023; 24:bbad392. [PMID: 37930029 PMCID: PMC10630086 DOI: 10.1093/bib/bbad392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 10/04/2023] [Accepted: 10/08/2023] [Indexed: 11/07/2023] Open
Abstract
The principal use of mass cytometry is to identify distinct cell types and changes in their composition, phenotype and function in different samples and conditions. Combining data from different studies has the potential to increase the power of these discoveries in diverse fields such as immunology, oncology and infection. However, current tools are lacking in scalable, reproducible and automated methods to integrate and study data sets from mass cytometry that often use heterogenous approaches to study similar samples. To address these limitations, we present two novel developments: (1) a pre-trained cell identification model named Immunopred that allows automated identification of immune cells without user-defined prior knowledge of expected cell types and (2) a fully automated cytometry meta-analysis pipeline built around Immunopred. We evaluated this pipeline on six COVID-19 study data sets comprising 270 unique samples and uncovered novel significant phenotypic changes in the wider immune landscape of COVID-19 that were not identified when each study was analyzed individually. Applied widely, our approach will support the discovery of novel findings in research areas where cytometry data sets are available for integration.
Collapse
Affiliation(s)
- Hajar Saihi
- Centre for Immunobiology, Blizard Institute, School of Medicine and Dentistry, Barts and the London, UK
| | - Conrad Bessant
- Digital Environment Research Institute, Queen Mary University of London, London, UK
- School of Biological and Behavioural Sciences, Queen Mary University of London, London, UK
- Alan Turing Institute, British Library, 96 Euston Rd., London NW1 2DB
| | - William Alazawi
- Centre for Immunobiology, Blizard Institute, School of Medicine and Dentistry, Barts and the London, UK
| |
Collapse
|
28
|
Sadighi Akha AA, Csomós K, Ujházi B, Walter JE, Kumánovics A. Evolving Approach to Clinical Cytometry for Immunodeficiencies and Other Immune Disorders. Clin Lab Med 2023; 43:467-483. [PMID: 37481324 DOI: 10.1016/j.cll.2023.05.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/24/2023]
Abstract
Primary immunodeficiencies were initially identified on the basis of recurrent, severe or unusual infections. Subsequently, it was noted that these diseases can also manifest with autoimmunity, autoinflammation, allergy, lymphoproliferation and malignancy, hence a conceptual change and their renaming as inborn errors of immunity. Ongoing advances in flow cytometry provide the opportunity to expand or modify the utility and scope of existing laboratory tests in this field to mirror this conceptual change. Here we have used the B cell subset, variably known as CD21low B cells, age-associated B cells and T-bet+ B cells, as an example to demonstrate this possibility.
Collapse
Affiliation(s)
- Amir A Sadighi Akha
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA
| | - Krisztián Csomós
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Boglárka Ujházi
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Jolán E Walter
- Division of Pediatric Allergy/Immunology, University of South Florida, Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA
| | - Attila Kumánovics
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
29
|
Räuber S, Nelke C, Schroeter CB, Barman S, Pawlitzki M, Ingwersen J, Akgün K, Günther R, Garza AP, Marggraf M, Dunay IR, Schreiber S, Vielhaber S, Ziemssen T, Melzer N, Ruck T, Meuth SG, Herty M. Classifying flow cytometry data using Bayesian analysis helps to distinguish ALS patients from healthy controls. Front Immunol 2023; 14:1198860. [PMID: 37600819 PMCID: PMC10434536 DOI: 10.3389/fimmu.2023.1198860] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2023] [Accepted: 07/13/2023] [Indexed: 08/22/2023] Open
Abstract
Introduction Given its wide availability and cost-effectiveness, multidimensional flow cytometry (mFC) became a core method in the field of immunology allowing for the analysis of a broad range of individual cells providing insights into cell subset composition, cellular behavior, and cell-to-cell interactions. Formerly, the analysis of mFC data solely relied on manual gating strategies. With the advent of novel computational approaches, (semi-)automated gating strategies and analysis tools complemented manual approaches. Methods Using Bayesian network analysis, we developed a mathematical model for the dependencies of different obtained mFC markers. The algorithm creates a Bayesian network that is a HC tree when including raw, ungated mFC data of a randomly selected healthy control cohort (HC). The HC tree is used to classify whether the observed marker distribution (either patients with amyotrophic lateral sclerosis (ALS) or HC) is predicted. The relative number of cells where the probability q is equal to zero is calculated reflecting the similarity in the marker distribution between a randomly chosen mFC file (ALS or HC) and the HC tree. Results Including peripheral blood mFC data from 68 ALS and 35 HC, the algorithm could correctly identify 64/68 ALS cases. Tuning of parameters revealed that the combination of 7 markers, 200 bins, and 20 patients achieved the highest AUC on a significance level of p < 0.0001. The markers CD4 and CD38 showed the highest zero probability. We successfully validated our approach by including a second, independent ALS and HC cohort (55 ALS and 30 HC). In this case, all ALS were correctly identified and side scatter and CD20 yielded the highest zero probability. Finally, both datasets were analyzed by the commercially available algorithm 'Citrus', which indicated superior ability of Bayesian network analysis when including raw, ungated mFC data. Discussion Bayesian network analysis might present a novel approach for classifying mFC data, which does not rely on reduction techniques, thus, allowing to retain information on the entire dataset. Future studies will have to assess the performance when discriminating clinically relevant differential diagnoses to evaluate the complementary diagnostic benefit of Bayesian network analysis to the clinical routine workup.
Collapse
Affiliation(s)
- Saskia Räuber
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Christopher Nelke
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Christina B. Schroeter
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Sumanta Barman
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Marc Pawlitzki
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Jens Ingwersen
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Katja Akgün
- Department of Neurology, Center of Clinical Neuroscience, University Hospital Carl Gustav Carus, Dresden University of Technology, Dresden, Germany
| | - Rene Günther
- Department of Neurology, Center of Clinical Neuroscience, University Hospital Carl Gustav Carus, Dresden University of Technology, Dresden, Germany
| | - Alejandra P. Garza
- Institute of Inflammation and Neurodegeneration, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| | - Michaela Marggraf
- Department of Neurology, Center of Clinical Neuroscience, University Hospital Carl Gustav Carus, Dresden University of Technology, Dresden, Germany
| | - Ildiko Rita Dunay
- Institute of Inflammation and Neurodegeneration, Otto-von-Guericke University Magdeburg, Magdeburg, Germany
| | - Stefanie Schreiber
- Department of Neurology, Otto von Guericke University, Magdeburg, Germany
| | - Stefan Vielhaber
- Department of Neurology, Otto von Guericke University, Magdeburg, Germany
| | - Tjalf Ziemssen
- Department of Neurology, Center of Clinical Neuroscience, University Hospital Carl Gustav Carus, Dresden University of Technology, Dresden, Germany
| | - Nico Melzer
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Tobias Ruck
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Sven G. Meuth
- Department of Neurology, Medical Faculty, Heinrich Heine University of Düsseldorf, Düsseldorf, Germany
| | - Michael Herty
- Department of Mathematics, Institute of Geometry and Applied Mathematics, RWTH Aachen University, Aachen, Germany
| |
Collapse
|
30
|
Robinson JP, Ostafe R, Iyengar SN, Rajwa B, Fischer R. Flow Cytometry: The Next Revolution. Cells 2023; 12:1875. [PMID: 37508539 PMCID: PMC10378642 DOI: 10.3390/cells12141875] [Citation(s) in RCA: 52] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 07/06/2023] [Accepted: 07/13/2023] [Indexed: 07/30/2023] Open
Abstract
Unmasking the subtleties of the immune system requires both a comprehensive knowledge base and the ability to interrogate that system with intimate sensitivity. That task, to a considerable extent, has been handled by an iterative expansion in flow cytometry methods, both in technological capability and also in accompanying advances in informatics. As the field of fluorescence-based cytomics matured, it reached a technological barrier at around 30 parameter analyses, which stalled the field until spectral flow cytometry created a fundamental transformation that will likely lead to the potential of 100 simultaneous parameter analyses within a few years. The simultaneous advance in informatics has now become a watershed moment for the field as it competes with mature systematic approaches such as genomics and proteomics, allowing cytomics to take a seat at the multi-omics table. In addition, recent technological advances try to combine the speed of flow systems with other detection methods, in addition to fluorescence alone, which will make flow-based instruments even more indispensable in any biological laboratory. This paper outlines current approaches in cell analysis and detection methods, discusses traditional and microfluidic sorting approaches as well as next-generation instruments, and provides an early look at future opportunities that are likely to arise.
Collapse
Affiliation(s)
- J Paul Robinson
- Department of Basic Medical Sciences, Purdue University, West Lafayette, IN 47907, USA
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN 47907, USA
| | - Raluca Ostafe
- Molecular Evolution, Protein Engineering and Production Facility (PI4D), Purdue University, West Lafayette, IN 47907, USA
| | | | - Bartek Rajwa
- Bindley Bioscience Center, Purdue University, West Lafayette, IN 47907, USA
| | - Rainer Fischer
- Department of Comparative Pathobiology, College of Veterinary Medicine, Purdue University, West Lafayette, IN 47907, USA
- Purdue Institute of Inflammation, Immunology and Infectious Diseases, Purdue University, West Lafayette, IN 47907, USA
| |
Collapse
|
31
|
Keyes TJ, Koladiya A, Lo YC, Nolan GP, Davis KL. tidytof: a user-friendly framework for scalable and reproducible high-dimensional cytometry data analysis. BIOINFORMATICS ADVANCES 2023; 3:vbad071. [PMID: 37351311 PMCID: PMC10281957 DOI: 10.1093/bioadv/vbad071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 05/03/2023] [Accepted: 06/07/2023] [Indexed: 06/24/2023]
Abstract
Summary While many algorithms for analyzing high-dimensional cytometry data have now been developed, the software implementations of these algorithms remain highly customized-this means that exploring a dataset requires users to learn unique, often poorly interoperable package syntaxes for each step of data processing. To solve this problem, we developed {tidytof}, an open-source R package for analyzing high-dimensional cytometry data using the increasingly popular 'tidy data' interface. Availability and implementation {tidytof} is available at https://github.com/keyes-timothy/tidytof and is released under the MIT license. It is supported on Linux, MS Windows and MacOS. Additional documentation is available at the package website (https://keyes-timothy.github.io/tidytof/). Supplementary information Supplementary data are available at Bioinformatics Advances online.
Collapse
Affiliation(s)
- Timothy J Keyes
- Medical Scientist Training Program, Stanford University School of Medicine, Stanford, CA 94305, USA
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA 94305, USA
- Department of Pediatrics, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Abhishek Koladiya
- Department of Pediatrics, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Yu-Chen Lo
- Department of Pediatrics, Stanford University School of Medicine, Stanford, CA 94305, USA
| | - Garry P Nolan
- Department of Pathology, Stanford University School of Medicine, Stanford, CA 94305, USA
| | | |
Collapse
|
32
|
Yang L, Dutta P, Davuluri RV, Wang J. Rapid, High-Throughput Single-Cell Multiplex In Situ Tagging (MIST) Analysis of Immunological Disease with Machine Learning. Anal Chem 2023; 95:7779-7787. [PMID: 37141575 PMCID: PMC10365012 DOI: 10.1021/acs.analchem.3c01157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
The cascade of immune responses involves activation of diverse immune cells and release of a large amount of cytokines, which leads to either normal, balanced inflammation or hyperinflammatory responses and even organ damage by sepsis. Conventional diagnosis of immunological disorders based on multiple cytokines in the blood serum has varied accuracy, and it is difficult to distinguish normal inflammation from sepsis. Herein, we present an approach to detect immunological disorders through rapid, ultrahigh-multiplex analysis of T cells using single-cell multiplex in situ tagging (scMIST) technology. scMIST permits simultaneous detection of 46 markers and cytokines from single cells without the assistance of special instruments. A cecal ligation and puncture sepsis model was built to supply T cells from two groups of mice that survived the surgery or died after 1 day. The scMIST assays have captured the T cell features and the dynamics over the course of recovery. Compared with cytokines in the peripheral blood, T cell markers show different dynamics and cytokine levels. We have applied a random forest machine learning model to single T cells from two groups of mice. Through training, the model has been able to predict the group of mice through T cell classification and majority rule with 94% accuracy. Our approach pioneers the direction of single-cell omics and could be widely applicable to human diseases.
Collapse
Affiliation(s)
- Liwei Yang
- Multiplex Biotechnology Laboratory, Department of Biomedical Engineering, State University of New York at Stony Brook, Stony Brook, NY 11794
| | - Pratik Dutta
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY 11794
| | - Ramana V. Davuluri
- Department of Biomedical Informatics, State University of New York at Stony Brook, Stony Brook, NY 11794
| | - Jun Wang
- Multiplex Biotechnology Laboratory, Department of Biomedical Engineering, State University of New York at Stony Brook, Stony Brook, NY 11794
| |
Collapse
|
33
|
Hourani T, Perez-Gonzalez A, Khoshmanesh K, Luwor R, Achuthan AA, Baratchi S, O'Brien-Simpson NM, Al-Hourani A. Label-free macrophage phenotype classification using machine learning methods. Sci Rep 2023; 13:5202. [PMID: 36997576 PMCID: PMC10061362 DOI: 10.1038/s41598-023-32158-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2022] [Accepted: 03/23/2023] [Indexed: 04/01/2023] Open
Abstract
Macrophages are heterogeneous innate immune cells that are functionally shaped by their surrounding microenvironment. Diverse macrophage populations have multifaceted differences related to their morphology, metabolism, expressed markers, and functions, where the identification of the different phenotypes is of an utmost importance in modelling immune response. While expressed markers are the most used signature to classify phenotypes, multiple reports indicate that macrophage morphology and autofluorescence are also valuable clues that can be used in the identification process. In this work, we investigated macrophage autofluorescence as a distinct feature for classifying six different macrophage phenotypes, namely: M0, M1, M2a, M2b, M2c, and M2d. The identification was based on extracted signals from multi-channel/multi-wavelength flow cytometer. To achieve the identification, we constructed a dataset containing 152,438 cell events each having a response vector of 45 optical signals fingerprint. Based on this dataset, we applied different supervised machine learning methods to detect phenotype specific fingerprint from the response vector, where the fully connected neural network architecture provided the highest classification accuracy of 75.8% for the six phenotypes compared simultaneously. Furthermore, by restricting the number of phenotypes in the experiment, the proposed framework produces higher classification accuracies, averaging 92.0%, 91.9%, 84.2%, and 80.4% for a pool of two, three, four, five phenotypes, respectively. These results indicate the potential of the intrinsic autofluorescence for classifying macrophage phenotypes, with the proposed method being quick, simple, and cost-effective way to accelerate the discovery of macrophage phenotypical diversity.
Collapse
Affiliation(s)
- Tetiana Hourani
- Department of Medicine, Royal Melbourne Hospital, The University of Melbourne, Parkville, VIC, 3050, Australia
| | - Alexis Perez-Gonzalez
- Melbourne Cytometry Platform, Department of Microbiology and Immunology, The University of Melbourne, at The Peter Doherty Institute of Infection and Immunity, Parkville, VIC, 3010, Australia
| | | | - Rodney Luwor
- Department of Surgery, Royal Melbourne Hospital, The University of Melbourne, Parkville, Victoria, 3050, Australia
- Fiona Elsey Cancer Research Institute, Ballarat, Victoria, 3350, Australia
- Federation University Australia, Ballarat, Victoria, 3350, Australia
| | - Adrian A Achuthan
- Department of Medicine, Royal Melbourne Hospital, The University of Melbourne, Parkville, VIC, 3050, Australia
| | - Sara Baratchi
- School of Health & Biomedical Sciences, RMIT University, Bundoora, Victoria, 3083, Australia
| | - Neil M O'Brien-Simpson
- ACTV Research Group, Division of Basic and Clinical Oral Sciences, Centre for Oral Health Research, Melbourne Dental School, Royal Dental Hospital, The University of Melbourne, 720 Swanston Street, Carlton, VIC, 3010, Australia
| | - Akram Al-Hourani
- School of Engineering, RMIT University, Melbourne, Victoria, 3000, Australia.
| |
Collapse
|
34
|
Robles EE, Jin Y, Smyth P, Scheuermann RH, Bui JD, Wang HY, Oak J, Qian Y. A cell-level discriminative neural network model for diagnosis of blood cancers. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.07.23285606. [PMID: 36798344 PMCID: PMC9934808 DOI: 10.1101/2023.02.07.23285606] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/12/2023]
Abstract
Motivation Precise identification of cancer cells in patient samples is essential for accurate diagnosis and clinical monitoring but has been a significant challenge in machine learning approaches for cancer precision medicine. In most scenarios, training data are only available with disease annotation at the subject or sample level. Traditional approaches separate the classification process into multiple steps that are optimized independently. Recent methods either focus on predicting sample-level diagnosis without identifying individual pathologic cells or are less effective for identifying heterogeneous cancer cell phenotypes. Results We developed a generalized end-to-end differentiable model, the Cell Scoring Neural Network (CSNN), which takes the available sample-level training data and predicts both the diagnosis of the testing samples and the identity of the diagnostic cells in the sample, simultaneously. The cell-level density differences between samples are linked to the sample diagnosis, which allows the probabilities of individual cells being diagnostic to be calculated using backpropagation. We applied CSNN to two independent clinical flow cytometry datasets for leukemia diagnosis. In both qualitative and quantitative assessments, CSNN outperformed preexisting neural network modeling approaches for both cancer diagnosis and cell-level classification. Post hoc decision trees and 2D dot plots were generated for interpretation of the identified cancer cells, showing that the identified cell phenotypes match the cancer endotypes observed clinically in patient cohorts. Independent data clustering analysis confirmed the identified cancer cell populations. Availability The source code of CSNN and datasets used in the experiments are publicly available on GitHub and FlowRepository. Contact Edgar E. Robles: roblesee@uci.edu and Yu Qian: mqian@jcvi.org. Supplementary information Supplementary data are available on GitHub and at Bioinformatics online.
Collapse
|
35
|
Obstfeld AE. Hematology and Machine Learning. J Appl Lab Med 2023; 8:129-144. [PMID: 36610431 DOI: 10.1093/jalm/jfac108] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 10/18/2022] [Indexed: 01/09/2023]
Abstract
BACKGROUND Substantial improvements in computational power and machine learning (ML) algorithm development have vastly increased the limits of what autonomous machines are capable of. Since its beginnings in the 19th century, laboratory hematology has absorbed waves of progress yielding improvements in both of accuracy and efficiency. The next wave of change in laboratory hematology will be the result of the ML revolution that has already touched many corners of healthcare and society at large. CONTENT This review will describe the manifestations of ML and artificial intelligence (AI) already utilized in the clinical hematology laboratory. This will be followed by a topical summary of the innovative and investigational applications of this technology in each of the major subdomains within laboratory hematology. SUMMARY Application of this technology to laboratory hematology will increase standardization and efficiency by reducing laboratory staff involvement in automatable activities. This will unleash time and resources for focus on more meaningful activities such as the complexities of patient care, research and development, and process improvement.
Collapse
Affiliation(s)
- Amrom E Obstfeld
- Department of Pathology & Laboratory Medicine, Children's Hospital of Philadelphia, Philadelphia, PA.,Department of Pathology & Laboratory Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
36
|
Barbetta A, Rocque B, Sarode D, Bartlett JA, Emamaullee J. Revisiting transplant immunology through the lens of single-cell technologies. Semin Immunopathol 2023; 45:91-109. [PMID: 35980400 PMCID: PMC9386203 DOI: 10.1007/s00281-022-00958-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Accepted: 07/06/2022] [Indexed: 11/03/2022]
Abstract
Solid organ transplantation (SOT) is the standard of care for end-stage organ disease. The most frequent complication of SOT involves allograft rejection, which may occur via T cell- and/or antibody-mediated mechanisms. Diagnosis of rejection in the clinical setting requires an invasive biopsy as there are currently no reliable biomarkers to detect rejection episodes. Likewise, it is virtually impossible to identify patients who exhibit operational tolerance and may be candidates for reduced or complete withdrawal of immunosuppression. Emerging single-cell technologies, including cytometry by time-of-flight (CyTOF), imaging mass cytometry, and single-cell RNA sequencing, represent a new opportunity for deep characterization of pathogenic immune populations involved in both allograft rejection and tolerance in clinical samples. These techniques enable examination of both individual cellular phenotypes and cell-to-cell interactions, ultimately providing new insights into the complex pathophysiology of allograft rejection. However, working with these large, highly dimensional datasets requires expertise in advanced data processing and analysis using computational biology techniques. Machine learning algorithms represent an optimal strategy to analyze and create predictive models using these complex datasets and will likely be essential for future clinical application of patient level results based on single-cell data. Herein, we review the existing literature on single-cell techniques in the context of SOT.
Collapse
Affiliation(s)
- Arianna Barbetta
- Department of Surgery, Division of Abdominal Organ Transplant, University of Southern California, 1510 San Pablo St. Suite 412, Los Angeles, CA, 90033, USA
- University of Southern California, Los Angeles, CA, USA
| | - Brittany Rocque
- Department of Surgery, Division of Abdominal Organ Transplant, University of Southern California, 1510 San Pablo St. Suite 412, Los Angeles, CA, 90033, USA
- University of Southern California, Los Angeles, CA, USA
| | - Deepika Sarode
- Department of Surgery, Division of Abdominal Organ Transplant, University of Southern California, 1510 San Pablo St. Suite 412, Los Angeles, CA, 90033, USA
- University of Southern California, Los Angeles, CA, USA
| | - Johanna Ascher Bartlett
- Pediatric Gastroenterology, Hepatology and Nutrition, Children's Hospital of Los Angeles, Los Angeles, CA, USA
| | - Juliet Emamaullee
- Department of Surgery, Division of Abdominal Organ Transplant, University of Southern California, 1510 San Pablo St. Suite 412, Los Angeles, CA, 90033, USA.
- University of Southern California, Los Angeles, CA, USA.
- Division of Hepatobiliary and Abdominal Organ Transplantation Surgery, Children's Hospital Los Angeles, Los Angeles, CA, USA.
| |
Collapse
|