Automated analysis of multidimensional flow cytometry data improves diagnostic accuracy between mantle cell lymphoma and small lymphocytic lymphoma

How artificial intelligence might disrupt diagnostics in hematology in the near future

Artificial intelligence (AI) is about to make itself indispensable in the health care sector. Examples of successful applications or promising approaches range from the application of pattern recognition software to pre-process and analyze digital medical images, to deep learning algorithms for subtype or disease classification, and digital twin technology and in silico clinical trials. Moreover, machine-learning techniques are used to identify patterns and anomalies in electronic health records and to perform ad-hoc evaluations of gathered data from wearable health tracking devices for deep longitudinal phenotyping. In the last years, substantial progress has been made in automated image classification, reaching even superhuman level in some instances. Despite the increasing awareness of the importance of the genetic context, the diagnosis in hematology is still mainly based on the evaluation of the phenotype. Either by the analysis of microscopic images of cells in cytomorphology or by the analysis of cell populations in bidimensional plots obtained by flow cytometry. Here, AI algorithms not only spot details that might escape the human eye, but might also identify entirely new ways of interpreting these images. With the introduction of high-throughput next-generation sequencing in molecular genetics, the amount of available information is increasing exponentially, priming the field for the application of machine learning approaches. The goal of all the approaches is to allow personalized and informed interventions, to enhance treatment success, to improve the timeliness and accuracy of diagnoses, and to minimize technically induced misclassifications. The potential of AI-based applications is virtually endless but where do we stand in hematology and how far can we go?

Machine learning and augmented human intelligence use in histomorphology for haematolymphoid disorders

Advances in digital pathology have allowed a number of opportunities such as decision support using artificial intelligence (AI). The application of AI to digital pathology data shows promise as an aid for pathologists in the diagnosis of haematological disorders. AI-based applications have embraced benign haematology, diagnosing leukaemia and lymphoma, as well as ancillary testing modalities including flow cytometry. In this review, we highlight the progress made to date in machine learning applications in haematopathology, summarise important studies in this field, and highlight key limitations. We further present our outlook on the future direction and trends for AI to support diagnostic decisions in haematopathology.

Computational analysis of flow cytometry data in hematological malignancies: future clinical practice?

PURPOSE OF REVIEW

This review outlines the advancements that have been made in computational analysis for clinical flow cytometry data in hematological malignancies.

RECENT FINDINGS

In recent years, computational analysis methods have been applied to clinical flow cytometry data of hematological malignancies with promising results. Most studies combined dimension reduction (principle component analysis) or clustering methods (FlowSOM, generalized mixture models) with machine learning classifiers (support vector machines, random forest). For diagnosis and classification of hematological malignancies, many studies have reported results concordant with manual expert analysis, including B-cell chronic lymphoid leukemia detection and acute leukemia classification. Other studies, e.g. concerning diagnosis of myelodysplastic syndromes and classification of lymphoma, have shown to be able to increase diagnostic accuracy. With respect to treatment response monitoring, studies have focused on, for example, computational minimal residual disease detection in multiple myeloma and posttreatment classification of healthy or diseased in acute myeloid leukemia. The results of these studies are encouraging, although accurate relapse prediction remains challenging. To facilitate clinical implementation, collaboration and (prospective) validation in multicenter setting are necessary.

SUMMARY

Computational analysis methods for clinical flow cytometry data hold the potential to increase ease of use, objectivity and accuracy in the clinical work-up of hematological malignancies.

A Clinically Applicable Approach to the Classification of B-Cell Non-Hodgkin Lymphomas with Flow Cytometry and Machine Learning

The immunophenotype is a key element to classify B-cell Non-Hodgkin Lymphomas (B-NHL); while it is routinely obtained through immunohistochemistry, the use of flow cytometry (FC) could bear several advantages. However, few FC laboratories can rely on a long-standing practical experience, and the literature in support is still limited; as a result, the use of FC is generally restricted to the analysis of lymphomas with bone marrow or peripheral blood involvement. In this work, we applied machine learning to our database of 1465 B-NHL samples from different sources, building four artificial predictive systems which could classify B-NHL in up to nine of the most common clinico-pathological entities. Our best model shows an overall accuracy of 92.68%, a mean sensitivity of 88.54% and a mean specificity of 98.77%. Beyond the clinical applicability, our models demonstrate (i) the strong discriminatory power of MIB1 and Bcl2, whose integration in the predictive model significantly increased the performance of the algorithm; (ii) the potential usefulness of some non-canonical markers in categorizing B-NHL; and (iii) that FC markers should not be described as strictly positive or negative according to fixed thresholds, but they rather correlate with different B-NHL depending on their level of expression.

From Bivariate to Multivariate Analysis of Cytometric Data: Overview of Computational Methods and Their Application in Vaccination Studies

Flow and mass cytometry are used to quantify the expression of multiple extracellular or intracellular molecules on single cells, allowing the phenotypic and functional characterization of complex cell populations. Multiparametric flow cytometry is particularly suitable for deep analysis of immune responses after vaccination, as it allows to measure the frequency, the phenotype, and the functional features of antigen-specific cells. When many parameters are investigated simultaneously, it is not feasible to analyze all the possible bi-dimensional combinations of marker expression with classical manual analysis and the adoption of advanced automated tools to process and analyze high-dimensional data sets becomes necessary. In recent years, the development of many tools for the automated analysis of multiparametric cytometry data has been reported, with an increasing record of publications starting from 2014. However, the use of these tools has been preferentially restricted to bioinformaticians, while few of them are routinely employed by the biomedical community. Filling the gap between algorithms developers and final users is fundamental for exploiting the advantages of computational tools in the analysis of cytometry data. The potentialities of automated analyses range from the improvement of the data quality in the pre-processing steps up to the unbiased, data-driven examination of complex datasets using a variety of algorithms based on different approaches. In this review, an overview of the automated analysis pipeline is provided, spanning from the pre-processing phase to the automated population analysis. Analysis based on computational tools might overcame both the subjectivity of manual gating and the operator-biased exploration of expected populations. Examples of applications of automated tools that have successfully improved the characterization of different cell populations in vaccination studies are also presented.

Machine Learning of Discriminative Gate Locations for Clinical Diagnosis

High-throughput single-cell cytometry technologies have significantly improved our understanding of cellular phenotypes to support translational research and the clinical diagnosis of hematological and immunological diseases. However, subjective and ad hoc manual gating analysis does not adequately handle the increasing volume and heterogeneity of cytometry data for optimal diagnosis. Prior work has shown that machine learning can be applied to classify cytometry samples effectively. However, many of the machine learning classification results are either difficult to interpret without using characteristics of cell populations to make the classification, or suboptimal due to the use of inaccurate cell population characteristics derived from gating boundaries. To date, little has been done to optimize both the gating boundaries and the diagnostic accuracy simultaneously. In this work, we describe a fully discriminative machine learning approach that can simultaneously learn feature representations (e.g., combinations of coordinates of gating boundaries) and classifier parameters for optimizing clinical diagnosis from cytometry measurements. The approach starts from an initial gating position and then refines the position of the gating boundaries by gradient descent until a set of globally-optimized gates across different samples are achieved. The learning procedure is constrained by regularization terms encoding domain knowledge that encourage the algorithm to seek interpretable results. We evaluate the proposed approach using both simulated and real data, producing classification results on par with those generated via human expertise, in terms of both the positions of the gating boundaries and the diagnostic accuracy. © 2019 The Authors. Cytometry Part A published by Wiley Periodicals, Inc. on behalf of International Society for Advancement of Cytometry.

From big flow cytometry datasets to smart diagnostic strategies: The EuroFlow approach

The rise in the analytical speed of mutiparameter flow cytometers made possible by the introduction of digital instruments, has brought up the possibility to manage progressively higher number of parameters simultaneously on significantly greater numbers of individual cells. This has led to an exponential increase in the complexity and volume of flow cytometry data generated about cells present in individual samples evaluated in a single measurement. This increase demands for new developments in flow cytometry data analysis, graphical representation, and visualization and interpretation tools to address the new big data challenges, i.e. processing data files of ≥10-25 parameters per cell in samples with >5-10 million cells (= up to 250 million data points per cell sample) obtained in a few minutes. Here, we present a comprehensive review of some of the tools developed by the EuroFlow consortium for processing flow cytometric big data files in diagnostic laboratories, particularly focused on automated EuroFlow approaches for: i) identification of all cell populations coexisting in a sample (automated gating); ii) smart classification of aberrant cell populations in routine diagnostics; iii) automated reporting; together with iv) new tools developed to visualize n-dimensional data in 2-dimensional plots to support expert-guided automated data analysis. The concept of using reference data bases implemented into software programs, in combination with multivariate statistical analysis pioneered by EuroFlow, provides an innovative, highly efficient and fast approach for diagnostic screening, classification and monitoring of patients with distinct hematological and immune disorders, as well as other diseases.

Development of a Novel Flow Cytometry-Based System for White Blood Cell Differential Counts: 10-color LeukoDiff

BACKGROUND

Flow cytometry (FCM) is commonly used to identify many cell populations. We developed a white blood cell (WBC) differential counting system for detecting abnormal cells using FCM incorporating 10 colors and 11 antibodies in a single tube, called "10-color LeukoDiff," and evaluated its performance.

METHODS

Ninety-one EDTA-anti-coagulated peripheral blood samples from 76 patients were analyzed using 10-color LeukoDiff. We compared 10 color LeukoDiff results with the results of manual differential count (manual diff). WBCs were classified into 17 cell populations: neutrophils, total lymphocytes, T lymphocytes, B lymphocytes, CD5 and CD19 co-expressing lymphocytes, natural killer cells, total monocytes, 16+ monocytes, eosinophils, immature granulocytes, basophils, myeloblasts, B-blasts, T-blasts, myeloid antigen-positive B-blasts, CD19- plasma cells, and 19+ plasma cells.

RESULTS

The correlations between the 10-color LeukoDiff and manual diff results were strong (r>0.9) for mature neutrophils, lymphocytes, eosinophils, immature granulocytes, and blasts and moderate for monocytes and basophils (r=0.86 and 0.74, respectively). There was no discrepancy in blast detection between 10-color LeukoDiff and manual diff results. Furthermore, 10-color LeukoDiff could differentiate the lineage of the blasts and separately count chronic lymphocytic leukemic cells and multiple myeloma cells.

CONCLUSIONS

The 10-color LeukoDiff provided an accurate and comprehensive WBC differential count. The most important ability of 10-color LeukoDiff is to detect blasts accurately. This system is clinically useful, especially for patients with hematologic diseases, such as acute leukemia, chronic lymphocytic leukemia, and multiple myeloma. Application of this system will improve the development of FCM gating strategy designs.

Clinical Implication of Multi-Parameter Flow Cytometry in Myelodysplastic Syndromes

Myelodysplastic syndromes (MDS) are a challenging group of diseases for clinicians and researchers, as both disease course and pathobiology are highly heterogeneous. In (suspected) MDS patients, multi-parameter flow cytometry can aid in establishing diagnosis, risk stratification and choice of therapy. This review addresses the developments and future directions of multi-parameter flow cytometry scores in MDS. Additionally, we propose an integrated diagnostic algorithm for suspected MDS.

Applications of Bayesian network models in predicting types of hematological malignancies

Network analysis is the preferred approach for the detection of subtle but coordinated changes in expression of an interacting and related set of genes. We introduce a novel method based on the analyses of coexpression networks and Bayesian networks, and we use this new method to classify two types of hematological malignancies; namely, acute myeloid leukemia (AML) and myelodysplastic syndrome (MDS). Our classifier has an accuracy of 93%, a precision of 98%, and a recall of 90% on the training dataset (n = 366); which outperforms the results reported by other scholars on the same dataset. Although our training dataset consists of microarray data, our model has a remarkable performance on the RNA-Seq test dataset (n = 74, accuracy = 89%, precision = 88%, recall = 98%), which confirms that eigengenes are robust with respect to expression profiling technology. These signatures are useful in classification and correctly predicting the diagnosis. They might also provide valuable information about the underlying biology of diseases. Our network analysis approach is generalizable and can be useful for classifying other diseases based on gene expression profiles. Our previously published Pigengene package is publicly available through Bioconductor, which can be used to conveniently fit a Bayesian network to gene expression data.

Automated Analysis of Clinical Flow Cytometry Data: A Chronic Lymphocytic Leukemia Illustration

Flow cytometry is used in cell-based diagnostic evaluation for blood-borne malignancies including leukemia and lymphoma. The current practice for cytometry data analysis relies on manual gating to identify cell subsets in complex mixtures, which is subjective, labor-intensive, and poorly reproducible. This article reviews recent efforts to develop, validate, and disseminate automated computational methods and pipelines for cytometry data analysis that could help overcome the limitations of manual analysis and provide for efficient and data-driven diagnostic applications. It demonstrates the performance of an optimized computational pipeline in a pilot study of chronic lymphocytic leukemia data from the authors' clinical diagnostic laboratory.

Automated Assessment of Disease Progression in Acute Myeloid Leukemia by Probabilistic Analysis of Flow Cytometry Data

OBJECTIVE

Flow cytometry (FC) is a widely acknowledged technology in diagnosis of acute myeloid leukemia (AML) and has been indispensable in determining progression of the disease. Although FC plays a key role as a posttherapy prognosticator and evaluator of therapeutic efficacy, the manual analysis of cytometry data is a barrier to optimization of reproducibility and objectivity. This study investigates the utility of our recently introduced nonparametric Bayesian framework in accurately predicting the direction of change in disease progression in AML patients using FC data.

METHODS

The highly flexible nonparametric Bayesian model based on the infinite mixture of infinite Gaussian mixtures is used for jointly modeling data from multiple FC samples to automatically identify functionally distinct cell populations and their local realizations. Phenotype vectors are obtained by characterizing each sample by the proportions of recovered cell populations, which are, in turn, used to predict the direction of change in disease progression for each patient.

RESULTS

We used 200 diseased and nondiseased immunophenotypic panels for training and tested the system with 36 additional AML cases collected at multiple time points. The proposed framework identified the change in direction of disease progression with accuracies of 90% (nine out of ten) for relapsing cases and 100% (26 out of 26) for the remaining cases.

CONCLUSIONS

We believe that these promising results are an important first step toward the development of automated predictive systems for disease monitoring and continuous response evaluation.

SIGNIFICANCE

Automated measurement and monitoring of therapeutic response is critical not only for objective evaluation of disease status prognosis but also for timely assessment of treatment strategies.

Computational flow cytometry: helping to make sense of high-dimensional immunology data

Recent advances in flow cytometry allow scientists to measure an increasing number of parameters per cell, generating huge and high-dimensional datasets. To analyse, visualize and interpret these data, newly available computational techniques should be adopted, evaluated and improved upon by the immunological community. Computational flow cytometry is emerging as an important new field at the intersection of immunology and computational biology; it allows new biological knowledge to be extracted from high-throughput single-cell data. This Review provides non-experts with a broad and practical overview of the many recent developments in computational flow cytometry.

Standardized Multi-Color Flow Cytometry and Computational Biomarker Discovery

Multi-color flow cytometry has become a valuable and highly informative tool for diagnosis and therapeutic monitoring of patients with immune deficiencies or inflammatory disorders. However, the method complexity and error-prone conventional manual data analysis often result in a high variability between different analysts and research laboratories. Here, we provide strategies and guidelines aiming at a more standardized multi-color flow cytometric staining and unsupervised data analysis for whole blood patient samples.

Understanding health and disease with multidimensional single-cell methods

Current efforts in the biomedical sciences and related interdisciplinary fields are focused on gaining a molecular understanding of health and disease, which is a problem of daunting complexity that spans many orders of magnitude in characteristic length scales, from small molecules that regulate cell function to cell ensembles that form tissues and organs working together as an organism. In order to uncover the molecular nature of the emergent properties of a cell, it is essential to measure multiple-cell components simultaneously in the same cell. In turn, cell heterogeneity requires multiple-cells to be measured in order to understand health and disease in the organism. This review summarizes current efforts towards a data-driven framework that leverages single-cell technologies to build robust signatures of healthy and diseased phenotypes. While some approaches focus on multicolor flow cytometry data and other methods are designed to analyze high-content image-based screens, we emphasize the so-called Supercell/SVM paradigm (recently developed by the authors of this review and collaborators) as a unified framework that captures mesoscopic-scale emergence to build reliable phenotypes. Beyond their specific contributions to basic and translational biomedical research, these efforts illustrate, from a larger perspective, the powerful synergy that might be achieved from bringing together methods and ideas from statistical physics, data mining, and mathematics to solve the most pressing problems currently facing the life sciences.

Flow cytometry bioinformatics

Flow cytometry bioinformatics is the application of bioinformatics to flow cytometry data, which involves storing, retrieving, organizing, and analyzing flow cytometry data using extensive computational resources and tools. Flow cytometry bioinformatics requires extensive use of and contributes to the development of techniques from computational statistics and machine learning. Flow cytometry and related methods allow the quantification of multiple independent biomarkers on large numbers of single cells. The rapid growth in the multidimensionality and throughput of flow cytometry data, particularly in the 2000s, has led to the creation of a variety of computational analysis methods, data standards, and public databases for the sharing of results. Computational methods exist to assist in the preprocessing of flow cytometry data, identifying cell populations within it, matching those cell populations across samples, and performing diagnosis and discovery using the results of previous steps. For preprocessing, this includes compensating for spectral overlap, transforming data onto scales conducive to visualization and analysis, assessing data for quality, and normalizing data across samples and experiments. For population identification, tools are available to aid traditional manual identification of populations in two-dimensional scatter plots (gating), to use dimensionality reduction to aid gating, and to find populations automatically in higher dimensional space in a variety of ways. It is also possible to characterize data in more comprehensive ways, such as the density-guided binary space partitioning technique known as probability binning, or by combinatorial gating. Finally, diagnosis using flow cytometry data can be aided by supervised learning techniques, and discovery of new cell types of biological importance by high-throughput statistical methods, as part of pipelines incorporating all of the aforementioned methods. Open standards, data, and software are also key parts of flow cytometry bioinformatics. Data standards include the widely adopted Flow Cytometry Standard (FCS) defining how data from cytometers should be stored, but also several new standards under development by the International Society for Advancement of Cytometry (ISAC) to aid in storing more detailed information about experimental design and analytical steps. Open data is slowly growing with the opening of the CytoBank database in 2010 and FlowRepository in 2012, both of which allow users to freely distribute their data, and the latter of which has been recommended as the preferred repository for MIFlowCyt-compliant data by ISAC. Open software is most widely available in the form of a suite of Bioconductor packages, but is also available for web execution on the GenePattern platform.

From cellular characteristics to disease diagnosis: uncovering phenotypes with supercells

Cell heterogeneity and the inherent complexity due to the interplay of multiple molecular processes within the cell pose difficult challenges for current single-cell biology. We introduce an approach that identifies a disease phenotype from multiparameter single-cell measurements, which is based on the concept of “supercell statistics”, a single-cell-based averaging procedure followed by a machine learning classification scheme. We are able to assess the optimal tradeoff between the number of single cells averaged and the number of measurements needed to capture phenotypic differences between healthy and diseased patients, as well as between different diseases that are difficult to diagnose otherwise. We apply our approach to two kinds of single-cell datasets, addressing the diagnosis of a premature aging disorder using images of cell nuclei, as well as the phenotypes of two non-infectious uveitides (the ocular manifestations of Behçet's disease and sarcoidosis) based on multicolor flow cytometry. In the former case, one nuclear shape measurement taken over a group of 30 cells is sufficient to classify samples as healthy or diseased, in agreement with usual laboratory practice. In the latter, our method is able to identify a minimal set of 5 markers that accurately predict Behçet's disease and sarcoidosis. This is the first time that a quantitative phenotypic distinction between these two diseases has been achieved. To obtain this clear phenotypic signature, about one hundred CD8⁺ T cells need to be measured. Although the molecular markers identified have been reported to be important players in autoimmune disorders, this is the first report pointing out that CD8⁺ T cells can be used to distinguish two systemic inflammatory diseases. Beyond these specific cases, the approach proposed here is applicable to datasets generated by other kinds of state-of-the-art and forthcoming single-cell technologies, such as multidimensional mass cytometry, single-cell gene expression, and single-cell full genome sequencing techniques.

The behavior of organisms is based on the concerted action occurring on an astonishing range of scales from the molecular to the organismal level. Molecular properties control the function of a cell, while cell ensembles form tissues and organs, which work together as an organism. In order to understand and characterize the molecular nature of the emergent properties of a cell, it is essential that multiple components of the cell are measured simultaneously in the same cell. Similarly, multiple cells must be measured in order to understand health and disease in the organism. In this work, we develop an approach that is able to determine how many cells, how many measurements per cell, and which measurements are needed to reliably diagnose disease. We apply this method to two different problems: the diagnosis of a premature aging disorder using images of cell nuclei, and the distinction between two similar autoimmune eye diseases using stained cells from patients' blood samples. Our findings shed new light on the role of specific kinds of immune system cells in systemic inflammatory diseases and may lead to improved diagnosis and treatment.

Computational analysis optimizes the flow cytometric evaluation for lymphoma

BACKGROUND

Although many clinical laboratories are adopting higher color flow cytometric assays, the approach to optimizing panel design and data analysis is often traditional and subjective. In order to address the question "What is the best flow cytometric strategy to reliably distinguish germinal center B-cell lymphoma (GC-L) from germinal center hyperplasia (GC-H)?" we applied a computational tool that identifies target populations correlated with a desired outcome, in this case diagnosis.

DESIGN

Cases of GC-H and GC-L evaluated by flow cytometric immunophenotyping using CD45, CD20, kappa, lambda, CD19, CD5, CD10, CD38, were analyzed with flowType and RchyOptimyx to construct cellular hierarchies that best distinguished the two diagnostic groups.

RESULTS

The population CD5-CD19+CD10+CD38- had the highest predictive power. Manual reanalysis confirmed significantly higher CD10+/CD38-B-cells in GC-L (median 12.44%, range 0.74-63.29, n = 52) than GC-H (median 0.24%, 0.03-4.49, n = 48, P = 0.0001), but was not entirely specific. Difficulties encountered using this computational approach included the presence of CD10+ granulocytes, continuously variable B-cell expression of CD38, more variable intensity antigen staining in GC-L and inability to assess the contribution of light chain restriction.

CONCLUSION

Computational analysis with construction of cellular hierarchies related to diagnosis helped guide manual analysis of high dimensional flow cytometric data. This approach highlighted the diagnostic utility of CD38 expression in the evaluation of B-cells with a CD10+ GC phenotype. In contrast to computational analysis of non-neoplastic cell populations, evaluation of neoplastic cells must be able to take into consideration increased variability in antigen expression.

Developments in the immunophenotypic analysis of haematological malignancies

Immunophenotyping is the method by which antibodies are used to detect cellular antigens in clinical samples. Although the major role is in the diagnosis and classification of haematological malignancies, applications have expanded over the past decade. Immunophenotyping is now used extensively for disease staging and monitoring, to detect surrogate markers of genetic aberrations, to identify potential immuno-therapeutic targets and to aid prognostic prediction. This expansion in applications has resulted from developments in antibodies, methodology, automation and data handling. In this review we describe recent advances in both the technology and applications for the analysis of haematological malignancies. We highlight the importance of the expanding repertoire of testing capability for diagnostic, prognostic and therapeutic applications. The impact and significance of immunophenotyping in the assessment of haematological neoplasms are evident.

Integration of lyoplate based flow cytometry and computational analysis for standardized immunological biomarker discovery

Discovery of novel immune biomarkers for monitoring of disease prognosis and response to therapy in immune-mediated inflammatory diseases is an important unmet clinical need. Here, we establish a novel framework for immunological biomarker discovery, comparing a conventional (liquid) flow cytometry platform (CFP) and a unique lyoplate-based flow cytometry platform (LFP) in combination with advanced computational data analysis. We demonstrate that LFP had higher sensitivity compared to CFP, with increased detection of cytokines (IFN-γ and IL-10) and activation markers (Foxp3 and CD25). Fluorescent intensity of cells stained with lyophilized antibodies was increased compared to cells stained with liquid antibodies. LFP, using a plate loader, allowed medium-throughput processing of samples with comparable intra- and inter-assay variability between platforms. Automated computational analysis identified novel immunophenotypes that were not detected with manual analysis. Our results establish a new flow cytometry platform for standardized and rapid immunological biomarker discovery with wide application to immune-mediated diseases.

A harmonized approach to intracellular cytokine staining gating: Results from an international multiconsortia proficiency panel conducted by the Cancer Immunotherapy Consortium (CIC/CRI)

Previous results from two proficiency panels of intracellular cytokine staining (ICS) from the Cancer Immunotherapy Consortium and panels from the National Institute of Allergy and Infectious Disease and the Association for Cancer Immunotherapy highlight the variability across laboratories in reported % CD8+ or % CD4+ cytokine-positive cells. One of the main causes of interassay variability in flow cytometry-based assays is due to differences in gating strategies between laboratories, which may prohibit the generation of robust results within single centers and across institutions. To study how gating strategies affect the variation in reported results, a gating panel was organized where all participants analyzed the same set of Flow Cytometry Standard (FCS) files from a four-color ICS assay using their own gating protocol (Phase I) and a gating protocol drafted by consensus from the organizers of the panel (Phase II). Focusing on analysis removed donor, assay, and instrument variation, enabling us to quantify the variability caused by gating alone. One hundred ten participating laboratories applied 110 different gating approaches. This led to high variability in the reported percentage of cytokine-positive cells and consequently in response detection in Phase I. However, variability was dramatically reduced when all laboratories used the same gating strategy (Phase II). Proximity of the cytokine gate to the negative population most impacted true-positive and false-positive response detection. Recommendations are provided for the (1) placement of the cytokine-positive gate, (2) identification of CD4+ CD8+ double-positive T cells, (3) placement of lymphocyte gate, (4) inclusion of dim cells, (5) gate uniformity, and 6) proper adjustment of the biexponential scaling.

Critical assessment of automated flow cytometry data analysis techniques

In this analysis, the authors directly compared the performance of flow cytometry data processing algorithms to manual gating approaches. The results offer information of practical utility about the performance of the algorithms as applied to different data sets and challenges.

Traditional methods for flow cytometry (FCM) data processing rely on subjective manual gating. Recently, several groups have developed computational methods for identifying cell populations in multidimensional FCM data. The Flow Cytometry: Critical Assessment of Population Identification Methods (FlowCAP) challenges were established to compare the performance of these methods on two tasks: (i) mammalian cell population identification, to determine whether automated algorithms can reproduce expert manual gating and (ii) sample classification, to determine whether analysis pipelines can identify characteristics that correlate with external variables (such as clinical outcome). This analysis presents the results of the first FlowCAP challenges. Several methods performed well as compared to manual gating or external variables using statistical performance measures, which suggests that automated methods have reached a sufficient level of maturity and accuracy for reliable use in FCM data analysis.

Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis

One challenge in applying bioinformatic tools to clinical or biological data is high number of features that might be provided to the learning algorithm without any prior knowledge on which ones should be used. In such applications, the number of features can drastically exceed the number of training instances which is often limited by the number of available samples for the study. The Lasso is one of many regularization methods that have been developed to prevent overfitting and improve prediction performance in high-dimensional settings. In this paper, we propose a novel algorithm for feature selection based on the Lasso and our hypothesis is that defining a scoring scheme that measures the "quality" of each feature can provide a more robust feature selection method. Our approach is to generate several samples from the training data by bootstrapping, determine the best relevance-ordering of the features for each sample, and finally combine these relevance-orderings to select highly relevant features. In addition to the theoretical analysis of our feature scoring scheme, we provided empirical evaluations on six real datasets from different fields to confirm the superiority of our method in exploratory data analysis and prediction performance. For example, we applied FeaLect, our feature scoring algorithm, to a lymphoma dataset, and according to a human expert, our method led to selecting more meaningful features than those commonly used in the clinics. This case study built a basis for discovering interesting new criteria for lymphoma diagnosis. Furthermore, to facilitate the use of our algorithm in other applications, the source code that implements our algorithm was released as FeaLect, a documented R package in CRAN.

Use of a single hybrid imaging agent for integration of target validation with in vivo and ex vivo imaging of mouse tumor lesions resembling human DCIS

Screening of biomarker expression levels in tumor biopsy samples not only provides an assessment of prognostic and predictive factors, but may also be used for selection of biomarker-specific imaging strategies. To assess the feasibility of using a biopsy specimen for a personalized selection of an imaging agent, the chemokine receptor 4 (CXCR4) was used as a reference biomarker.

Computational analysis of high-dimensional flow cytometric data for diagnosis and discovery

Recent technological advancements have enabled the flow cytometric measurement of tens of parameters on millions of cells. Conventional manual data analysis and bioinformatics tools cannot provide a complete analysis of these datasets due to this complexity. In this chapter we will provide an overview of a general data analysis pipeline both for automatic identification of cell populations of known importance (e.g., diagnosis by identification of predefined cell population) and for exploratory analysis of cohorts of flow cytometry assays (e.g., discovery of new correlates of a malignancy). We provide three real-world examples of how unsupervised discovery has been used in basic and clinical research. We also discuss challenges for evaluation of the algorithms developed for (1) identification of cell populations using clustering, (2) identification of specific cell populations, and (3) supervised analysis for discriminating between patient subgroups.

RchyOptimyx: cellular hierarchy optimization for flow cytometry

Analysis of high-dimensional flow cytometry datasets can reveal novel cell populations with poorly understood biology. Following discovery, characterization of these populations in terms of the critical markers involved is an important step, as this can help to both better understand the biology of these populations and aid in designing simpler marker panels to identify them on simpler instruments and with fewer reagents (i.e., in resource poor or highly regulated clinical settings). However, current tools to design panels based on the biological characteristics of the target cell populations work exclusively based on technical parameters (e.g., instrument configurations, spectral overlap, and reagent availability). To address this shortcoming, we developed RchyOptimyx (cellular hieraRCHY OPTIMization), a computational tool that constructs cellular hierarchies by combining automated gating with dynamic programming and graph theory to provide the best gating strategies to identify a target population to a desired level of purity or correlation with a clinical outcome, using the simplest possible marker panels. RchyOptimyx can assess and graphically present the trade-offs between marker choice and population specificity in high-dimensional flow or mass cytometry datasets. We present three proof-of-concept use cases for RchyOptimyx that involve 1) designing a panel of surface markers for identification of rare populations that are primarily characterized using their intracellular signature; 2) simplifying the gating strategy for identification of a target cell population; 3) identification of a non-redundant marker set to identify a target cell population.