Label-Free Identification of White Blood Cells Using Machine Learning

Label-free white blood cells classification using a deep feature fusion neural network

White blood cell (WBC) classification is a valuable diagnostic approach for identifying diseases. However, conventional methods for WBC detection, such as flow cytometers, have limitations in terms of their high cost, large system size, and laborious staining procedures. As a result, deep learning-based label-free WBC image analysis methods are gaining popularity. Nevertheless, most existing deep learning WBC classification techniques fail to effectively utilize the subtle differences in the internal structures of WBCs observed under a microscope. To address this issue, we propose a neural network with feature fusion in this study, which enables the detection of label-free WBCs. Unlike conventional convolutional neural networks (CNNs), our approach combines low-level features extracted by shallow layers with high-level features extracted by deep layers, generating fused features for accurate bright-field WBC identification. Our method achieves an accuracy of 80.3 % on the testing set, demonstrating a potential solution for deep-learning-based biomedical diagnoses. Considering the proposed method simplifies the cell detection process and eliminates the need for complex operations like fluorescent staining, we anticipate that this automatic and label-free WBC classification network could facilitate more precise and effective analysis, and it could contribute to the future adoption of miniatured flow cytometers for point-of-care (POC) diagnostics applications.

Stain-Free Approach to Determine and Monitor Cell Heath Using Supervised and Unsupervised Image-Based Deep Learning

Cell-based medicinal products (CBMPs) are a growing class of therapeutics that promise new treatments for complex and rare diseases. Given the inherent complexity of the whole human cells comprising CBMPs, there is a need for robust and fast analytical methods for characterization, process monitoring, and quality control (QC) testing during their manufacture. Existing techniques to evaluate and monitor cell quality typically constitute labor-intensive, expensive, and highly specific staining assays. In this work, we combine image-based deep learning with flow imaging microscopy (FIM) to predict cell health metrics using cellular morphology "fingerprints" extracted from images of unstained Jurkat cells (immortalized human T-lymphocyte cells). A supervised (i.e., algorithm trained with human-generated labels for images) fingerprinting algorithm, trained on images of unstained healthy and dead cells, provides a robust stain-free, non-invasive, and non-destructive method for determining cell viability. Results from the stain-free method are in good agreement with traditional stain-based cytometric viability measurements. Additionally, when trained with images of healthy cells, dead cells and cells undergoing chemically induced apoptosis, the supervised fingerprinting algorithm is able to distinguish between the three cell states, and the results are independent of specific treatments or signaling pathways. We then show that an unsupervised variational autoencoder (VAE) algorithm trained on the same images, but without human-generated labels, is able to distinguish between samples of healthy, dead and apoptotic cells along with cellular debris based on learned morphological features and without human input. With this, we demonstrate that VAEs are a powerful exploratory technique that can be used as a process monitoring analytical tool.

Comparative analysis of feature-based ML and CNN for binucleated erythroblast quantification in myelodysplastic syndrome patients using imaging flow cytometry data

Myelodysplastic syndrome is primarily characterized by dysplasia in the bone marrow (BM), presenting a challenge in consistent morphology interpretation. Accurate diagnosis through traditional slide-based analysis is difficult, necessitating a standardized objective technique. Over the past two decades, imaging flow cytometry (IFC) has proven effective in combining image-based morphometric analyses with high-parameter phenotyping. We have previously demonstrated the effectiveness of combining IFC with a feature-based machine learning algorithm to accurately identify and quantify rare binucleated erythroblasts (BNEs) in dyserythropoietic BM cells. However, a feature-based workflow poses challenges requiring software-specific expertise. Here we employ a Convolutional Neural Network (CNN) algorithm for BNE identification and differentiation from doublets and cells with irregular nuclear morphology in IFC data. We demonstrate that this simplified AI workflow, coupled with a powerful CNN algorithm, achieves comparable BNE quantification accuracy to manual and feature-based analysis with substantial time savings, eliminating workflow complexity. This streamlined approach holds significant clinical value, enhancing IFC accessibility for routine diagnostic purposes.

Classification and counting of cells in brightfield microscopy images: an application of convolutional neural networks

Microscopy is integral to medical research, facilitating the exploration of various biological questions, notably cell quantification. However, this process's time-consuming and error-prone nature, attributed to human intervention or automated methods usually applied to fluorescent images, presents challenges. In response, machine learning algorithms have been integrated into microscopy, automating tasks and constructing predictive models from vast datasets. These models adeptly learn representations for object detection, image segmentation, and target classification. An advantageous strategy involves utilizing unstained images, preserving cell integrity and enabling morphology-based classification-something hindered when fluorescent markers are used. The aim is to introduce a model proficient in classifying distinct cell lineages in digital contrast microscopy images. Additionally, the goal is to create a predictive model identifying lineage and determining optimal quantification of cell numbers. Employing a CNN machine learning algorithm, a classification model predicting cellular lineage achieved a remarkable accuracy of 93%, with ROC curve results nearing 1.0, showcasing robust performance. However, some lineages, namely SH-SY5Y (78%), HUH7_mayv (85%), and A549 (88%), exhibited slightly lower accuracies. These outcomes not only underscore the model's quality but also emphasize CNNs' potential in addressing the inherent complexities of microscopic images.

PXPermute reveals staining importance in multichannel imaging flow cytometry

Imaging flow cytometry (IFC) allows rapid acquisition of numerous single-cell images per second, capturing information from multiple fluorescent channels. However, the traditional process of staining cells with fluorescently labeled conjugated antibodies for IFC analysis is time consuming, expensive, and potentially harmful to cell viability. To streamline experimental workflows and reduce costs, it is crucial to identify the most relevant channels for downstream analysis. In this study, we introduce PXPermute, a user-friendly and powerful method for assessing the significance of IFC channels, particularly for cell profiling. Our approach evaluates channel importance by permuting pixel values within each channel and analyzing the resulting impact on machine learning or deep learning models. Through rigorous evaluation of three multichannel IFC image datasets, we demonstrate PXPermute's potential in accurately identifying the most informative channels, aligning with established biological knowledge. PXPermute can assist biologists with systematic channel analysis, experimental design optimization, and biomarker identification.

DeepIFC: Virtual fluorescent labeling of blood cells in imaging flow cytometry data with deep learning

Imaging flow cytometry (IFC) combines flow cytometry with microscopy, allowing rapid characterization of cellular and molecular properties via high-throughput single-cell fluorescent imaging. However, fluorescent labeling is costly and time-consuming. We present a computational method called DeepIFC based on the Inception U-Net neural network architecture, able to generate fluorescent marker images and learn morphological features from IFC brightfield and darkfield images. Furthermore, the DeepIFC workflow identifies cell types from the generated fluorescent images and visualizes the single-cell features generated in a 2D space. We demonstrate that rarer cell types are predicted well when a balanced data set is used to train the model, and the model is able to recognize red blood cells not seen during model training as a distinct entity. In summary, DeepIFC allows accurate cell reconstruction, typing and recognition of unseen cell types from brightfield and darkfield images via virtual fluorescent labeling.

High-Throughput Single-Cell Analysis of Nanoparticle-Cell Interactions

Understanding nanoparticle-cell interactions at single-nanoparticle and single-cell resolutions is crucial to improving the design of next-generation nanoparticles for safer, more effective, and more efficient applications in nanomedicine. This review focuses on recent advances in the continuous high-throughput analysis of nanoparticle-cell interactions at the single-cell level. We highlight and discuss the current trends in continual flow high-throughput methods for analyzing single cells, such as advanced flow cytometry techniques and inductively coupled plasma mass spectrometry methods, as well as their intersection in the form of mass cytometry. This review further discusses the challenges and opportunities with current single-cell analysis approaches and provides proposed directions for innovation in the high-throughput analysis of nanoparticle-cell interactions.

Machine learning implementation strategy in imaging and impedance flow cytometry

Imaging and impedance flow cytometry is a label-free technique that has shown promise as a potential replacement for standard flow cytometry. This is due to its ability to provide rich information and archive high-throughput analysis. Recently, significant efforts have been made to leverage machine learning for processing the abundant data generated by those techniques, enabling rapid and accurate analysis. Harnessing the power of machine learning, imaging and impedance flow cytometry has demonstrated its capability to address various complex phenotyping scenarios. Herein, we present a comprehensive overview of the detailed strategies for implementing machine learning in imaging and impedance flow cytometry. We initiate the discussion by outlining the commonly employed setup to acquire the data (i.e., image or signal) from the cell. Subsequently, we delve into the necessary processes for extracting features from the acquired image or signal data. Finally, we discuss how these features can be utilized for cell phenotyping through the application of machine learning algorithms. Furthermore, we discuss the existing challenges and provide insights for future perspectives of intelligent imaging and impedance flow cytometry.

On-chip label-free cell classification based directly on off-axis holograms and spatial-frequency-invariant deep learning

We present a rapid label-free imaging flow cytometry and cell classification approach based directly on raw digital holograms. Off-axis holography enables real-time acquisition of cells during rapid flow. However, classification of the cells typically requires reconstruction of their quantitative phase profiles, which is time-consuming. Here, we present a new approach for label-free classification of individual cells based directly on the raw off-axis holographic images, each of which contains the complete complex wavefront (amplitude and quantitative phase profiles) of the cell. To obtain this, we built a convolutional neural network, which is invariant to the spatial frequencies and directions of the interference fringes of the off-axis holograms. We demonstrate the effectiveness of this approach using four types of cancer cells. This approach has the potential to significantly improve both speed and robustness of imaging flow cytometry, enabling real-time label-free classification of individual cells.

High-throughput measurement of double strand break global repair phenotype in peripheral blood mononuclear cells after long-term cryopreservation

Peripheral blood mononuclear cells (PBMCs) are a useful model for biochemical assays, particularly for etiological studies. We describe here a method for measuring DNA repair capacity (DRC) in archival cryogenically preserved PBMCs. To model DRC, we measured γ-H2AX repair kinetics in thawed PBMCs after irradiation with 3 Gy gamma rays. Time-dependent fluorescently labeled γ-H2AX levels were measured at five time points from 1 to 20 h, yielding an estimate of global DRC repair kinetics as well as a measure of unrepaired double strand breaks at 20 h. While γ-H2AX levels are traditionally measured by either microscopy or flow-cytometry, we developed a protocol for imaging flow cytometry (IFC) that combines the detailed information of microscopy with the statistical power of flow methods. The visual imaging component of the IFC allows for monitoring aspects such as cellular health and apoptosis as well as fluorescence localization of the γ-H2AX signal, which ensures the power and significance of this technique. Application of a machine-learning based image classification improved flow cytometry fluorescent measurements by identifying apoptotic cells unable to undergo DNA repair. We present here DRC repair parameters from 18 frozen archival PBMCs and 28 fresh blood samples collected from a demographically diverse cohort of women measured in a high-throughput IFC format. This thaw method and assay can be used alone or in conjunction with other assays to measure etiological phenotypes in cryogenic biobanks of PBMCs.

Recent Approaches to Design and Analysis of Electrical Impedance Systems for Single Cells Using Machine Learning

Individual cells have many unique properties that can be quantified to develop a holistic understanding of a population. This can include understanding population characteristics, identifying subpopulations, or elucidating outlier characteristics that may be indicators of disease. Electrical impedance measurements are rapid and label-free for the monitoring of single cells and generate large datasets of many cells at single or multiple frequencies. To increase the accuracy and sensitivity of measurements and define the relationships between impedance and biological features, many electrical measurement systems have incorporated machine learning (ML) paradigms for control and analysis. Considering the difficulty capturing complex relationships using traditional modelling and statistical methods due to population heterogeneity, ML offers an exciting approach to the systemic collection and analysis of electrical properties in a data-driven way. In this work, we discuss incorporation of ML to improve the field of electrical single cell analysis by addressing the design challenges to manipulate single cells and sophisticated analysis of electrical properties that distinguish cellular changes. Looking forward, we emphasize the opportunity to build on integrated systems to address common challenges in data quality and generalizability to save time and resources at every step in electrical measurement of single cells.

Single-Cell Classification Based on Population Nucleus Size Combining Microwave Impedance Spectroscopy and Machine Learning

Many recent efforts in the diagnostic field address the accessibility of cancer diagnosis. Typical histological staining methods identify cancer cells visually by a larger nucleus with more condensed chromatin. Machine learning (ML) has been incorporated into image analysis for improving this process. Recently, impedance spectrometers have been shown to generate all-inclusive lab-on-a-chip platforms to detect nucleus abnormities. In this paper, a wideband electrical sensor and data analysis paradigm that can identify nuclear changes shows the realization of a single-cell microfluidic device to detect nuclei of altered sizes. To model cells of altered nucleus, Jurkat cells were treated to enlarge or shrink their nucleus followed by broadband sensing to obtain the S-parameters of single cells. The ability to deduce important frequencies associated with nucleus size is demonstrated and used to improve classification models in both binary and multiclass scenarios, despite a heterogeneous and overlapping cell population. The important frequency features match those predicted in a double-shell circuit model published in prior work, demonstrating a coherent new analytical technique for electrical data analysis. The electrical sensing platform assisted by ML with impressive accuracy of cell classification looks forward to a label-free and flexible approach to cancer diagnosis.

Microscopic image-based classification of adipocyte differentiation by machine learning

Adipocyte differentiation is a sequential process involving increased expression of peroxisome proliferator-activated receptor gamma (PPARγ), adipocyte-specific gene expression, and accumulation of lipid droplets in the cytoplasm. Expression of the transcription factors involved is usually detected using canonical biochemical or biomolecular procedures such as Western blotting or qPCR of pooled cell lysates. While this provides a useful average index for adipogenesis for some populations, the precise stage of adipogenesis cannot be distinguished at the single-cell level, because the heterogenous nature of differentiation among cells limits the utility of averaged data. We have created a classifier to sort cells, and used it to determine the stage of adipocyte differentiation at the single-cell level. We used a machine learning method with microscopic images of cell stained for PPARγ and lipid droplets as input data. Our results show that the classifier can successfully determine the precise stage of differentiation. Stage classification and subsequent model fitting using the sequential reaction model revealed the action of pioglitazone and rosiglitazone to be promotion of transition from the stage of increased PPARγ expression to the next stage. This indicates that these drugs are PPARγ agonists, and that our classifier and model can accurately estimate drug action points and would be suitable for evaluating the stage/state of individual cells during differentiation or disease progression. The incorporation of both biochemical and morphological information derived from immunofluorescence image of cells and so overcomes limitations of current models.

White blood cell detection, classification and analysis using phase imaging with computational specificity (PICS)

Treatment of blood smears with Wright's stain is one of the most helpful tools in detecting white blood cell abnormalities. However, to diagnose leukocyte disorders, a clinical pathologist must perform a tedious, manual process of locating and identifying individual cells. Furthermore, the staining procedure requires considerable preparation time and clinical infrastructure, which is incompatible with point-of-care diagnosis. Thus, rapid and automated evaluations of unlabeled blood smears are highly desirable. In this study, we used color spatial light interference microcopy (cSLIM), a highly sensitive quantitative phase imaging (QPI) technique, coupled with deep learning tools, to localize, classify and segment white blood cells (WBCs) in blood smears. The concept of combining QPI label-free data with AI for the purpose of extracting cellular specificity has recently been introduced in the context of fluorescence imaging as phase imaging with computational specificity (PICS). We employed AI models to first translate SLIM images into brightfield micrographs, then ran parallel tasks of locating and labelling cells using EfficientNet, which is an object detection model. Next, WBC binary masks were created using U-net, a convolutional neural network that performs precise segmentation. After training on digitally stained brightfield images of blood smears with WBCs, we achieved a mean average precision of 75% for localizing and classifying neutrophils, eosinophils, lymphocytes, and monocytes, and an average pixel-wise majority-voting F1 score of 80% for determining the cell class from semantic segmentation maps. Therefore, PICS renders and analyzes synthetically stained blood smears rapidly, at a reduced cost of sample preparation, providing quantitative clinical information.

Imaging flow cytometry: a primer

Imaging flow cytometry combines the high throughput nature of flow cytometry with the advantages of single cell image acquisition associated with microscopy. The measurement of large numbers of features from the resulting images provides rich datasets which have resulted in a wide range of novel biomedical applications. In this primer we discuss the typical imaging flow instrumentation, the form of data acquired and the typical analysis tools that can be applied to this data. Using examples from the literature we discuss the progression of the analysis methods that have been applied to imaging flow cytometry data. These methods start from the use of simple single image features and multiple channel gating strategies, followed by the design and use of custom features for phenotype classification, through to powerful machine and deep learning methods. For each of these methods, we outline the processes involved in analyzing typical datasets and provide details of example applications. Finally we discuss the current limitations of imaging flow cytometry and the innovations which are addressing these challenges.

Combining imaging flow cytometry and machine learning for high-throughput schistocyte quantification: A SVM classifier development and external validation cohort

Background

Schistocyte counts are a cornerstone of the diagnosis of thrombotic microangiopathy syndrome (TMA). Their manual quantification is complex and alternative automated methods suffer from pitfalls that limit their use. We report a method combining imaging flow cytometry (IFC) and artificial intelligence for the direct label-free and operator-independent quantification of schistocytes in whole blood.

Methods

We used 135,045 IFC images from blood acquisition among 14 patients to extract 188 features with IDEAS® software and 128 features from a convolutional neural network (CNN) with Keras framework in order to train a support vector machine (SVM) blood elements’ classifier used for schistocytes quantification.

Finding

Keras features showed better accuracy (94.03%, CI: 93.75-94.31%) than ideas features (91.54%, CI: 91.21-91.87%) in recognising whole-blood elements, and together they showed the best accuracy (95.64%, CI: 95.39-95.88%). We obtained an excellent correlation (0.93, CI: 0.90-0.96) between three haematologists and our method on a cohort of 102 patient samples. All patients with schistocytosis (>1% schistocytes) were detected with excellent specificity (91.3%, CI: 82.0-96.7%) and sensitivity (100%, CI: 89.4-100.0%). We confirmed these results with a similar specificity (91.1%, CI: 78.8-97.5%) and sensitivity (100%, CI: 88.1-100.0%) on a validation cohort (n=74) analysed in an independent healthcare centre. Simultaneous analysis of 16 samples in both study centres showed a very good correlation between the 2 imaging flow cytometers (Y=1.001x).

Interpretation

We demonstrate that IFC can represent a reliable tool for operator-independent schistocyte quantification with no pre-analytical processing which is of most importance in emergency situations such as TMA.

Funding

None.

Virtual Staining, Segmentation, and Classification of Blood Smears for Label-Free Hematology Analysis

Objective and Impact Statement. We present a fully automated hematological analysis framework based on single-channel (single-wavelength), label-free deep-ultraviolet (UV) microscopy that serves as a fast, cost-effective alternative to conventional hematology analyzers. Introduction. Hematological analysis is essential for the diagnosis and monitoring of several diseases but requires complex systems operated by trained personnel, costly chemical reagents, and lengthy protocols. Label-free techniques eliminate the need for staining or additional preprocessing and can lead to faster analysis and a simpler workflow. In this work, we leverage the unique capabilities of deep-UV microscopy as a label-free, molecular imaging technique to develop a deep learning-based pipeline that enables virtual staining, segmentation, classification, and counting of white blood cells (WBCs) in single-channel images of peripheral blood smears. Methods. We train independent deep networks to virtually stain and segment grayscale images of smears. The segmented images are then used to train a classifier to yield a quantitative five-part WBC differential. Results. Our virtual staining scheme accurately recapitulates the appearance of cells under conventional Giemsa staining, the gold standard in hematology. The trained cellular and nuclear segmentation networks achieve high accuracy, and the classifier can achieve a quantitative five-part differential on unseen test data. Conclusion. This proposed automated hematology analysis framework could greatly simplify and improve current complete blood count and blood smear analysis and lead to the development of a simple, fast, and low-cost, point-of-care hematology analyzer.

Immunomagnetic leukocyte differential in whole blood on an electronic microdevice

Leukocytes are the frontline defense mechanism of the immune system. Their composition dynamically changes as a response to a foreign body, infection, inflammation, or other malignant behavior occurring within the body. Monitoring the composition of leukocytes, namely leukocyte differential, is a crucial assay periodically performed to diagnose an infection or to assess a person's vulnerability for a health anomaly. Currently, leukocyte differential analysis is performed using hematology analyzers or flow cytometers, both of which are bulky instruments that require trained and certified personnel for operation. In this work, we demonstrate a new technique to obtain leukocyte differentials in a highly portable and integrated microfluidic chip by magnetically analyzing the CD33 expression of leukocytes. When benchmarked against conventional laboratory instruments, our technology demonstrated <5% difference on average for all subtypes. Our results show that hematology testing could be performed beyond the centralized laboratories at a low cost and ultimately provide point-of-care and at-home testing opportunities.

Rapid and Label-Free Classification of Blood Leukocytes for Immune State Monitoring

A fully automated and label-free sample-to-answer white blood cell (WBC) cytometry platform for rapid immune state monitoring is demonstrated. The platform integrates (1) a WBC separation process using the multidimensional double spiral (MDDS) device and (2) an imaging process where images of the separated WBCs are captured and analyzed. Using the deep-learning-based image processing technique, we analyzed the captured bright-field images to classify the WBCs into their subtypes. Furthermore, in addition to cell classification, we can detect activation-induced morphological changes in WBCs for functional immune assessment, which could allow the early detection of various diseases. The integrated platform operates in a rapid (<30 min), fully automated, and label-free manner. The platform could provide a promising solution to future point-of-care WBC diagnostics applications.

Application of Machine Learning Solutions to Optimize Parameter Prediction to Enhance Automatic NMR Metabolite Profiling

The quality of automatic metabolite profiling in NMR datasets from complex matrices can be affected by the numerous sources of variability. These sources, as well as the presence of multiple low-intensity signals, cause uncertainty in the metabolite signal parameters. Lineshape fitting approaches often produce suboptimal resolutions to adapt them in a complex spectrum lineshape. As a result, the use of software tools for automatic profiling tends to be restricted to specific biological matrices and/or sample preparation protocols to obtain reliable results. However, the analysis and modelling of the signal parameters collected during initial iteration can be further optimized to reduce uncertainty by generating narrow and accurate predictions of the expected signal parameters. In this study, we show that, thanks to the predictions generated, better profiling quality indicators can be outputted, and the performance of automatic profiling can be maximized. Our proposed workflow can learn and model the sample properties; therefore, restrictions in the biological matrix, or sample preparation protocol, and limitations of lineshape fitting approaches can be overcome.

Deep Learning-Based Classification of Cancer Cell in Leptomeningeal Metastasis on Cytomorphologic Features of Cerebrospinal Fluid

Background

It is a critical challenge to diagnose leptomeningeal metastasis (LM), given its technical difficulty and the lack of typical symptoms. The existing gold standard of diagnosing LM is to use positive cerebrospinal fluid (CSF) cytology, which consumes significantly more time to classify cells under a microscope.

Objective

This study aims to establish a deep learning model to classify cancer cells in CSF, thus facilitating doctors to achieve an accurate and fast diagnosis of LM in an early stage.

Method

The cerebrospinal fluid laboratory of Xijing Hospital provides 53,255 cells from 90 LM patients in the research. We used two deep convolutional neural networks (CNN) models to classify cells in the CSF. A five-way cell classification model (CNN1) consists of lymphocytes, monocytes, neutrophils, erythrocytes, and cancer cells. A four-way cancer cell classification model (CNN2) consists of lung cancer cells, gastric cancer cells, breast cancer cells, and pancreatic cancer cells. Here, the CNN models were constructed by Resnet-inception-V2. We evaluated the performance of the proposed models on two external datasets and compared them with the results from 42 doctors of various levels of experience in the human-machine tests. Furthermore, we develop a computer-aided diagnosis (CAD) software to generate cytology diagnosis reports in the research rapidly.

Results

With respect to the validation set, the mean average precision (mAP) of CNN1 is over 95% and that of CNN2 is close to 80%. Hence, the proposed deep learning model effectively classifies cells in CSF to facilitate the screening of cancer cells. In the human-machine tests, the accuracy of CNN1 is similar to the results from experts, with higher accuracy than doctors in other levels. Moreover, the overall accuracy of CNN2 is 10% higher than that of experts, with a time consumption of only one-third of that consumed by an expert. Using the CAD software saves 90% working time of cytologists.

Conclusion

A deep learning method has been developed to assist the LM diagnosis with high accuracy and low time consumption effectively. Thanks to labeled data and step-by-step training, our proposed method can successfully classify cancer cells in the CSF to assist LM diagnosis early. In addition, this unique research can predict cancer’s primary source of LM, which relies on cytomorphologic features without immunohistochemistry. Our results show that deep learning can be widely used in medical images to classify cerebrospinal fluid cells. For complex cancer classification tasks, the accuracy of the proposed method is significantly higher than that of specialist doctors, and its performance is better than that of junior doctors and interns. The application of CNNs and CAD software may ultimately aid in expediting the diagnosis and overcoming the shortage of experienced cytologists, thereby facilitating earlier treatment and improving the prognosis of LM.

Machine learning-based approaches for identifying human blood cells harboring CRISPR-mediated fetal chromatin domain ablations

Two common hemoglobinopathies, sickle cell disease (SCD) and β-thalassemia, arise from genetic mutations within the β-globin gene. In this work, we identified a 500-bp motif (Fetal Chromatin Domain, FCD) upstream of human ϒ-globin locus and showed that the removal of this motif using CRISPR technology reactivates the expression of ϒ-globin. Next, we present two different cell morphology-based machine learning approaches that can be used identify human blood cells (KU-812) that harbor CRISPR-mediated FCD genetic modifications. Three candidate models from the first approach, which uses multilayer perceptron algorithm (MLP 20-26, MLP26-18, and MLP 30-26) and flow cytometry-derived cellular data, yielded 0.83 precision, 0.80 recall, 0.82 accuracy, and 0.90 area under the ROC (receiver operating characteristic) curve when predicting the edited cells. In comparison, the candidate model from the second approach, which uses deep learning (T2D5) and DIC microscopy-derived imaging data, performed with less accuracy (0.80) and ROC AUC (0.87). We envision that equivalent machine learning-based models can complement currently available genotyping protocols for specific genetic modifications which result in morphological changes in human cells.

Non-invasive and label-free identification of human natural killer cell subclasses by biophysical single-cell features in microfluidic flow

Natural killer (NK) cells are indicated as favorite candidates for innovative therapeutic treatment and are divided into two subclasses: immature regulatory NK CD56^bright and mature cytotoxic NK CD56^dim. Therefore, the ability to discriminate CD56^dim from CD56^bright could be very useful because of their higher cytotoxicity. Nowadays, NK cell classification is routinely performed by cytometric analysis based on surface receptor expression. Here, we present an in-flow, label-free and non-invasive biophysical analysis of NK cells through a combination of light scattering and machine learning (ML) for NK cell subclass classification. In this respect, to identify relevant biophysical cell features, we stimulated NK cells with interleukine-15 inducing a subclass transition from CD56^bright to CD56^dim. We trained our ML algorithm with sorted NK cell subclasses (≥86% accuracy). Next, we applied our NK cell classification algorithm to cells stimulated over time, to investigate the transition of CD56^bright to CD56^dim and their biophysical feature changes. Finally, we tested our approach on several proband samples, highlighting the potential of our measurement approach. We show a label-free way for the robust identification of NK cell subclasses based on biophysical features, which can be applied in both cell biology and cell therapy.

A weakly supervised deep learning approach for label-free imaging flow-cytometry-based blood diagnostics

The application of machine learning approaches to imaging flow cytometry (IFC) data has the potential to transform the diagnosis of hematological diseases. However, the need for manually labeled single-cell images for machine learning model training has severely limited its clinical application. To address this, we present iCellCnn, a weakly supervised deep learning approach for label-free IFC-based blood diagnostics. We demonstrate the capability of iCellCnn to achieve diagnosis of Sézary syndrome (SS) from patient samples on the basis of bright-field IFC images of T cells obtained after fluorescence-activated cell sorting of human peripheral blood mononuclear cell specimens. With a sample size of four healthy donors and five SS patients, iCellCnn achieved a 100% classification accuracy. As iCellCnn is not restricted to the diagnosis of SS, we expect such weakly supervised approaches to tap the diagnostic potential of IFC by providing automatic data-driven diagnosis of diseases with so-far unknown morphological manifestations.

High-Efficiency Classification of White Blood Cells Based on Object Detection

White blood cells (WBCs) play a significant role in the human immune system, and the content of various subtypes of WBCs is usually maintained within a certain range in the human body, while deviant levels are important warning signs for diseases. Hence, the detection and classification of WBCs is an essential diagnostic technique. However, traditional WBC classification technologies based on image processing usually need to segment the collected target cell images from the background. This preprocessing operation not only increases the workload but also heavily affects the classification quality and efficiency. Therefore, we proposed one high-efficiency object detection technology that combines the segmentation and recognition of targets into one step to realize the detection and classification of WBCs in an image at the same time. Two state-of-the-art object detection models, Faster RCNN and Yolov4, were employed and comparatively studied to classify neutrophils, eosinophils, monocytes, and lymphocytes on a balanced and enhanced Blood Cell Count Dataset (BCCD). Our experimental results showed that the Faster RCNN and Yolov4 based deep transfer learning models achieved classification accuracy rates of 96.25% and 95.75%, respectively. For the one-stage model, Yolov4, while ensuring more than 95% accuracy, its detection speed could reach 60 FPS, which showed better performance compared with the two-stage model, Faster RCNN. The high-efficiency object detection network that does not require cell presegmentation can remove the difficulty of image preprocessing and greatly improve the efficiency of the entire classification task, which provides a potential solution for future real-time point-of-care diagnostic systems.

Autofluorescence of blood and its application in biomedical and clinical research

Autofluorescence of blood has been explored as a label free approach for detection of cell types, as well as for diagnosis and detection of infection, cancer, and other diseases. Although blood autofluorescence is used to indicate the presence of several physiological abnormalities with high sensitivity, it often lacks disease specificity due to use of a limited number of fluorophores in the detection of several abnormal conditions. In addition, the measurement of autofluorescence is sensitive to the type of sample, sample preparation, and spectroscopy method used for the measurement. Therefore, while current blood autofluorescence detection approaches may not be suitable for primary clinical diagnosis, it certainly has tremendous potential in developing methods for large scale screening that can identify high risk groups for further diagnosis using highly specific diagnostic tests. This review discusses the source of blood autofluorescence, the role of spectroscopy methods, and various applications that have used autofluorescence of blood, to explore the potential of blood autofluorescence in biomedical research and clinical applications.

Label-Free White Blood Cell Classification Using Refractive Index Tomography and Deep Learning

Objective and Impact Statement. We propose a rapid and accurate blood cell identification method exploiting deep learning and label-free refractive index (RI) tomography. Our computational approach that fully utilizes tomographic information of bone marrow (BM) white blood cell (WBC) enables us to not only classify the blood cells with deep learning but also quantitatively study their morphological and biochemical properties for hematology research. Introduction. Conventional methods for examining blood cells, such as blood smear analysis by medical professionals and fluorescence-activated cell sorting, require significant time, costs, and domain knowledge that could affect test results. While label-free imaging techniques that use a specimen's intrinsic contrast (e.g., multiphoton and Raman microscopy) have been used to characterize blood cells, their imaging procedures and instrumentations are relatively time-consuming and complex. Methods. The RI tomograms of the BM WBCs are acquired via Mach-Zehnder interferometer-based tomographic microscope and classified by a 3D convolutional neural network. We test our deep learning classifier for the four types of bone marrow WBC collected from healthy donors (n = 10 ): monocyte, myelocyte, B lymphocyte, and T lymphocyte. The quantitative parameters of WBC are directly obtained from the tomograms. Results. Our results show >99% accuracy for the binary classification of myeloids and lymphoids and >96% accuracy for the four-type classification of B and T lymphocytes, monocyte, and myelocytes. The feature learning capability of our approach is visualized via an unsupervised dimension reduction technique. Conclusion. We envision that the proposed cell classification framework can be easily integrated into existing blood cell investigation workflows, providing cost-effective and rapid diagnosis for hematologic malignancy.

Image3C, a multimodal image-based and label-independent integrative method for single-cell analysis

Image-based cell classification has become a common tool to identify phenotypic changes in cell populations. However, this methodology is limited to organisms possessing well-characterized species-specific reagents (e.g., antibodies) that allow cell identification, clustering, and convolutional neural network (CNN) training. In the absence of such reagents, the power of image-based classification has remained mostly off-limits to many research organisms. We have developed an image-based classification methodology we named Image3C (Image-Cytometry Cell Classification) that does not require species-specific reagents nor pre-existing knowledge about the sample. Image3C combines image-based flow cytometry with an unbiased, high-throughput cell clustering pipeline and CNN integration. Image3C exploits intrinsic cellular features and non-species-specific dyes to perform de novo cell composition analysis and detect changes between different conditions. Therefore, Image3C expands the use of image-based analyses of cell population composition to research organisms in which detailed cellular phenotypes are unknown or for which species-specific reagents are not available.

Deepometry, a framework for applying supervised and weakly supervised deep learning to imaging cytometry

Deep learning offers the potential to extract more than meets the eye from images captured by imaging flow cytometry. This protocol describes the application of deep learning to single-cell images to perform supervised cell classification and weakly supervised learning, using example data from an experiment exploring red blood cell morphology. We describe how to acquire and transform suitable input data as well as the steps required for deep learning training and inference using an open-source web-based application. All steps of the protocol are provided as open-source Python as well as MATLAB runtime scripts, through both command-line and graphic user interfaces. The protocol enables a flexible and friendly environment for morphological phenotyping using supervised and weakly supervised learning and the subsequent exploration of the deep learning features using multi-dimensional visualization tools. The protocol requires 40 h when training from scratch and 1 h when using a pre-trained model.

AIDeveloper: Deep Learning Image Classification in Life Science and Beyond

Artificial intelligence (AI)-based image analysis has increased drastically in recent years. However, all applications use individual solutions, highly specialized for a particular task. Here, an easy-to-use, adaptable, and open source software, called AIDeveloper (AID) to train neural nets (NN) for image classification without the need for programming is presented. AID provides a variety of NN-architectures, allowing to apply trained models on new data, obtain performance metrics, and export final models to different formats. AID is benchmarked on large image datasets (CIFAR-10 and Fashion-MNIST). Furthermore, models are trained to distinguish areas of differentiated stem cells in images of cell culture. A conventional blood cell count and a blood count obtained using an NN are compared, trained on >1.2 million images, and demonstrated how AID can be used for label-free classification of B- and T-cells. All models are generated by non-programmers on generic computers, allowing for an interdisciplinary use.

Cell morphology-based machine learning models for human cell state classification

Herein, we implement and access machine learning architectures to ascertain models that differentiate healthy from apoptotic cells using exclusively forward (FSC) and side (SSC) scatter flow cytometry information. To generate training data, colorectal cancer HCT116 cells were subjected to miR-34a treatment and then classified using a conventional Annexin V/propidium iodide (PI)-staining assay. The apoptotic cells were defined as Annexin V-positive cells, which include early and late apoptotic cells, necrotic cells, as well as other dying or dead cells. In addition to fluorescent signal, we collected cell size and granularity information from the FSC and SSC parameters. Both parameters are subdivided into area, height, and width, thus providing a total of six numerical features that informed and trained our models. A collection of logistical regression, random forest, k-nearest neighbor, multilayer perceptron, and support vector machine was trained and tested for classification performance in predicting cell states using only the six aforementioned numerical features. Out of 1046 candidate models, a multilayer perceptron was chosen with 0.91 live precision, 0.93 live recall, 0.92 live f value and 0.97 live area under the ROC curve when applied on standardized data. We discuss and highlight differences in classifier performance and compare the results to the standard practice of forward and side scatter gating, typically performed to select cells based on size and/or complexity. We demonstrate that our model, a ready-to-use module for any flow cytometry-based analysis, can provide automated, reliable, and stain-free classification of healthy and apoptotic cells using exclusively size and granularity information.

Attention-Aware Residual Network Based Manifold Learning for White Blood Cells Classification

The classification of six types of white blood cells (WBCs) is considered essential for leukemia diagnosis, while the classification is labor-intensive and strict with the clinical experience. To relieve the complicated process with an efficient and automatic method, we propose the Attention-aware Residual Network based Manifold Learning model (ARML) to classify WBCs. The proposed ARML model leverages the adaptive attention-aware residual learning to exploit the category-relevant image-level features and strengthen the first-order feature representation ability. To learn more discriminatory information than the first-order ones, the second-order features are characterized. Afterwards, ARML encodes both the first- and second-order features with Gaussian embedding into the Riemannian manifold to learn the underlying non-linear structure of the features for classification. ARML can be trained in an end-to-end fashion, and the learnable parameters are iteratively optimized. 10800 WBCs images (1800 images for each type) is collected, 9000 images and five-fold cross-validation are used for training and validation of the model, while additional 1800 images for testing. The results show that ARML achieving average classification accuracy of 0.953 outperforms other state-of-the-art methods with fewer trainable parameters. In the ablation study, ARML achieves improved accuracy against its three variants: without manifold learning (AR), without attention-aware learning (RML), and AR without attention-aware learning. The t-SNE results illustrate that ARML has learned more distinguishable features than the comparison methods, which benefits the WBCs classification. ARML provides a clinically feasible WBCs classification solution for leukemia diagnose with an efficient manner.

Classification of white blood cells using weighted optimized deformable convolutional neural networks

BACKGROUND

Machine learning (ML) algorithms have been widely used in the classification of white blood cells (WBCs). However, the performance of ML algorithms still needs to be addressed for being short of gold standard data sets, and even the implementation of the proposed algorithms.

METHODS

In this study, the method of two-module weighted optimized deformable convolutional neural networks (TWO-DCNN) was proposed for WBC classification. Our algorithm is characterized as two-module transfer learning and deformable convolutional (DC) layers for the betterment of robustness. To validate the performance, our method was compared with classical MLs of VGG16, VGG19, Inception-V3, ResNet-50, support vector machine (SVM), multilayer perceptron (MLP), decision tree (DT) and random forest (RF) on our undisclosed WBC data set and public BCCD data set.

RESULTS

TWO-DCNN achieved the best performance with the precisions (PREs) of 95.7%, 94.5% and 91.6%, recalls (RECs) of 95.7%, 94.5% and 91.6%, F1-scores (F1s) of 95.7%, 94.5% and 91.6%, area under curves (AUCs) of 0.98, 0.97 and 0.95 for low-resolution and noisy undisclosed data sets, BCCD data set, respectively.

CONCLUSIONS

With accurate feature extraction and optimized network weights, the proposed TWO-DCNN showed the best performance in WBC classification for low-resolution and noisy data sets. It could be used as an alternative method for clinical applications.

Deep-Learning Based Label-Free Classification of Activated and Inactivated Neutrophils for Rapid Immune State Monitoring

The differential count of white blood cells (WBCs) is one widely used approach to assess the status of a patient’s immune system. Currently, the main methods of differential WBC counting are manual counting and automatic instrument analysis with labeling preprocessing. But these two methods are complicated to operate and may interfere with the physiological states of cells. Therefore, we propose a deep learning-based method to perform label-free classification of three types of WBCs based on their morphologies to judge the activated or inactivated neutrophils. Over 90% accuracy was finally achieved by a pre-trained fine-tuning Resnet-50 network. This deep learning-based method for label-free WBC classification can tackle the problem of complex instrumental operation and interference of fluorescent labeling to the physiological states of the cells, which is promising for future point-of-care applications.

Touchable cell biophysics property recognition platforms enable multifunctional blood smart health care

As a crucial biophysical property, red blood cell (RBC) deformability is pathologically altered in numerous disease states, and biochemical and structural changes occur over time in stored samples of otherwise normal RBCs. However, there is still a gap in applying it further to point-of-care blood devices due to the large external equipment (high-resolution microscope and microfluidic pump), associated operational difficulties, and professional analysis. Herein, we revolutionarily propose a smart optofluidic system to provide a differential diagnosis for blood testing via precise cell biophysics property recognition both mechanically and morphologically. Deformation of the RBC population is caused by pressing the hydrogel via an integrated mechanical transfer device. The biophysical properties of the cell population are obtained by the designed smartphone algorithm. Artificial intelligence-based modeling of cell biophysics properties related to blood diseases and quality was developed for online testing. We currently achieve 100% diagnostic accuracy for five typical clinical blood diseases (90 megaloblastic anemia, 78 myelofibrosis, 84 iron deficiency anemia, 48 thrombotic thrombocytopenic purpura, and 48 thalassemias) via real-world prospective implementation; furthermore, personalized blood quality (for transfusion in cardiac surgery) monitoring is achieved with an accuracy of 96.9%. This work suggests a potential basis for next-generation blood smart health care devices.

Classification of the mechanism of toxicity as applied to human cell line ECV304

The objective of this study was to identify the pattern of cytotoxicity testing of the human cell line ECV304 using three techniques of an ensemble learning algorithm (bagging, boosting and stacking). The study of cell morphology of ECV304 cell line was conducted using impedimetric measurement. Three types of toxins were applied to the ECV304 cell line namely 1 mM hydrogen peroxide (H₂O₂), 5% dimethyl sulfoxide and 10 μg Saponin. The measurement was conducted using electrodes and lock-in amplifier to detect impedance changes during cytotoxicity testing within a frequency range 200 and 830 kHz. The results were analysed, processed and extracted using detrended fluctuation analysis to obtain characteristics and features of the cells when exposed to the each of the toxins. Three ensemble algorithms applied showed slightly different results on the performance for classifying the data set from the feature extraction that was performed. However, the results show that the cell reaction to the toxins could be classified.

Real-Time Stain-Free Classification of Cancer Cells and Blood Cells Using Interferometric Phase Microscopy and Machine Learning

We present a method for real-time visualization and automatic processing for detection and classification of untreated cancer cells in blood during stain-free imaging flow cytometry using digital holographic microscopy and machine learning in throughput of 15 cells per second. As a preliminary model for circulating tumor cells in the blood, following an initial label-free rapid enrichment stage based on the cell size, we applied our holographic imaging approach, providing the quantitative optical thickness profiles of the cells during flow. We automatically classified primary and metastatic colon cancer cells, where the two types of cancer cells were isolated from the same individual, as well as four types of blood cells. We used low-coherence off-axis interferometric phase microscopy and a microfluidic channel to image cells during flow quantitatively. The acquired images were processed and classified based on their morphology and quantitative phase features during the cell flow. We achieved high accuracy of 92.56% for distinguishing between the cells, enabling further automatic enrichment and cancer-cell grading from blood. © 2020 International Society for Advancement of Cytometry.

AI on a chip

Artificial intelligence (AI) has dramatically changed the landscape of science, industry, defence, and medicine in the last several years. Supported by considerably enhanced computational power and cloud storage, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to diverse real-life applications such as drug design, material discovery, speech recognition, self-driving cars, advertising, finance, medical imaging, and astronomical observation, where AI-produced outcomes have been proven to be comparable or even superior to the performance of human experts. In these applications, what is essentially important for the development of AI is the data needed for machine learning. Despite its prominent importance, the very first process of the AI development, namely data collection and data preparation, is typically the most laborious task and is often a limiting factor of constructing functional AI algorithms. Lab-on-a-chip technology, in particular microfluidics, is a powerful platform for both the construction and implementation of AI in a large-scale, cost-effective, high-throughput, automated, and multiplexed manner, thereby overcoming the above bottleneck. On this platform, high-throughput imaging is a critical tool as it can generate high-content information (e.g., size, shape, structure, composition, interaction) of objects on a large scale. High-throughput imaging can also be paired with sorting and DNA/RNA sequencing to conduct a massive survey of phenotype-genotype relations whose data is too complex to analyze with traditional computational tools, but is analyzable with the power of AI. In addition to its function as a data provider, lab-on-a-chip technology can also be employed to implement the developed AI for accurate identification, characterization, classification, and prediction of objects in mixed, heterogeneous, or unknown samples. In this review article, motivated by the excellent synergy between AI and lab-on-a-chip technology, we outline fundamental elements, recent advances, future challenges, and emerging opportunities of AI with lab-on-a-chip technology or "AI on a chip" for short.

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single-cell images from six donors, we evaluate classifiers ranging from traditional models that use previously-extracted image features to convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter-lab/t-cell-classification.

Blood rheology biomarkers in sickle cell disease

Sickle cell disease (SCD) is the most common inherited blood disorder, affecting approximately 100,000 patients in the U.S. and millions more worldwide. Patients with SCD experience a wide range of clinical complications, including frequent pain crises, stroke, and early mortality, all originating from a single-point mutation in the β-globin subunit. The RBC changes resulting from the sickle mutation lead to a host of rheological abnormalities that diminish microvascular blood flow, and produce severe anemia due to RBC hemolysis, and ischemia from vaso-occlusion initiated by sticky, rigid sickle RBCs. While the pathophysiology and mechanisms of SCD have been investigated for many years, therapies to treat the disease are limited. In addition to RBC transfusion, there are only two US Food and Drug Administration (FDA)-approved drugs to ameliorate SCD complications: hydroxyurea (HU) and L-glutamine (Endari™). The only curative therapy currently available is allogeneic hematopoietic stem cell transplantation (HSCT), which is generally reserved for individuals with a matched related donor, comprising only 10–15% of the total SCD population. Potentially curative advanced gene therapy approaches for SCD are under investigation in ongoing clinical trials. The ultimate goal of any curative treatment should be to repair the hemorheological abnormalities caused by SCD, and thus normalize blood flow and prevent clinical complications. Our mini-review highlights a set of key hemorheological biomarkers (and the current and emerging technologies used to measure them) that may be used to guide the development of novel curative and palliative therapies for SCD, and functionally assess outcomes.

Impact statement

Severe impairment of blood rheology is the hallmark of SCD pathophysiology, and one of the key factors predisposing SCD patients to pain crises, organ damage, and early mortality. As novel therapies emerge to treat or cure SCD, it is crucial that these treatments are functionally evaluated for their effect on blood rheology. This review describes a comprehensive panel of rheological biomarkers, their clinical uses, and the technologies used to obtain them. The described technologies can produce highly sensitive measurements of the ability of current treatments to improve blood rheology of SCD patients. The goal of curative therapies should be to achieve blood rheology biomarkers measurements in the range of sickle cell trait individuals (HbAS). The use of the panel of rheological biomarkers proposed in this review could significantly accelerate the development, optimization, and clinical translation of novel therapies for SCD.

CD4+ versus CD8+ T-lymphocyte identification in an integrated microfluidic chip using light scattering and machine learning

T lymphocytes are a group of cells representing the main effectors of human adaptive immunity. Characterization of the most representative T-lymphocyte subclasses, CD4+ and CD8+, is challenging, but has a significant impact on clinical decisions. Up to now, T lymphocytes have been identified by quite complex cytometric assays, which are based on antibody labeling. However, a label-free approach based on pure biophysical evaluation at a single-cell level could enable the ability to distinguish between these subclasses. Here, we report a light-scattering approach, supported by accurate data mining, to evaluate cell biophysical properties on an integrated microfluidic chip. In order to perform single-cell optical analysis in viscoelastic fluids, such a chip is composed of mixing, alignment, readout and collection sections. In particular, we measured the cell dimensions, the refractive index of the cell nucleus, the refractive index of the cytosol, and the nucleus-to-cytosol ratio. Combining measurement of biophysical properties and machine learning allows us to both distinguish and count human CD4+ and CD8+ cells with an accuracy of 79%. An enhanced identification accuracy of 88% can be achieved by stimulating the cells with a selective anti-apoptotic protein, which results in increased biophysical differences between CD4+ and CD8+ cells. This approach has been successfully validated by analysis of samples that recapitulate physiological and pathological scenarios (CD4+/CD8+ ratios). The results are encouraging for the possible application of our approach in hematological clinical routines, as well as in diagnosis and follow-up of specific pathologies, such as human immunodeficiency virus (HIV) progression.

Looking beyond the hype: Applied AI and machine learning in translational medicine

Big data problems are becoming more prevalent for laboratory scientists who look to make clinical impact. A large part of this is due to increased computing power, in parallel with new technologies for high quality data generation. Both new and old techniques of artificial intelligence (AI) and machine learning (ML) can now help increase the success of translational studies in three areas: drug discovery, imaging, and genomic medicine. However, ML technologies do not come without their limitations and shortcomings. Current technical limitations and other limitations including governance, reproducibility, and interpretation will be discussed in this article. Overcoming these limitations will enable ML methods to be more powerful for discovery and reduce ambiguity within translational medicine, allowing data-informed decision-making to deliver the next generation of diagnostics and therapeutics to patients quicker, at lowered costs, and at scale.