Ghost cytometry

A multi-kingdom genetic barcoding system for precise clone isolation

Cell-tagging strategies with DNA barcodes have enabled the analysis of clone size dynamics and clone-restricted transcriptomic landscapes in heterogeneous populations. However, isolating a target clone that displays a specific phenotype from a complex population remains challenging. Here we present a multi-kingdom genetic barcoding system, CloneSelect, which enables a target cell clone to be triggered to express a reporter gene for isolation through barcode-specific CRISPR base editing. In CloneSelect, cells are first stably tagged with DNA barcodes and propagated so that their subpopulation can be subjected to a given experiment. A clone that shows a phenotype or genotype of interest at a given time can then be isolated from the initial or subsequent cell pools stored during the experiment using CRISPR base editing. CloneSelect is scalable and compatible with single-cell RNA sequencing. We demonstrate the versatility of CloneSelect in human embryonic kidney 293T cells, mouse embryonic stem cells, human pluripotent stem cells, yeast cells and bacterial cells.

Deep compressed multichannel adaptive optics scanning light ophthalmoscope

Adaptive optics scanning light ophthalmoscopy (AOSLO) reveals individual retinal cells and their function, microvasculature, and micropathologies in vivo. As compared to the single-channel offset pinhole and two-channel split-detector nonconfocal AOSLO designs, by providing multidirectional imaging capabilities, a recent generation of multidetector and (multi-)offset aperture AOSLO modalities has been demonstrated to provide critical information about retinal microstructures. However, increasing detection channels requires expensive optical components and/or critically increases imaging time. To address this issue, we present an innovative combination of machine learning and optics as an integrated technology to compressively capture 12 nonconfocal channel AOSLO images simultaneously. Imaging of healthy participants and diseased subjects using the proposed deep compressed multichannel AOSLO showed enhanced visualization of rods, cones, and mural cells with over an order-of-magnitude improvement in imaging speed as compared to conventional offset aperture imaging. To facilitate the adaptation and integration with other in vivo microscopy systems, we made optical design, acquisition, and computational reconstruction codes open source.

BCR::ABL1-induced mitochondrial morphological alterations as a potential clinical biomarker in chronic myeloid leukemia

The BCR::ABL1 oncogene plays a crucial role in the development of chronic myeloid leukemia (CML). Previous studies have investigated the involvement of mitochondrial dynamics in various cancers, revealing potential therapeutic strategies. However, the impact of BCR::ABL1 on mitochondrial dynamics remains unclear. In this study, we demonstrated that BCR::ABL1 is sufficient to induce excessive mitochondrial fragmentation by activating dynamin-related protein (DRP)1 through the mitogen-activated protein kinase (MAPK) pathway. Leukocytes obtained from patients with CML and the BCR::ABL1-positive cell lines exhibited increased mitochondrial fragmentation compared to leukocytes obtained from healthy donors and BCR::ABL1-negative cells. Furthermore, the analysis of BCR::ABL1-transduced cells showed increased phosphorylation of DRP1 at serine 616 and extracellular signal-regulated kinase (ERK) 1/2. Moreover, the inhibition of DRP1 and upstream mitogen-activated extracellular signal-regulated kinase (MEK) 1/2 suppressed mitochondrial fragmentation. Strikingly, DRP1 inhibition effectively reduced the viability of BCR::ABL1-positive cells and induced necrotic cell death. Additionally, a label-free artificial intelligence-driven flow cytometry successfully identified not only the BCR::ABL1-transduced cells but also peripheral leukocytes from CML patients by assessing mitochondrial morphological alterations. These findings suggested the crucial role of BCR::ABL1-induced mitochondrial fragmentation in driving BCR::ABL1-positive cell proliferation, and the potential use of mitochondrial morphological alterations as a clinical biomarker for the label-free detection of CML cells.

Cytometry at the Intersection of Metabolism and Epigenetics in Lymphocyte Dynamics

Landmark studies at the turn of the century revealed metabolic reprogramming as a driving force for lymphocyte differentiation and function. In addition to metabolic changes, differentiating lymphocytes must remodel their epigenetic landscape to properly rewire their gene expression. Recent discoveries have shown that metabolic shifts can shape the fate of lymphocytes by altering their epigenetic state, bringing together these two areas of inquiry. The ongoing evolution of high-dimensional cytometry has enabled increasingly comprehensive analyses of metabolic and epigenetic landscapes in lymphocytes that transcend the technical limitations of the past. Here, we review recent insights into the interplay between metabolism and epigenetics in lymphocytes and how its dysregulation can lead to immunological dysfunction and disease. We also discuss the latest technical advances in cytometry that have enabled these discoveries and that we anticipate will advance future work in this area.

Optical sorting: past, present and future

Optical sorting combines optical tweezers with diverse techniques, including optical spectrum, artificial intelligence (AI) and immunoassay, to endow unprecedented capabilities in particle sorting. In comparison to other methods such as microfluidics, acoustics and electrophoresis, optical sorting offers appreciable advantages in nanoscale precision, high resolution, non-invasiveness, and is becoming increasingly indispensable in fields of biophysics, chemistry, and materials science. This review aims to offer a comprehensive overview of the history, development, and perspectives of various optical sorting techniques, categorised as passive and active sorting methods. To begin, we elucidate the fundamental physics and attributes of both conventional and exotic optical forces. We then explore sorting capabilities of active optical sorting, which fuses optical tweezers with a diversity of techniques, including Raman spectroscopy and machine learning. Afterwards, we reveal the essential roles played by deterministic light fields, configured with lens systems or metasurfaces, in the passive sorting of particles based on their varying sizes and shapes, sorting resolutions and speeds. We conclude with our vision of the most promising and futuristic directions, including AI-facilitated ultrafast and bio-morphology-selective sorting. It can be envisioned that optical sorting will inevitably become a revolutionary tool in scientific research and practical biomedical applications.

Microfluidics for morpholomics and spatial omics applications

Creative designs, precise fluidic manipulation, and automation have supported the development of microfluidics for single-cell applications. Together with the advancements in detection technologies and artificial intelligence (AI), microfluidic-assisted platforms have been increasingly used for new modalities of single-cell investigations and in spatial omics applications. This review explores the use of microfluidic technologies for morpholomics and spatial omics with a focus on single-cell and tissue characterization. We emphasize how various fluid dynamic principles and unique design integrations enable highly precise fluid manipulation, enhancing sample handling in morpholomics. Additionally, we examine the use of microfluidics-assisted spatial barcoding with micrometer resolutions for the spatial profiling of tissue specimens. Finally, we discuss how microfluidics can serve as a bridge for integrating multiple unique fields in omics research and outline key challenges that these technologies may face in practical translation.

Imaging flow cytometry reveals divergent mitochondrial phenotypes in mitochondrial disease patients

Traditional classification by clinical phenotype or oxidative phosphorylation (OXPHOS) complex deficiencies often fails to clarify complex genotype-phenotype correlations in mitochondrial disease. A multimodal functional assessment may better reveal underlying disease patterns. Using imaging flow cytometry (IFC), we evaluated mitochondrial fragmentation, swelling, membrane potential, reactive oxygen species (ROS) production, and mitochondrial mass in fibroblasts from 31 mitochondrial disease patients. Significant changes were observed in 97% of patients, forming two overarching groups with distinct responses to mitochondrial pathology. One group displayed low-to-normal membrane potential, indicating a hypometabolic state, while the other showed elevated membrane potential and swelling, suggesting a hypermetabolic state. Literature analysis linked these clusters to complex I stability defects (hypometabolic) and proton pumping activity (hypermetabolic). Thus, our IFC-based platform offers a novel approach to identify disease-specific patterns through functional responses, supporting improved diagnostic and therapeutic strategies.

Environmental community transcriptomics: strategies and struggles

Transcriptomics is the study of RNA transcripts, the portion of the genome that is transcribed, in a specific cell, tissue, or organism. Transcriptomics provides insight into gene expression patterns, regulation, and the underlying mechanisms of cellular processes. Community transcriptomics takes this a step further by studying the RNA transcripts from environmental assemblies of organisms, with the intention of better understanding the interactions between members of the community. Community transcriptomics requires successful extraction of RNA from a diverse set of organisms and subsequent analysis via mapping those reads to a reference genome or de novo assembly of the reads. Both, extraction protocols and the analysis steps can pose hurdles for community transcriptomics. This review covers advances in transcriptomic techniques and assesses the viability of applying them to community transcriptomics.

Deep Learning-Assisted Label-Free Parallel Cell Sorting with Digital Microfluidics

Sorting specific cells from heterogeneous samples is important for research and clinical applications. In this work, a novel label-free cell sorting method is presented that integrates deep learning image recognition with microfluidic manipulation to differentiate cells based on morphology. Using an Active-Matrix Digital Microfluidics (AM-DMF) platform, the YOLOv8 object detection model ensures precise droplet classification, and the Safe Interval Path Planning algorithm manages multi-target, collision-free droplet path planning. Simulations and experiments revealed that detection model precision, concentration ratios, and sorting cycles significantly affect recovery rates and purity. With HeLa cells and polystyrene beads as samples, the method achieved 98.5% sorting precision, 96.49% purity, and an 80% recovery over three cycles. After a series of experimental validations, this method can also be used to sort HeLa cells from red blood cells, cancer cells from white blood cells (represented by HeLa and Jurkat cells), and differentiate white blood cell subtypes (represented by HL-60 cells and Jurkat cells). Cells sorted using this method can be lysed directly on chip within their hosting droplets, ensuring minimal sample loss and suitability for downstream bioanalysis. This innovative AM-DMF cell sorting technique holds significant potential to advance diagnostics, therapeutics, and fundamental research in cell biology.

Recent Technologies on 2D and 3D Imaging Flow Cytometry

Imaging flow cytometry is a technology that performs microscopy image analysis of cells within flow cytometry and allows high-throughput, high-content cell analysis based on their intracellular molecular distribution and/or cellular morphology. While the technology has been available for a couple of decades, it has recently gained significant attention as technical limitations for higher throughput, sorting capability, and additional imaging dimensions have been overcome with various approaches. These evolutions have enabled imaging flow cytometry to offer a variety of solutions for life science and medicine that are not possible with conventional flow cytometry or microscopy-based screening. It is anticipated that the extent of applications will expand in the upcoming years as the technology becomes more accessible through dissemination. In this review, we will cover the technical advances that have led to this new generation of imaging flow cytometry, focusing on the advantages and limitations of each technique.

Systematic data analysis pipeline for quantitative morphological cell phenotyping

Quantitative morphological phenotyping (QMP) is an image-based method used to capture morphological features at both the cellular and population level. Its interdisciplinary nature, spanning from data collection to result analysis and interpretation, can lead to uncertainties, particularly among those new to this actively growing field. High analytical specificity for a typical QMP is achieved through sophisticated approaches that can leverage subtle cellular morphological changes. Here, we outline a systematic workflow to refine the QMP methodology. For a practical review, we describe the main steps of a typical QMP; in each step, we discuss the available methods, their applications, advantages, and disadvantages, along with the R functions and packages for easy implementation. This review does not cover theoretical backgrounds, but provides several references for interested researchers. It aims to broaden the horizons for future phenome studies and demonstrate how to exploit years of endeavors to achieve more with less.

Cell sorting based on pulse shapes from angle resolved detection of scattered light

Flow cytometry is a key technology for the analysis and sorting of cells or particles at high throughput. Conventional and current flow cytometry is primarily based on fluorescent stains to detect the cells of interest. However, such stains also have disadvantages, as their effect on cells must be carefully tested to avoid effects on the results of the experiments. Alternative approaches using imaging or other label-free techniques often require highly sophisticated setups, are commonly limited in resolution, and produce challenging amounts of data. Our technology exploits scattered light instead. The custom-built flow cytometry setup comprises a fiber array in forward scatter detection for angular resolution and captures the whole pulse shape with advanced signal processing. Thereby this setup enables cell analysis and sorting purely based on scattered light signals without the need for fluorescent labels. We demonstrate the feasibility of this cell sorting technology by sorting cell lines for their cell cycle stages based on scattered light. Furthermore, we demonstrate the ability to classify human peripheral blood T- and B-cell subsets.

Machine learning assisted microfluidics dual fluorescence flow cytometry for detecting bladder tumor cells based on morphological characteristic parameters

BACKGROUND

Bladder cancer (BC) is the most common malignant tumor and has become a major public health problem, leading the causes of death worldwide. The detection of BC cells is of great significance for clinical diagnosis and disease treatment. Urinary cytology based liquid biopsy remains high specificity for early diagnosis of BC, however, it still requires microscopy examination which heavily relies on manual operations. It is imperative to investigate the potential of automated and indiscriminate cell differentiation technology to enhance the sensitivity and efficiency of urine cytology.

RESULTS

Here, we developed a machine learning algorithm empowered dual-fluorescence flow cytometry platform (μ-FCM) for urinary cytology analysis. A phenotype characteristic parameter (CP) which correlated with the size of the cell and nucleus was defined to achieve the differentiation of the BC cells and uroepithelial cells with high throughput and high accuracy. Based on CP analysis, SV-HUC-1 cells were almost differentiated from EJ cells and effectively reduced the overlap with 5637 cells. To further differentiate SV-HUC-1 cells and 5637 cells, support vector machine (SVM) machine learning algorithm was optimized to assist data analysis with the highest accuracies of 84.7 % for cell differentiation including the specificity of 91.0 % and the sensitivity of 75.0 %. Furthermore, the false positive rate (FPR) compensation enabled the detection rates of rare BC cells predicted by the well-trained SVM model were close to the true proportions with the recognition error in 0.4 % for the tumor cells.

SIGNIFICANCE

As a proof of concept, the developed μ-FCM system successfully demonstrates the capacity to identify the distribution of exfoliated cells in real urine samples. This system underscores the significance of integrating AI with microfluidics to perform high-throughput phenotyping of exfoliated cells, offering a pathway toward scalable, efficient, and automatic microfluidic systems in the fields of both biosensing and in vitro diagnosis of BC.

Accessible high-speed image-activated cell sorting

Over the past six decades, fluorescence-activated cell sorting (FACS) has become an essential technology for basic and clinical research by enabling the isolation of cells of interest in high throughput. Recent technological advancements have started a new era of flow cytometry. By combining the spatial resolution of microscopy with high-speed cell sorting, new instruments allow cell sorting based on simple image-derived parameters or sophisticated image analysis algorithms, thereby greatly expanding the scope of applications. In this review, we discuss the systems that are commercially available or have been described in enough methodological and engineering detail to allow their replication. We summarize their strengths and limitations and highlight applications that have the potential to transform various fields in basic life science research and clinical settings.

Deep Learning-Based Identification of Intraocular Pressure-Associated Genes Influencing Trabecular Meshwork Cell Morphology

Purpose

Genome-wide association studies have recently uncovered many loci associated with variation in intraocular pressure (IOP). Artificial intelligence (AI) can be used to interrogate the effect of specific genetic knockouts on the morphology of trabecular meshwork cells (TMCs) and thus, IOP regulation.

Design

Experimental study.

Subjects

Primary TMCs collected from human donors.

Methods

Sixty-two genes at 55 loci associated with IOP variation were knocked out in primary TMC lines. All cells underwent high-throughput microscopy imaging after being stained with a 5-channel fluorescent cell staining protocol. A convolutional neural network was trained to distinguish between gene knockout and normal control cell images. The area under the receiver operator curve (AUC) metric was used to quantify morphological variation in gene knockouts to identify potential pathological perturbations.

Main Outcome Measures

Degree of morphological variation as measured by deep learning algorithm accuracy of differentiation from normal controls.

Results

Cells where LTBP2 or BCAS3 had been perturbed demonstrated the greatest morphological variation from normal TMCs (AUC 0.851, standard deviation [SD] 0.030; and AUC 0.845, SD 0.020, respectively). Of 7 multigene loci, 5 had statistically significant differences in AUC (P < 0.05) between genes, allowing for pathological gene prioritization. The mitochondrial channel most frequently showed the greatest degree of morphological variation (33.9% of cell lines).

Conclusions

We demonstrate a robust method for functionally interrogating genome-wide association signals using high-throughput microscopy and AI. Genetic variations inducing marked morphological variation can be readily identified, allowing for the gene-based dissection of loci associated with complex traits.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Imaging Flow Cytometry and Convolutional Neural Network-Based Classification Enable Discrimination of Hematopoietic and Leukemic Stem Cells in Acute Myeloid Leukemia

Acute myeloid leukemia (AML) is a heterogenous blood cancer with a dismal prognosis. It emanates from leukemic stem cells (LSCs) arising from the genetic transformation of hematopoietic stem cells (HSCs). LSCs hold prognostic value, but their molecular and immunophenotypic heterogeneity poses challenges: there is no single marker for identifying all LSCs across AML samples. We hypothesized that imaging flow cytometry (IFC) paired with artificial intelligence-driven image analysis could visually distinguish LSCs from HSCs based solely on morphology. Initially, a seven-color IFC panel was employed to immunophenotypically identify LSCs and HSCs in bone marrow samples from five AML patients and ten healthy donors, respectively. Next, we developed convolutional neural network (CNN) models for HSC-LSC discrimination using brightfield (BF), side scatter (SSC), and DNA images. Classification using only BF images achieved 86.96% accuracy, indicating significant morphological differences. Accuracy increased to 93.42% when combining BF with DNA images, highlighting differences in nuclear morphology, although DNA images alone were inadequate for accurate HSC-LSC discrimination. Model development using SSC images revealed minor granularity differences. Performance metrics varied substantially between AML patients, indicating considerable morphologic variations among LSCs. Overall, we demonstrate proof-of-concept results for accurate CNN-based HSC-LSC differentiation, instigating the development of a novel technique within AML monitoring.

Cellular architecture shapes the naïve T cell response

After antigen stimulation, naïve T cells display reproducible population-level responses, which arise from individual T cells pursuing specific differentiation trajectories. However, cell-intrinsic predeterminants controlling these single-cell decisions remain enigmatic. We found that the subcellular architectures of naïve CD8 T cells, defined by the presence (TØ) or absence (TO) of nuclear envelope invaginations, changed with maturation, activation, and differentiation. Upon T cell receptor (TCR) stimulation, naïve TØ cells displayed increased expression of the early-response gene Nr4a1, dependent upon heightened calcium entry. Subsequently, in vitro differentiation revealed that TØ cells generated effector-like cells more so compared with TO cells, which proliferated less and preferentially adopted a memory-precursor phenotype. These data suggest that cellular architecture may be a predeterminant of naïve CD8 T cell fate.

Pooled CRISPR screening of high-content cellular phenotypes using ghost cytometry

Recent advancements in image-based pooled CRISPR screening have facilitated the mapping of diverse genotype-phenotype associations within mammalian cells. However, the rapid enrichment of cells based on morphological information continues to pose a challenge, constraining the capacity for large-scale gene perturbation screening across diverse high-content cellular phenotypes. In this study, we demonstrate the applicability of multimodal ghost cytometry-based cell sorting, including both fluorescent and label-free high-content phenotypes, for rapid pooled CRISPR screening within vast cell populations. Using the high-content cell sorter operating in fluorescence mode, we successfully executed kinase-specific CRISPR screening targeting genes influencing the nuclear translocation of RelA. Furthermore, using the multiparametric, label-free mode, we performed large-scale screening to identify genes involved in macrophage polarization. Notably, the label-free platform can enrich target phenotypes without requiring invasive staining, preserving untouched cells for downstream assays and expanding the potential for screening cellular phenotypes even when suitable markers are absent.

Imaging Flow Cytometry: Development, Present Applications, and Future Challenges

Imaging flow cytometry (ImFC) represents a significant technological advancement in the field of cytometry, effectively merging the high-throughput capabilities of flow analysis with the detailed imaging characteristics of microscopy. In our comprehensive review, we adopt a historical perspective to chart the development of ImFC, highlighting its origins and current state of the art and forecasting potential future advancements. The genesis of ImFC stemmed from merging the hydraulic system of a flow cytometer with advanced camera technology. This synergistic coupling facilitates the morphological analysis of cell populations at a high-throughput scale, effectively evolving the landscape of cytometry. Nevertheless, ImFC's implementation has encountered hurdles, particularly in developing software capable of managing its sophisticated data acquisition and analysis needs. The scale and complexity of the data generated by ImFC necessitate the creation of novel analytical tools that can effectively manage and interpret these data, thus allowing us to unlock the full potential of ImFC. Notably, artificial intelligence (AI) algorithms have begun to be applied to ImFC, offering promise for enhancing its analytical capabilities. The adaptability and learning capacity of AI may prove to be essential in knowledge mining from the high-dimensional data produced by ImFC, potentially enabling more accurate analyses. Looking forward, we project that ImFC may become an indispensable tool, not only in research laboratories, but also in clinical settings. Given the unique combination of high-throughput cytometry and detailed imaging offered by ImFC, we foresee a critical role for this technology in the next generation of scientific research and diagnostics. As such, we encourage both current and future scientists to consider the integration of ImFC as an addition to their research toolkit and clinical diagnostic routine.

Label-free cell detection of acute leukemia using ghost cytometry

Early diagnosis and prompt initiation of appropriate treatment are critical for improving the prognosis of acute leukemia. Acute leukemia is diagnosed by microscopic morphological examination of bone marrow smears and flow cytometric immunophenotyping of bone marrow cells stained with fluorophore-conjugated antibodies. However, these diagnostic processes require trained professionals and are time and resource-intensive. Here, we present a novel diagnostic approach using ghost cytometry, a recently developed high-content flow cytometric approach, which enables machine vision-based, stain-free, high-speed analysis of cells, leveraging their detailed morphological information. We demonstrate that ghost cytometry can detect leukemic cells from the bone marrow cells of patients diagnosed with acute lymphoblastic leukemia and acute myeloid leukemia without relying on biological staining. The approach presented here holds promise as a precise, simple, swift, and cost-effective diagnostic method for acute leukemia in clinical practice.

Label-free enrichment of human pluripotent stem cell-derived early retinal progenitor cells for cell-based regenerative therapies

Pluripotent stem cell-based therapy for retinal degenerative diseases is a promising approach to restoring visual function. A clinical study using retinal organoid (RO) sheets was recently conducted in patients with retinitis pigmentosa. However, the graft preparation currently requires advanced skills to identify and excise suitable segments from the transplantable area of the limited number of suitable ROs. This remains a challenge for consistent clinical implementations. Herein, we enabled the enrichment of wild-type (non-reporter) retinal progenitor cells (RPCs) from dissociated ROs using a label-free ghost cytometry (LF-GC)-based sorting system, where a machine-based classifier was trained in advance with another RPC reporter line. The sorted cells reproducibly formed retinal spheroids large enough for transplantation and developed mature photoreceptors in the retinal degeneration rats. This method of enriching early RPCs with no specific surface antigens and without any reporters or chemical labeling is promising for robust preparation of graft tissues during cell-based therapy.

Optical cryptosystem based on computational ghost imaging and nonlinear authentication

We propose an optical encryption system that combines computational ghost imaging (CGI) with image authentication to enhance security. In this scheme, Hadamard patterns are projected onto the secret images, while their reflected light intensities are captured using a bucket detector (BD). To further strengthen the security of the collected secret data, we encrypt it as a series of binary matrices serving as ciphertext. During the authentication key generation, these encoded binary matrices serve as illumination patterns in the CGI system for a non-secret image, which is used as a reference image for authentication. The data captured by the BD is then binarized to generate the authentication key. Upon successful authentication, the receiver obtains the decryption keys. This method achieves both data compression for secret images and enhanced security during information transmission. We validate the feasibility of this method through computer simulations and optical experiments.

Robust compressed ghost imaging against environmental influence factors

Ghost imaging based on sparse sampling is sensitive to the environmental influence factors frequently encountered in practice, such as instrumental drift and ambient light change, which could cause degradation of image quality. In this manuscript, we report a robust compressed sensing technique which could effectively reduce the influence of measurement errors on image quality. For demonstration purposes, we implement the proposed technique to ghost imaging, namely differential compressed sensing ghost imaging (DCSGI). By applying differential measurements n times, the first n Taylor expansion polynomials of the error could be eliminated in n-order DCSGI. It has been verified theoretically and experimentally that DCSGI works well with typical errors which exists in the realities of ghost imaging applications, while the conventional approach can hardly. In addition, the proposed technique may also replace conventional compressed sensing in other applications for anti-interference high-quality reconstruction.

Imaging privacy threats from an ambient light sensor

Embedded sensors in smart devices pose privacy risks, often unintentionally leaking user information. We investigate how combining an ambient light sensor with a device display can capture an image of touch interaction without a camera. By displaying a known video sequence, we use the light sensor to capture reflected light intensity variations partially blocked by the touching hand, formulating an inverse problem similar to single-pixel imaging. Because of the sensors' heavy quantization and low sensitivity, we propose an inversion algorithm involving an ℓp-norm dequantizer and a deep denoiser as natural image priors, to reconstruct images from the screen's perspective. We demonstrate touch interactions and eavesdropping hand gestures on an off-the-shelf Android tablet. Despite limitations in resolution and speed, we aim to raise awareness of potential security/privacy threats induced by the combination of passive and active components in smart devices and promote the development of ways to mitigate them.

Image-Enabled Cell Sorting Using the BD CellView Technology

This chapter is an extension of the original publication by Schraivogel et al. (Science 375:315-320, 2022) which described, for the first time, image-enabled and high-speed cell sorting based on the BD CellView technology. It summarizes the technical aspects of the instrument in an easy-to-digest form and provides example-based guidance toward implementation of the CellView-based image cell sorting technology. As an example, it explains how to use the image-enabled cell sorter to analyze the chemically induced fragmentation of the Golgi apparatus in HeLa cells-an experiment that was alluded to in the original publication but was not included in the manuscript due to space constraints. The chemically induced Golgi fragmentation sort illustrates an elegant example of the utility of image-enabled cell sorting as a significant expansion of the single-cell toolbox. It is such a striking phenotype when analyzed with image cytometry but undetectable when using conventional flow cytometry. Described in a straightforward and concise manner, this experiment serves as a standard system assurance for image-based cell sorters.

Real-time quantitative characterization of ion channel activities for automated control of a lipid bilayer system

This paper describes a novel signal processing method to characterize the activity of ion channels on a lipid bilayer system in a real-time and quantitative manner. Lipid bilayer systems, which enable single-channel level recordings of ion channel activities against physiological stimuli in vitro, are gaining attention in various research fields. However, the characterization of ion channel activities has heavily relied on time-consuming analyses after recording, and the inability to return the quantitative results in real time has long been a bottleneck to incorporating the system into practical products. Herein, we report a lipid bilayer system that integrates real-time characterization of ion channel activities and real-time response based on the characterization result. Unlike conventional batch processing, an ion channel signal is divided into short segments and processed during the recording. After optimizing the system to maintain the same characterization accuracy as conventional operation, we demonstrated the usability of the system with two applications. One is quantitative control of a robot based on ion channel signals. The velocity of the robot was controlled every second, which was around tens of times faster than the conventional operation, in proportion to the stimulus intensity estimated from changes in ion channel activities. The other is the automation of data collection and characterization of ion channels. By constantly monitoring and maintaining the functionality of a lipid bilayer, our system enabled continuous recording of ion channels over 2 h without human intervention, and the time of manual labor has been reduced from conventional 3 h to 1 min at a minimum. We believe the accelerated characterization and response in the lipid bilayer systems presented in this work will facilitate the transformation of lipid bilayer technology toward a practical level, finally leading to its industrialization.

Mask-based single-pixel tracking and imaging for moving objects

Tracking and imaging for high-speed moving objects have a wide range of application prospects in many fields, such as transportation and security monitoring. In this paper, the chrome plated masks are designed to carry geometric moment and random binary encoding patterns, combined with single pixel detectors, to achieve real-time tracking and imaging of fast-moving object. By using the geometric moment principle to obtain the motion trajectory of the object, coding sub-patterns and corresponding detection signals are extracted at different positions to reconstruct the image of the object. Multiple optical paths are established to avoid the side effects of motion error, and a dedicated calibration approach is proposed to improve the accuracy of tracking. The feasibility of the method is demonstrated by simulations and experiments. The proposed scheme, which modulates light with static mask instead of spatial light modulator (SLM), improves the speed and spectral range meanwhile reduces the system cost.

COSMOS: a platform for real-time morphology-based, label-free cell sorting using deep learning

Cells are the singular building blocks of life, and a comprehensive understanding of morphology, among other properties, is crucial to the assessment of underlying heterogeneity. We developed Computational Sorting and Mapping of Single Cells (COSMOS), a platform based on Artificial Intelligence (AI) and microfluidics to characterize and sort single cells based on real-time deep learning interpretation of high-resolution brightfield images. Supervised deep learning models were applied to characterize and sort cell lines and dissociated primary tissue based on high-dimensional embedding vectors of morphology without the need for biomarker labels and stains/dyes. We demonstrate COSMOS capabilities with multiple human cell lines and tissue samples. These early results suggest that our neural networks embedding space can capture and recapitulate deep visual characteristics and can be used to efficiently purify unlabeled viable cells with desired morphological traits. Our approach resolves a technical gap in the ability to perform real-time deep learning assessment and sorting of cells based on high-resolution brightfield images.

Machine learning implementation strategy in imaging and impedance flow cytometry

Imaging and impedance flow cytometry is a label-free technique that has shown promise as a potential replacement for standard flow cytometry. This is due to its ability to provide rich information and archive high-throughput analysis. Recently, significant efforts have been made to leverage machine learning for processing the abundant data generated by those techniques, enabling rapid and accurate analysis. Harnessing the power of machine learning, imaging and impedance flow cytometry has demonstrated its capability to address various complex phenotyping scenarios. Herein, we present a comprehensive overview of the detailed strategies for implementing machine learning in imaging and impedance flow cytometry. We initiate the discussion by outlining the commonly employed setup to acquire the data (i.e., image or signal) from the cell. Subsequently, we delve into the necessary processes for extracting features from the acquired image or signal data. Finally, we discuss how these features can be utilized for cell phenotyping through the application of machine learning algorithms. Furthermore, we discuss the existing challenges and provide insights for future perspectives of intelligent imaging and impedance flow cytometry.

Optical trapping with holographically structured light for single-cell studies

A groundbreaking work in 1970 by Arthur Ashkin paved the way for developing various optical trapping techniques. Optical tweezers have become an established method for the manipulation of biological objects, due to their noninvasiveness and precise controllability. Recent innovations are accelerating and now enable single-cell manipulation through holographic light structuring. In this review, we provide an overview of recent advances in optical tweezer techniques for studies at the individual cell level. Our review focuses on holographic optical tweezers that utilize active spatial light modulators to noninvasively manipulate live cells. The versatility of the technology has led to valuable integrations with microscopy, microfluidics, and biotechnological techniques for various single-cell studies. We aim to recapitulate the basic principles of holographic optical tweezers, highlight trends in their biophysical applications, and discuss challenges and future prospects.

Large-scale investigation of zoonotic viruses in the era of high-throughput sequencing

Zoonotic diseases considerably impact public health and socioeconomics. RNA viruses reportedly caused approximately 94% of zoonotic diseases documented from 1990 to 2010, emphasizing the importance of investigating RNA viruses in animals. Furthermore, it has been estimated that hundreds of thousands of animal viruses capable of infecting humans are yet to be discovered, warning against the inadequacy of our understanding of viral diversity. High-throughput sequencing (HTS) has enabled the identification of viral infections with relatively little bias. Viral searches using both symptomatic and asymptomatic animal samples by HTS have revealed hidden viral infections. This review introduces the history of viral searches using HTS, current analytical limitations, and future potentials. We primarily summarize recent research on large-scale investigations on viral infections reusing HTS data from public databases. Furthermore, considering the accumulation of uncultivated viruses, we discuss current studies and challenges for connecting viral sequences to their phenotypes using various approaches: performing data analysis, developing predictive modeling, or implementing high-throughput platforms of virological experiments. We believe that this article provides a future direction in large-scale investigations of potential zoonotic viruses using the HTS technology.

Scalable Functional Assays for the Interpretation of Human Genetic Variation

Scalable sequence-function studies have enabled the systematic analysis and cataloging of hundreds of thousands of coding and noncoding genetic variants in the human genome. This has improved clinical variant interpretation and provided insights into the molecular, biophysical, and cellular effects of genetic variants at an astonishing scale and resolution across the spectrum of allele frequencies. In this review, we explore current applications and prospects for the field and outline the principles underlying scalable functional assay design, with a focus on the study of single-nucleotide coding and noncoding variants.

Upgraded User-Friendly Image-Activated Microfluidic Cell Sorter Using an Optimized and Fast Deep Learning Algorithm

Image-based cell sorting is essential in biological and biomedical research. The sorted cells can be used for downstream analysis to expand our knowledge of cell-to-cell differences. We previously demonstrated a user-friendly image-activated microfluidic cell sorting technique using an optimized and fast deep learning algorithm. Real-time isolation of cells was carried out using this technique with an inverted microscope. In this study, we devised a recently upgraded sorting system. The cell sorting techniques shown on the microscope were implemented as a real system. Several new features were added to make it easier for the users to conduct the real-time sorting of cells or particles. The newly added features are as follows: (1) a high-resolution linear piezo-stage is used to obtain in-focus images of the fast-flowing cells; (2) an LED strobe light was incorporated to minimize the motion blur of fast-flowing cells; and (3) a vertical syringe pump setup was used to prevent the cell sedimentation. The sorting performance of the upgraded system was demonstrated through the real-time sorting of fluorescent polystyrene beads. The sorter achieved a 99.4% sorting purity for 15 μm and 10 μm beads with an average throughput of 22.1 events per second (eps).

Hybrid double-spiral microfluidic chip for RBC-lysis-free enrichment of rare cells from whole blood

Drug selection and treatment monitoring via minimally invasive liquid biopsy using circulating tumor cells (CTCs) are expected to be realized in the near future. For clinical applications of CTCs, simple, high-throughput, single-step CTC isolation from whole blood without red blood cell (RBC) lysis and centrifugation remains a crucial challenge. In this study, we developed a novel cancer cell separation chip, "hybrid double-spiral chip", that involves the serial combination of two different Dean flow fractionation (DFF) separation modes of half and full Dean cycles, which is the hybrid DFF separation mode for ultra-high-throughput blood processing at high precision and size-resolution separation. The chip allows fast processing of 5 mL whole blood within 30 min without RBC lysis and centrifugation. RBC and white blood cell (WBC) depletion rates of over 99.9% and 99%, respectively, were achieved. The average recovery rate of spiked A549 cancer cells was 87% with as low as 200 cells in 5 mL blood. The device can achieve serial reduction in the number of cells from approximately 10¹⁰ cells of whole blood to 10⁸ cells, and subsequently to an order of 10⁶ cells. The developed method can be combined with measurements of all recovered cells using imaging flow cytometry. As proof of concept, CTCs were successfully enriched and enumerated from the blood of metastatic breast cancer patients (N = 10, 1-69 CTCs per 5 mL) and metastatic prostate cancer patients (N = 10, 1-39 CTCs per 5 mL). We believe that the developed method will be beneficial for automated clinical analysis of rare CTCs from whole blood.

Dual-pixel tracking of the fast-moving target based on window complementary modulation

Real-time tracking of fast-moving targets has been utilized in various fields. However, the tracking performance of image-based systems for fast-moving targets is still limited by the huge data throughput and computation. In this study, an image-free target tracking system utilizing a digital micromirror device (DMD) is proposed. The proposed system effectively combines the dual-pixel measurement and window complementary modulation, and the alternating interpolation Kalman filter is implemented to fully use the performance of the DMD and maximize the update rate of the system. The accuracy of the proposed system at the maximum update rate of 22.2 kHz can achieve 0.1 pixels according to the experimental results. Meanwhile, we experimentally demonstrated that the accuracy of the proposed image-free target tracking system is within 0.3 pixels at a maximal velocity of 2 × 10⁴ pixel/s at 22.2 kHz by evaluating the performance of the proposed image-free target tracking system when tracking fast-moving targets with different maximal velocity.

Perovskite Wide-Angle Field-Of-View Camera

Researchers have attempted to create wide-angle field-of-view (FOV) cameras inspired by the structure of the eyes of animals, including fisheye and compound eye cameras. However, realizing wide-angle FOV cameras simultaneously exhibiting low distortion and high spatial resolution remains a significant challenge. In this study, a novel wide-angle FOV camera is developed by combining a single large-area flexible perovskite photodetector (FP-PD) using computational technology. With this camera, the proposed single-photodetector imaging technique can obtain high-spatial-resolution images using only a single detector, and the large-area FP-PD can be bent further to collect light from a wide-angle FOV. The proposed camera demonstrates remarkable features of an extraordinarily tunable wide FOV (greater than 150°), high spatial resolution of 256 × 256 pixels, and low distortion. It is believed that the proposed compatible and extensible camera prototype will promote the development of high-performance versatile FOV cameras.

Single-pixel imaging for edge images using deep neural networks

Edge images are often used in computer vision, cellular morphology, and surveillance cameras, and are sufficient to identify the type of object. Single-pixel imaging (SPI) is a promising technique for wide-wavelength, low-light-level measurements. Conventional SPI-based edge-enhanced techniques have used shifting illumination patterns; however, this increases the number of the illumination patterns. We propose two deep neural networks to obtain SPI-based edge images without shifting illumination patterns. The first network is an end-to-end mapping between the measured intensities and entire edge image. The latter comprises two path convolutional layers for restoring horizontal and vertical edges individually; subsequently, both edges are combined to obtain full edge reconstructions, such as in the Sobel filter.

Real-time single-pixel imaging using a system on a chip field-programmable gate array

Unlike conventional imaging, the single-pixel imaging technique uses a single-element detector, which enables high sensitivity, broad wavelength, and noise robustness imaging. However, it has several challenges, particularly requiring extensive computations for image reconstruction with high image quality. Therefore, high-performance computers are required for real-time reconstruction with higher image quality. In this study, we developed a compact dedicated computer for single-pixel imaging using a system on a chip field-programmable gate array (FPGA), which enables real-time reconstruction at 40 frames per second with an image size of 128 × 128 pixels. An FPGA circuit was implemented with the proposed reconstruction algorithm to obtain higher image quality by introducing encoding mask pattern optimization. The dedicated computer can accelerate the reconstruction 10 times faster than a recent CPU. Because it is very compact compared with typical computers, it can expand the application of single-pixel imaging to the Internet of Things and outdoor applications.

High-resolution ghost imaging through complex scattering media via a temporal correction

In this Letter, we propose high-resolution ghost imaging (GI) through complex scattering media using temporal correction. We provide evidence that the theoretical description about GI based on spatially correlated beams is still incomplete and cannot work in complex scenarios. We complete the description of temporal correction of beam correlations in GI. The optical experiments demonstrate that high-resolution ghost images can always be retrieved by using the rectified temporally corrected beam correlation algorithm even in complex, dynamic, and highly strong scattering environments where conventional GI cannot work. By using the proposed method, the quality of the retrieved ghost images through complex scattering media can be enhanced effectively as the number of realizations increases, which cannot be achieved by conventional GI. The established general framework provides optical insights beyond the current understanding of GI, and the rectified theory and experimental results would represent a key step toward applications of GI over a wide range of free-space wave propagation environments.

Compressive phase object classification using single-pixel digital holography

A single-pixel camera (SPC) is a computational imaging system that obtains compressed signals of a target scene using a single-pixel detector. The compressed signals can be directly used for image classification, thereby bypassing image reconstruction, which is computationally intensive and requires a high measurement rate. Here, we extend this direct inference to phase object classification using single-pixel digital holography (SPDH). Our method obtains compressed measurements of target complex amplitudes using SPDH and trains a classifier using those measurements for phase object classification. Furthermore, we present a joint optimization of the sampling patterns used in SPDH and a classifier to improve classification accuracy. The proposed method successfully classified phase object images of handwritten digits from the MNIST database, which is challenging for SPCs that can only capture intensity images.

Gigahertz-rate random speckle projection for high-speed single-pixel image classification

Imaging techniques based on single-pixel detection, such as ghost imaging, can reconstruct or recognize a target scene from multiple measurements using a sequence of random mask patterns. However, the processing speed is limited by the low rate of the pattern generation. In this study, we propose an ultrafast method for random speckle pattern generation, which has the potential to overcome the limited processing speed. The proposed approach is based on multimode fiber speckles induced by fast optical phase modulation. We experimentally demonstrate dynamic speckle projection with phase modulation at 10 GHz rates, which is five to six orders of magnitude higher than conventional modulation approaches using spatial light modulators. Moreover, we combine the proposed generation approach with a wavelength-division multiplexing technique and apply it for image classification. As a proof-of-concept demonstration, we show that 28×28-pixel images of digits acquired at GHz rates can be accurately classified using a simple neural network. The proposed approach opens a novel pathway for an all-optical image processor.

Semantic ghost imaging based on recurrent-neural-network

Ghost imaging (GI) illuminates an object with a sequence of light patterns and obtains the corresponding total echo intensities with a bucket detector. The correlation between the patterns and the bucket signals results in the image. Due to such a mechanism different from the traditional imaging methods, GI has received extensive attention during the past two decades. However, this mechanism also makes GI suffer from slow imaging speed and poor imaging quality. In previous work, each sample, including an illumination pattern and its detected bucket signal, was treated independently with each other. The correlation is therefore a linear superposition of the sequential data. Inspired by human's speech, where sequential words are linked with each other by a certain semantic logic and an incomplete sentence could still convey a correct meaning, we here propose a different perspective that there is potentially a non-linear connection between the sequential samples in GI. We therefore built a system based on a recurrent neural network (RNN), called GI-RNN, which enables recovering high-quality images at low sampling rates. The test with MNIST's handwriting numbers shows that, under a sampling rate of 1.28%, GI-RNN have a 12.58 dB higher than the traditional basic correlation algorithm and a 6.61 dB higher than compressed sensing algorithm in image quality. After trained with natural images, GI-RNN exhibits a strong generalization ability. Not only does GI-RNN work well with the standard images such as "cameraman", but also it can recover the natural scenes in reality at the 3% sampling rate while the SSIMs are greater than 0.7.

Single pixel imaging based on large capacity spatial multiplexing metasurface

Single pixel imaging as an alternative to traditional imaging methods, has attracted extensive attention in various research fields. Metasurfaces with subwavelength unit cells and compact footprint can be used as a substitute for traditional optical elements. In this work, we propose a single pixel imaging scheme based on metasurface composed of photon sieves, where spatial modulation is realized through shifting. Spatial multiplexing capability is demonstrated by this shifting mode, which can obtain more patterns in limited space and greatly increase the mask capacity. Benefited from the simple structure and easy manufacture of photon sieves, large capacity metasurface can be manufactured. Meanwhile, metasurfaces can simplify the single pixel imaging system, leading to the system miniaturization and integration. In addition, numerical and optical experiments prove that our proposal can operate at the range from the entire visible light to near-infrared light. Such scheme provides a new way for single pixel imaging and would be applied in microscopic imaging, dynamic imaging, hyperspectral imaging, and so on.

High-speed 3D imaging flow cytometry with optofluidic spatial transformation

Three-dimensional (3D) fluorescence imaging is important to accurately capture and understand biological structures and phenomena. However, because of its slow acquisition speed, it was difficult to implement 3D fluorescence imaging for imaging flow cytometry. Especially, modern flow cytometers operate at a flow velocity of 1-10 m/s, and no 3D fluorescence imaging technique was able to capture cells at such high velocity. Here, we present a high-speed 3D fluorescence imaging technique in which a set of optical cross sections of a cell is captured within a single frame of a camera by combining strobe light-sheet excitation and optofluidic spatial transformation. Using this technique, we demonstrated 3D fluorescence imaging of cells flowing at a velocity of over 10 m/s, which is the fastest to our knowledge. Such technology can allow integration of 3D imaging with flow systems of common flow cytometers and cell sorters.

Induced Pluripotent Stem Cell-Based Drug Screening by Use of Artificial Intelligence

Induced pluripotent stem cells (iPSCs) are terminally differentiated somatic cells that differentiate into various cell types. iPSCs are expected to be used for disease modeling and for developing novel treatments because differentiated cells from iPSCs can recapitulate the cellular pathology of patients with genetic mutations. However, a barrier to using iPSCs for comprehensive drug screening is the difficulty of evaluating their pathophysiology. Recently, the accuracy of image analysis has dramatically improved with the development of artificial intelligence (AI) technology. In the field of cell biology, it has become possible to estimate cell types and states by examining cellular morphology obtained from simple microscopic images. AI can evaluate disease-specific phenotypes of iPS-derived cells from label-free microscopic images; thus, AI can be utilized for disease-specific drug screening using iPSCs. In addition to image analysis, various AI-based methods can be applied to drug development, including phenotype prediction by analyzing genomic data and virtual screening by analyzing structural formulas and protein-protein interactions of compounds. In the future, combining AI methods may rapidly accelerate drug discovery using iPSCs. In this review, we explain the details of AI technology and the application of AI for iPSC-based drug screening.

Microfluidic Devices Controlled by Machine Learning with Failure Experiments

A critical microchannel technique is to isolate specific objects, such as cells, in a biological solution. Generally, this particle sorting is achieved by designing a microfluidic device and tuning its control values; however, unpredictable motions of the particle mixture make this approach time-consuming and labor intensive. Here, we show that microfluidic control with reinforced learning characterized by utilizing failure results can maximize the training effect with limited data. This method uses microscopic images of the separation process, including failed conditions (inappropriate flow speeds or dilution rates of biological samples), and insights for efficient learning are provided by setting gradient rewards according to the degree of failure. Once learning is performed in this manner, the optimal separating condition for other related samples can be automatically found. Failed experiments are not wasteful; they increase training data and make it easier to reach correct answers. This device control could be useful in automatic synthetic chemistry, biomedical analysis, and microfabrication robotics.