Fluorescence imaging of flowing cells using a temporally coded excitation

Neuromorphic-enabled video-activated cell sorting

Imaging flow cytometry allows image-activated cell sorting (IACS) with enhanced feature dimensions in cellular morphology, structure, and composition. However, existing IACS frameworks suffer from the challenges of 3D information loss and processing latency dilemma in real-time sorting operation. Herein, we establish a neuromorphic-enabled video-activated cell sorter (NEVACS) framework, designed to achieve high-dimensional spatiotemporal characterization content alongside high-throughput sorting of particles in wide field of view. NEVACS adopts event camera, CPU, spiking neural networks deployed on a neuromorphic chip, and achieves sorting throughput of 1000 cells/s with relatively economic hybrid hardware solution (~$10 K for control) and simple-to-make-and-use microfluidic infrastructures. Particularly, the application of NEVACS in classifying regular red blood cells and blood-disease-relevant spherocytes highlights the accuracy of using video over a single frame (i.e., average error of 0.99% vs 19.93%), indicating NEVACS' potential in cell morphology screening and disease diagnosis.

Real-time fluorescence imaging flow cytometry enabled by motion deblurring and deep learning algorithms

Fluorescence imaging flow cytometry (IFC) has been demonstrated as a crucial biomedical technique for analyzing specific cell subpopulations from heterogeneous cellular populations. However, the high-speed flow of fluorescent cells leads to motion blur in cell images, making it challenging to identify cell types from the raw images. In this study, we present a real-time single-cell imaging and classification system based on a fluorescence microscope and deep learning algorithm, which is able to directly identify cell types from motion-blur images. To obtain annotated datasets of blurred images for deep learning model training, we developed a motion deblurring algorithm for the reconstruction of blur-free images. To demonstrate the ability of this system, deblurred images of HeLa cells with various fluorescent labels and HeLa cells at different cell cycle stages were acquired. The trained ResNet achieved a high accuracy of 96.6% for single-cell classification of HeLa cells in three different mitotic stages, with a short processing time of only 2 ms. This technology provides a simple way to realize single-cell fluorescence IFC and real-time cell classification, offering significant potential in various biological and medical applications.

Light sheet based volume flow cytometry (VFC) for rapid volume reconstruction and parameter estimation on the go

Optical imaging is paramount for disease diagnosis and to access its progression over time. The proposed optical flow imaging (VFC/iLIFE) is a powerful technique that adds new capabilities (3D volume visualization, organelle-level resolution, and multi-organelle screening) to the existing system. Unlike state-of-the-art point-illumination-based biomedical imaging techniques, the sheet-based VFC technique is capable of single-shot sectional visualization, high throughput interrogation, real-time parameter estimation, and instant volume reconstruction with organelle-level resolution of live specimens. The specimen flow system was realized on a multichannel (Y-type) microfluidic chip that enables visualization of organelle distribution in several cells in-parallel at a relatively high flow-rate (2000 nl/min). The calibration of VFC system requires the study of point emitters (fluorescent beads) at physiologically relevant flow-rates (500-2000 nl/min) for determining flow-induced optical aberration in the system point spread function (PSF). Subsequently, the recorded raw images and volumes were computationally deconvolved with flow-variant PSF to reconstruct the cell volume. High throughput investigation of the mitochondrial network in HeLa cancer cell was carried out at sub-cellular resolution in real-time and critical parameters (mitochondria count and size distribution, morphology, entropy, and cell strain statistics) were determined on-the-go. These parameters determine the physiological state of cells, and the changes over-time, revealing the metastatic progression of diseases. Overall, the developed VFC system enables real-time monitoring of sub-cellular organelle organization at a high-throughput with high-content capacity.

Progressive use of multispectral imaging flow cytometry in various research areas

Multi-spectral imaging flow cytometry (MIFC) has become one of the most powerful technologies for investigating general analytics, molecular and cell biology, biotechnology, medicine, and related fields. It combines the capabilities of the morphometric and photometric analysis of single cells and micrometer-sized particles in flux with regard to thousands of events. It has become the tool of choice for a wide range of research and clinical applications. By combining the features of flow cytometry and fluorescence microscopy, it offers researchers the ability to couple the spatial resolution of multicolour images of cells and organelles with the simultaneous analysis of a large number of events in a single system. This provides the opportunity to visually confirm findings and collect novel data that would otherwise be more difficult to obtain. This has led many researchers to design innovative assays to gain new insight into important research questions. To date, it has been successfully used to study cell morphology, surface and nuclear protein co-localization, protein-protein interactions, cell signaling, cell cycle, cell death, and cytotoxicity, intracellular calcium, drug uptake, pathogen internalization, and other applications. Herein we describe some of the recent advances in the field of multiparametric imaging flow cytometry methods in various research areas.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera

SIGNIFICANCE

Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity.

AIM

We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination.

APPROACH

We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes.

RESULTS

Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels.

CONCLUSIONS

Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities.

Temporal super-resolution microscopy using a hue-encoded shutter

Limited time-resolution in microscopy is an obstacle to many biological studies. Despite recent advances in hardware, digital cameras have limited operation modes that constrain frame-rate, integration time, and color sensing patterns. In this paper, we propose an approach to extend the temporal resolution of a conventional digital color camera by leveraging a multi-color illumination source. Our method allows for the imaging of single-hue objects at an increased frame-rate by trading spectral for temporal information (while retaining the ability to measure base hue). It also allows rapid switching to standard RGB acquisition. We evaluated the feasibility and performance of our method via experiments with mobile resolution targets. We observed a time-resolution increase by a factor 2.8 with a three-fold increase in temporal sampling rate. We further illustrate the use of our method to image the beating heart of a zebrafish larva, allowing the display of color or fast grayscale images. Our method is particularly well-suited to extend the capabilities of imaging systems where the flexibility of rapidly switching between high frame rate and color imaging are necessary.

Optical Imaging Approaches to Monitor Static and Dynamic Cell-on-Chip Platforms: A Tutorial Review

Miniaturized laboratories on chip platforms play an important role in handling life sciences studies. The platforms may contain static or dynamic biological cells. Examples are a fixed medium of an organ-on-a-chip and individual cells moving in a microfluidic channel, respectively. Due to feasibility of control or investigation and ethical implications of live targets, both static and dynamic cell-on-chip platforms promise various applications in biology. To extract necessary information from the experiments, the demand for direct monitoring is rapidly increasing. Among different microscopy methods, optical imaging is a straightforward choice. Considering light interaction with biological agents, imaging signals may be generated as a result of scattering or emission effects from a sample. Thus, optical imaging techniques could be categorized into scattering-based and emission-based techniques. In this review, various optical imaging approaches used in monitoring static and dynamic platforms are introduced along with their optical systems, advantages, challenges, and applications. This review may help biologists to find a suitable imaging technique for different cell-on-chip studies and might also be useful for the people who are going to develop optical imaging systems in life sciences studies.

Field of view scanning based quantitative interferometric microscopic cytometers for cellular imaging and analysis

Microimaging is of great significance in the biological and medical fields, since it can realize observations acting as important references for cellular research and disease diagnosis. However, traditional microscopy only offers qualitative sample contours; moreover, it is difficult to reach large-amount sample observations limited by the fixed field of view (FoV). To realize massive cellular measurements quantitatively, three designed quantitative interferometric microscopic cytometers based on the FoV scanning are introduced and compared in details in this article. These devices not only retrieve the quantitative sample phase distributions in the extended FoV, but also provide the detailed information of massive cells, such as cellular volume, area, and roundness. Considering their capabilities as quantitative imaging and large-amount sampling, it is believed that these quantitative interferometric microscopic cytometers (QIMCs) can be potentially adopted in high-throughput cell imaging and statistical analysis for both the biological and medical applications.

Hadamard circular masks: high focal depth with high throughput

We present a class of binary masks that encode, in polar coordinates, the values of a Hadamard matrix of order N. For order N ≥ 2, the binary masks increase the Strehl ratio vs. focus error by the factor N, with the highest possible light throughput. Since a Strehl ratio with high tolerance to defocus does not guarantee a modulation transfer function (MTF) with low sensitivity to focus errors, then, we show that for N = 16 the binary mask reduces also the impact of focus error on the MTF. Equivalently, the discrete binary mask has Fisher information with low variations to defocus.

Multi-Image Deblurring Using Complementary Sets of Fluttering Patterns

We present a novel coded exposure video technique for multi-image motion deblurring. The key idea of this paper is to capture video frames with a set of complementary fluttering patterns, which enables us to preserve all spectrum bands of a latent image and recover a sharp latent image. To achieve this, we introduce an algorithm for generating a complementary set of binary sequences based on the modern communication theory and implement the coded exposure video system with an off-the-shelf machine vision camera. To demonstrate the effectiveness of our method, we provide in-depth analyses of the theoretical bounds and the spectral gains of our method and other state-of-the-art computational imaging approaches. We further show deblurring results on various challenging examples with quantitative and qualitative comparisons to other computational image capturing methods used for image deblurring, and show how our method can be applied for protecting privacy in videos.

Image-Based Phenotypic Screening with Human Primary T Cells Using One-Dimensional Imaging Cytometry with Self-Tuning Statistical-Gating Algorithms

The parallel microfluidic cytometer (PMC) is an imaging flow cytometer that operates on statistical analysis of low-pixel-count, one-dimensional (1D) line scans. It is highly efficient in data collection and operates on suspension cells. In this article, we present a supervised automated pipeline for the PMC that minimizes operator intervention by incorporating multivariate logistic regression for data scoring. We test the self-tuning statistical algorithms in a human primary T-cell activation assay in flow using nuclear factor of activated T cells (NFAT) translocation as a readout and readily achieve an average Z' of 0.55 and strictly standardized mean difference of 13 with standard phorbol myristate acetate/ionomycin induction. To implement the tests, we routinely load 4 µL samples and can readout 3000 to 9000 independent conditions from 15 mL of primary human blood (buffy coat fraction). We conclude that the new technology will support primary-cell protein-localization assays and "on-the-fly" data scoring at a sample throughput of more than 100,000 wells per day and that it is, in principle, consistent with a primary pharmaceutical screen.

Review: imaging technologies for flow cytometry

High-throughput single cell imaging is a critical enabling and driving technology in molecular and cellular biology, biotechnology, medicine and related areas. Imaging flow cytometry combines the single-cell imaging capabilities of microscopy with the high-throughput capabilities of conventional flow cytometry. Recent advances in imaging flow cytometry are remarkably revolutionizing single-cell analysis. This article describes recent imaging flow cytometry technologies and their challenges.

A virtually imaged defocused array (VIDA) for high-speed 3D microscopy

We report a method to capture a multifocus image stack based on recording multiple reflections generated by imaging through a custom etalon. The focus stack is collected in a single camera exposure and consequently the information needed for 3D reconstruction is recorded in the camera integration time, which is only 100 µs. We have used the VIDA microscope to temporally resolve the multi-lobed 3D morphology of neutrophil nuclei as they rotate and deform through a microfluidic constriction. In addition, we have constructed a 3D imaging flow cytometer and quantified the nuclear morphology of nearly a thousand white blood cells flowing at a velocity of 3 mm per second. The VIDA microscope is compact and simple to construct, intrinsically achromatic, and the field-of-view and stack number can be easily reconfigured without redesigning diffraction gratings and prisms.

Microfluidic microscopy-assisted label-free approach for cancer screening: automated microfluidic cytology for cancer screening

Each year, about 7-8 million deaths occur due to cancer around the world. More than half of the cancer-related deaths occur in the less-developed parts of the world. Cancer mortality rate can be reduced with early detection and subsequent treatment of the disease. In this paper, we introduce a microfluidic microscopy-based cost-effective and label-free approach for identification of cancerous cells. We outline a diagnostic framework for the same and detail an instrumentation layout. We have employed classical computer vision techniques such as 2D principal component analysis-based cell type representation followed by support vector machine-based classification. Analogous to criminal face recognition systems implemented with help of surveillance cameras, a signature-based approach for cancerous cell identification using microfluidic microscopy surveillance is demonstrated. Such a platform would facilitate affordable mass screening camps in the developing countries and therefore help decrease cancer mortality rate.

Automated quantitative cytological analysis using portable microfluidic microscopy

In this article, a portable microfluidic microscopy based approach for automated cytological investigations is presented. Inexpensive optical and electronic components have been used to construct a simple microfluidic microscopy system. In contrast to the conventional slide-based methods, the presented method employs microfluidics to enable automated sample handling and image acquisition. The approach involves the use of simple in-suspension staining and automated image acquisition to enable quantitative cytological analysis of samples. The applicability of the presented approach to research in cellular biology is shown by performing an automated cell viability assessment on a given population of yeast cells. Further, the relevance of the presented approach to clinical diagnosis and prognosis has been demonstrated by performing detection and differential assessment of malaria infection in a given sample.

Optofluidic time-stretch imaging - an emerging tool for high-throughput imaging flow cytometry

Optical imaging is arguably the most effective tool to visualize living cells with high spatiotemporal resolution and in a nearly noninvasive manner. Driven by this capability, state-of-the-art cellular assay techniques have increasingly been adopting optical imaging for classifying different cell types/stages, and thus dissecting the respective cellular functions. However, it is still a daunting task to image and characterize cell-to-cell variability within an enormous and heterogeneous population - an unmet need in single-cell analysis, which is now widely advocated in modern biology and clinical diagnostics. The challenge stems from the fact that current optical imaging technologies still lack the practical speed and sensitivity for measuring thousands to millions of cells down to the single-cell precision. Adopting the wisdom in high-speed fiber-optics communication, optical time-stretch imaging has emerged as a completely new optical imaging concept which is now proven for ultrahigh-throughput optofluidic single-cell imaging, at least 1-2 orders-of-magnitude higher (up to ∼100 000 cells per second) compared to the existing imaging flow cytometers. It also uniquely enables quantification of intrinsic biophysical markers of individual cells - a largely unexploited class of single-cell signatures that is known to be correlated with the overwhelmingly investigated biochemical markers. With the aim of reaching a wider spectrum of experts specializing in cellular assay developments and applications, this paper highlights the essential basics of optical time-stretch imaging, followed by reviewing the recent developments and applications of optofluidic time-stretch imaging. We will also discuss the current challenges of this technology, in terms of providing new insights in basic biology and enriching the clinical diagnostic toolsets.

Handheld Fluorescence Microscopy based Flow Analyzer

Fluorescence microscopy has the intrinsic advantages of favourable contrast characteristics and high degree of specificity. Consequently, it has been a mainstay in modern biological inquiry and clinical diagnostics. Despite its reliable nature, fluorescence based clinical microscopy and diagnostics is a manual, labour intensive and time consuming procedure. The article outlines a cost-effective, high throughput alternative to conventional fluorescence imaging techniques. With system level integration of custom-designed microfluidics and optics, we demonstrate fluorescence microscopy based imaging flow analyzer. Using this system we have imaged more than 2900 FITC labeled fluorescent beads per minute. This demonstrates high-throughput characteristics of our flow analyzer in comparison to conventional fluorescence microscopy. The issue of motion blur at high flow rates limits the achievable throughput in image based flow analyzers. Here we address the issue by computationally deblurring the images and show that this restores the morphological features otherwise affected by motion blur. By further optimizing concentration of the sample solution and flow speeds, along with imaging multiple channels simultaneously, the system is capable of providing throughput of about 480 beads per second.

High throughput imaging cytometer with acoustic focussing

We demonstrate an imaging flow cytometer that uses acoustic levitation to assemble cells and other particles into a sheet structure. This technique enables a high resolution, low noise CMOS camera to capture images of thousands of cells with each frame. While ultrasonic focussing has previously been demonstrated for 1D cytometry systems, extending the technology to a planar, much higher throughput format and integrating imaging is non-trivial, and represents a significant jump forward in capability, leading to diagnostic possibilities not achievable with current systems. A galvo mirror is used to track the images of the moving cells permitting exposure times of 10 ms at frame rates of 50 fps with motion blur of only a few pixels. At 80 fps, we demonstrate a throughput of 208 000 beads per second. We investigate the factors affecting motion blur and throughput, and demonstrate the system with fluorescent beads, leukaemia cells and a chondrocyte cell line. Cells require more time to reach the acoustic focus than beads, resulting in lower throughputs; however a longer device would remove this constraint.

Motion deblurring with temporally coded illumination in an LED array microscope

Motion blur, which results from time-averaging an image over the camera's exposure time, is a common problem in microscopy of moving samples. Here, we demonstrate linear motion deblurring using temporally coded illumination in an LED array microscope. By illuminating moving objects with a well-designed temporal coded sequence that varies during each single camera exposure, the resulting motion blur is invertible and can be computationally removed. This scheme is implemented in an existing LED array microscope, providing benefits of being grayscale, fast, and adaptive, which leads to high-quality deblur performance and a flexible implementation with no moving parts. The proposed method is demonstrated experimentally for fast moving targets in a microfluidic environment.

Automated cell viability assessment using a microfluidics based portable imaging flow analyzer

In this work, we report a system-level integration of portable microscopy and microfluidics for the realization of optofluidic imaging flow analyzer with a throughput of 450 cells/s. With the use of a cellphone augmented with off-the-shelf optical components and custom designed microfluidics, we demonstrate a portable optofluidic imaging flow analyzer. A multiple microfluidic channel geometry was employed to demonstrate the enhancement of throughput in the context of low frame-rate imaging systems. Using the cell-phone based digital imaging flow analyzer, we have imaged yeast cells present in a suspension. By digitally processing the recorded videos of the flow stream on the cellphone, we demonstrated an automated cell viability assessment of the yeast cell population. In addition, we also demonstrate the suitability of the system for blood cell counting.

Miniaturized, multiplexed readout of droplet-based microfluidic assays using time-domain modulation

Recent advances in microfluidics to generate and control picoliter emulsions of water in oil have enabled ultra-sensitive assays for small molecules, proteins, nucleic acids, and cells. Unfortunately, the conventional fluorescence detection used to measure the outcome of these droplet-based assays has not proven suited to match the time and space multiplexing capabilities of microfluidic systems. To address this challenge, we developed an in-flow fluorescence detection platform that enables multiple streams of droplets to be monitored using only a single photodetector and no lenses. The key innovation of our technology is the amplitude modulation of the signal from fluorescent droplets using distinct micro-patterned masks for each channel. By taking advantage of the high bandwidth of electronics, our technique enables the velocity-independent recovery of weak fluorescent signals (SNR ≪ 1) using only simple hardware, obviating the need for lasers, bulky detectors, and complex fluid control. We demonstrated a handheld-sized device that simultaneously monitors four independent channels with the capability to be scaled-up to more than sixteen, limited primarily by the droplet density.

Pose estimation using time-resolved inversion of diffuse light

We present a novel approach for evaluation of position and orientation of geometric shapes from scattered time-resolved data. Traditionally, imaging systems treat scattering as unwanted and are designed to mitigate the effects. Instead, we show here that scattering can be exploited by implementing a system based on a femtosecond laser and a streak camera. The result is accurate estimation of object pose, which is a fundamental tool in analysis of complex scenarios and plays an important role in our understanding of physical phenomena. Here, we experimentally show that for a given geometry, a single incident illumination point yields enough information for pose estimation and tracking after multiple scattering events. Our technique can be used for single-shot imaging behind walls or through turbid media.

Estimating wide-angle, spatially varying reflectance using time-resolved inversion of backscattered light

Imaging through complex media is a well-known challenge, as scattering distorts a signal and invalidates imaging equations. For coherent imaging, the input field can be reconstructed using phase conjugation or knowledge of the complex transmission matrix. However, for incoherent light, wave interference methods are limited to small viewing angles. On the other hand, time-resolved methods do not rely on signal or object phase correlations, making them suitable for reconstructing wide-angle, larger-scale objects. Previously, a time-resolved technique was demonstrated for uniformly reflecting objects. Here, we generalize the technique to reconstruct the spatially varying reflectance of shapes hidden by angle-dependent diffuse layers. The technique is a noninvasive method of imaging three-dimensional objects without relying on coherence. For a given diffuser, ultrafast measurements are used in a convex optimization program to reconstruct a wide-angle, three-dimensional reflectance function. The method has potential use for biological imaging and material characterization.

A fast fluorescence imaging flow cytometer for phytoplankton analysis

We report a fast fluorescence imaging flow cytometer for phytoplankton analysis that can achieve a volume flow rate up to 1ml/min. The instrument shows a high immunity to motion blur in image captured with a lateral resolution of 0.75 ± 0.06 μm for a wide size range ~1 μm to ~200 μm. This is made possible by suppressing the out-of-focus light using thin light sheet illumination and image deconvolution, and by precluding the motion-blur with a unique flow configuration. Preliminary results from untreated coastal water samples show the technique has high potential as a practical field instrument for monitoring phytoplankton abundance and species composition.